BiLSTM-BiDAF named entity recognition method based on machine reading comprehension

Abstract

The present invention discloses a BiLSTM‑BiDAF named entity recognition method based on machine reading comprehension. First, in order to fully explore the contextual features of the text, NEZHA is used to obtain the full-text context information, and further extract local features through BiLSTM to enhance the model's ability to capture local dependency information. Secondly, a bidirectional attention mechanism is introduced to learn the semantic association between text and entity categories. Finally, a boundary detector based on a gating mechanism is designed to strengthen the correlation between entity boundaries and predict the position of entities in the text. At the same time, by establishing an answer quantity detector, unanswered questions are identified. The present invention was experimented on the CCKS2020 Chinese electronic medical record and CMeEE datasets. The results show that the model constructed by the present invention can effectively identify named entities in text.

Description

Technical Field

[0001] This paper belongs to the field of natural language processing and named entity recognition, and proposes a BiLSTM-BiDAF named entity recognition model based on machine reading comprehension. This model can be used for named entity recognition tasks of Chinese medical texts, and can provide basic services for information extraction, question answering, machine translation, etc. Background Art

[0002] Natural language processing is an important research direction in the fields of computer science and artificial intelligence. As a key task in natural language processing, named entity recognition (NER) has been widely used in information retrieval, question-answering systems, and machine translation.

[0003] With the rapid development of neural networks, named entity recognition methods based on deep learning have gradually replaced statistical machine learning methods with complicated feature engineering and become the mainstream research method for named entity recognition tasks. Named entity recognition methods based on deep learning transform entity recognition into a sequence labeling task, and use the strong computing power of neural networks to automatically extract context features at the sentence level to achieve entity recognition. For example, Huang et al. first proposed a method combining a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM) with a conditional random field to handle named entity recognition tasks. BiLSTM can learn the context information of character dependencies in the sequence through forward and backward transmission, and to a certain extent solve the problem of long-distance dependency of text. Rei et al., based on recurrent neural networks, used the attention mechanism to dynamically utilize character vectors and word vector information to solve the problem of unknown words appearing in the text. Devlin et al. used the pre-trained language model BERT to generate word embedding encoding, and fine-tuned it in combination with specific entity recognition tasks to effectively solve the problem of insufficient context representation in named entity recognition tasks. The method of named entity recognition using sequence labeling can effectively obtain contextual information within a sentence. However, since the relevant information of the entity category is not fully utilized during the training process, the model relies heavily on feature extraction capabilities, which makes learning difficult and the recognition effect poor.

[0004] In recent years, some scholars have proposed to transform the named entity recognition task into a machine reading comprehension (MRC) task and achieved good results. In 2020, Li et al. designed entity category query questions, introduced prior semantic knowledge of entity categories, used the machine reading comprehension framework to learn the interaction between text and entity category questions, and transformed entity recognition into sentence fragment extraction, achieving a high accuracy rate in the named entity recognition task. Xue et al. followed the above reading comprehension-based named entity recognition framework, used a variety of weakly supervised data to assist model training, and integrated entity knowledge into the pre-trained language model. Compared with traditional sequence labeling methods, the above MRC-based method encodes important prior information about entity categories through questions, making it easier to distinguish similar classification labels, but it still only models at the sentence level, ignoring the semantic information between sentences, which easily causes inconsistent entity labeling in different sentences. Summary of the invention

[0005] In response to the above problems, the present invention extends sentence-level named entity recognition to text-level named entity recognition, and proposes a BiLSTM-BiDAF named entity recognition model MRC-NER based on machine reading comprehension. First, NEZHA is used to obtain the full-text context information of the text, and further captures the local dependency information through BiLSTM. Secondly, the bidirectional attention flow (BiDAF) mechanism is introduced to learn the semantic association between text and entity categories. Finally, when predicting the entity position, a boundary detector based on the gating mechanism is designed to fully consider the constraints of the entity start position on the end position, and at the same time, an answer quantity detector is established to identify unanswered questions.

[0006] The method comprises the following steps:

[0007] Step 1: Construct an entity category query statement containing semantic prior information based on the entity category to be identified in the text. Through data preprocessing, the dataset is constructed into a form that conforms to the input of the machine reading comprehension framework. Each piece of data contains text, entity category query statement and the start and end positions of the entity of this type in the text.

[0008] Step 2: Extract features from the semantic information of the text and the prior information contained in the question. Use the NEZHA pre-trained language model as the embedding layer to extract global features from the text and question, and obtain the character embedding vectors of the text and question. Use BiLSTM to further extract local features from the character embedding vectors of the text and question, respectively, to alleviate the problem that the pre-trained language model is unable to capture local dependency information.

[0009] Step 3: Use the bidirectional attention mechanism to learn the semantic association between text and entity categories. Calculate the similarity matrix between text and question, and calculate the attention representation vector of text to question and the attention representation vector of question to text based on the similarity matrix. After fully integrating the semantic prior information of text and entity category, input them into the answer prediction layer together.

[0010] Step 4: Predict the position of the entity in the text. The interactive fusion vector obtained in step 3 is used to calculate the probability of whether it is the starting position of the entity position by position using a semi-pointer and semi-annotation strategy. Considering the constraints of the entity's starting position on the ending position, a boundary detector based on a gating mechanism is designed to dynamically fuse the starting position probability distribution with the interactive fusion vector, making the model's prediction of the ending position more accurate. The two classifier results that calculate the probability of the entity's starting and ending positions are matched closely by index order to mark the specific position of the entity.

[0011] Step 5: In order to reduce the model's extraction errors when the text does not contain certain entity categories, an answer quantity detector is established to identify unanswered questions. The weighted text representation is calculated using the predicted entity start and end position probabilities and the interaction fusion vector, and then concatenated with the [CLS] special tag in the pre-trained language model to train a classifier to effectively identify unanswered questions.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] 1. The present invention uses NEZHA as an embedding layer to perform global feature extraction on text and questions, and obtains character embedding vectors for text and questions. Not only can it effectively solve the problem of polysemy, but it also allows the character embedding vector to contain rich contextual information and entity boundary information, enhancing the model's extraction effect on the global semantics of the text. The present invention further uses BiLSTM to encode the text and question character embedding vectors, effectively solving the problem of insufficient ability of the pre-trained language model to capture local dependency information, by assigning greater weights to neighboring words, improving the model's ability to capture local features of the text, further giving each word richer contextual information, and enhancing the model's ability to represent the semantics of the text.

[0014] 2. Introduce a bidirectional attention mechanism to learn the semantic association between text and entity categories. By paying attention to the text on the question, the text and the prior information contained in the question are fully integrated. By paying attention to the question on the text, the question focuses on the key information related to the category in the text.

[0015] 3. By designing a boundary detector based on a gating mechanism, the temporal and logical relationships between entity boundaries are fully considered, making the model more accurate in predicting entity locations and improving the accuracy of named entity recognition. At the same time, an answer quantity detector is established to identify unanswered questions, effectively solving the problem of text not containing certain types of entities. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the overall structure of the BiLSTM-BiDAF named entity recognition model based on machine reading comprehension.

[0017] Figure 2 This is the network structure diagram of the embedding layer of the language model pre-trained using NEZHA.

[0018] Figure 3 This is the network structure diagram of BiLSTM.

[0019] Figure 4 Schematic diagram of the bidirectional attention mechanism.

[0020] Figure 5 The network structure diagram of the answer boundary detector based on the gating mechanism.

[0021] Figure 6 This is the overall network structure diagram of the answer prediction layer.

[0022] Figure 7 This is the network structure diagram of the answer number detector.

[0023] Figure 8 The data processing flow chart is shown in Figure 2. DETAILED DESCRIPTION

[0024] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0025] The text contains rich contextual information. In order to learn the semantic associations between sentences and characters in the text and establish a deep interactive relationship between text and entity categories, the present invention uses a machine reading comprehension framework to model and solve the named entity recognition task. The so-called machine reading comprehension framework can be specifically divided into three steps: given context, asking relevant questions, and marking the answer location. Based on this framework, the present invention first designs an entity category query sentence containing semantic prior information according to the entity category to be identified, and then builds a model to encode the semantic information of the text and the prior information in the question, learns the complex interactive relationship between the text and the entity category query sentence, and finally finds the location of the entity in the text.

[0026] To achieve the above, the present invention constructs Figure 1The BiLSTM-BiDAF named entity recognition model MRC-NER based on machine reading comprehension shown in the figure is divided into four layers: 1) Embedding layer, which uses the pre-trained language model NEZHA to obtain global semantic information and obtain character embedding representations of text and questions; 2) Encoding layer, which uses BiLSTM to encode local features to solve the problem that the pre-trained language model is insufficient in capturing local dependency information; 3) Interaction layer, which introduces a bidirectional attention mechanism to fuse the interactive information between text and entity categories; 4) Answer prediction layer, which predicts the start and end positions of the entity in the text.

[0027] 1. Embedding layer

[0028] Step 1: The present invention converts the named entity recognition task into a machine reading comprehension problem. According to the entity type to be recognized in the text, a query statement containing semantic prior information is constructed, and an input form that conforms to the machine reading comprehension framework is constructed through data preprocessing. In order to include more semantic prior knowledge in the entity category query statement, the present invention integrates relevant descriptions of the entity type and some simple examples when constructing the entity category query statement. Taking the CCKS2020 dataset as an example, Table 1 shows the related issues of different entity types constructed by the present invention for this dataset.

[0029] Table 1 Question construction table

[0030]

[0031] Step 2: Use the NEZHA pre-trained language model as the embedding layer to perform global feature extraction on the semantic information of the text and the prior information contained in the question.

[0032] In order to learn the intrinsic correlation between different sentences in the text and enhance the model's ability to represent text contextual semantic information, the present invention uses the NEZHA pre-trained language model as an embedding layer to perform global feature extraction on text and questions, and obtain character embedding vectors for text and questions. NEZHA is an open source pre-trained language model from Huawei Noah Labs. It is an improved model based on BERT. Since fully functional relative position encoding and full-word masking technology are used in the pre-training stage, using NEZHA as an embedding layer can make the character embedding vector contain rich contextual information and entity boundary information, enhancing the model's extraction effect on the global semantics of the text.

[0033] The present invention uses Figure 2 The NEZHA pre-trained language model shown is used as an embedding layer to perform character embedding encoding on text and questions. Let X = {c1, c2, ..., c n} is a given piece of text, n is the length of the text, Q y =(q1, q2, ..., q m} is an entity category query statement constructed based on the text content, where Y is the set of entity categories, Q y represents a predefined question query y∈Y, q i represents the i-th character, and m is the length of the question. The query Q constructed from the text for entity recognition y ={q1, q2, ..., q m} and text X = {c1, c2, ..., c n} as input, using specific classification tags [CLS] and separation tags [SEP], the question and text are concatenated in sequence to form the following input sequence:

[0034] I={[CLS];Q;[SEP];X;[SEP]}

[0035] The final text vector representation X = {c1, c2, ..., c n}, the problem vector is Q = {q1, q2, ..., q m In case the text length exceeds the maximum input length limit of the pre-trained language model, the present invention uses the dynamic programming solution long text segmentation algorithm proposed by Cai et al. to segment the text to meet the length limit of the model.

[0036] 2. Coding layer

[0037] Step 3: Use BiLSTM to extract local features from the character embedding vectors of the text and question respectively to alleviate the problem that the pre-trained language model is unable to capture local dependency information.

[0038] By using the embedding layer, the model can obtain the global semantic information of the text, and for the named entity recognition task, it is also very important to consider the influence of the sequence before and after the character on the current character. In order to effectively obtain the contextual information that the character depends on in the sentence, the present invention further uses a bidirectional long short-term memory network (BiLSTM) to encode the text and question character embedding vectors respectively, and by assigning greater weights to neighboring words, the model's ability to capture local features of the text is improved, and each word is further given richer contextual information.

[0039] The basic structure of the BiLSTM network used in the present invention is as follows: Figure 3 As shown. The text vector X and question vector Q obtained in step 2 are respectively input into the BiLSTM network for feature extraction to obtain the encoded text vector H∈R 2d×n , the problem vector U∈R 2d×m The network output corresponding to the tth word in the text is expressed as follows:

[0040]

[0041]

[0042]

[0043] in, Represents x t The hidden layer output of the corresponding forward LSTM unit; Represents x t The corresponding hidden layer output of the backward LSTM unit; h t Represents x t The output of the corresponding bidirectional LSTM network is given by Vector Sum Vector mosaic.

[0044] Using BiLSTM can alleviate the problem of the pre-trained language model's insufficient ability to capture local dependency information. Through the two parts of the embedding layer and the encoding layer, the model can not only extract the global semantic information of the text, but also fully capture the local dependency information of the text, thereby improving the model's ability to represent the semantics of the text.

[0045] Three: Interaction Layer

[0046] Step 4: Use the bidirectional attention mechanism to learn the semantic association between text and entity categories. Calculate the similarity matrix between text and question, and calculate the attention representation vector of text to question and the attention representation vector of question to text based on the similarity matrix. After fully integrating the semantic prior information of text and entity category, input them into the answer prediction layer together.

[0047] Entity category query statements contain rich entity prior information. In order to fully learn the semantic association between text and entity categories, the present invention introduces a bidirectional attention mechanism. Through the attention of the text to the question, the prior information contained in the text and the question is fully integrated. Through the attention of the question to the text, the question focuses on the key information related to the category in the text.

[0048] First, the text and question output vector H∈R obtained by the encoding layer 2d×n and U∈R 2d×m Use the trilinear function to calculate the similarity matrix S between the text and the question:

[0049]

[0050] S=α(H ：n , U ：m )∈R n×m

[0051] Among them, α is a trainable scalar function, is the element-wise multiplication, ";" is the vector row concatenation, W s is a trainable weight vector. ：n is the nth column vector of the text H, U ：m is the mth column vector of the question encoding matrix U, S nm is the similarity between the nth word in text H and the mth word in question U.

[0052] Then, the attention representation vector of the text to the question is calculated based on the similarity matrix, and each word vector in the text is represented by the weighted sum of all words in the question. Each row of the similarity matrix S is normalized and then the attention weighted calculation is performed with the question. The calculation process is as follows:

[0053] α n =softmax(S n： )

[0054]

[0055] Among them, S n： Represents the vector of the nth row in the similarity matrix.

[0056] At the same time, the attention representation vector of the question to the text is calculated based on the similarity matrix. In order to better answer the question and extract the key information of the entity in the text, it is not necessary to calculate the attention of the question to each word in the text, but only to find the words in the text that are most relevant to the question. Therefore, the weight of the question with the greatest relevance to the text is selected to form a new column vector, and the weighted fusion vector for the answer question is calculated, and it is copied n times to obtain to keep the dimensions consistent.

[0057] b = softmax(max row (S)

[0058]

[0059] Among them, max row (·) means taking the maximum value for each row of the matrix.

[0060] Finally, the text representation vector H obtained by the encoding layer is combined with the text-question attention representation vector Question-Text Attention Representation Vector The three are fused and input into the answer prediction layer together. The fusion method is as follows:

[0061]

[0062]

[0063] 4. Answer prediction layer

[0064] Step 5: Predict the position of the entity in the text. The starting position probability distribution predicted by the dynamic fusion model and the fusion vector obtained by the interaction layer fully consider the temporal relationship and logical relationship between the start and end positions of the entity, making the model's prediction of the end position more accurate.

[0065] First, the interaction vector G obtained in step 4 is used to calculate the probability distribution of whether it is the starting position of the answer position position by position using the half-pointer and half-marking strategy:

[0066] P start =softmax(W start ·G+b start )∈R 2×n

[0067] Then, a gating mechanism-based boundary detector is designed to control the probability distribution of the starting position and the interaction fusion vector G∈R 8d×n The two are organically combined, and then the probability distribution of each character being the end position is calculated position by position:

[0068] g=σ(W1·P start +b1)

[0069]

[0070] P end =softmax(W end ·X gated +b end )∈R 2×n

[0071] Among them, W1, W3∈R d×2 , W2∈R d×8d , W end ∈R 2×d , g is the control gate, which is calculated by using the sigmoid activation function to predict the starting position in the answer prediction layer, thereby controlling the degree of fusion between the starting position and the interaction vector G, X gated represents a vector that incorporates the information of the starting position of the answer. When the value of g is 0, X gated It is only related to the output vector G of the interaction layer.

[0072] Finally, the results of the two classifiers at the start and end positions of the answer are matched closely by index order to mark the specific location of the entity.

[0073] Step 6: In order to reduce the model’s extraction errors when the text does not contain certain entity categories, an answer quantity detector is established to identify questions without answers.

[0074] Existing named entity recognition models based on machine reading comprehension often assume that for an input entity category query statement, the corresponding type of entity can be extracted from the text. Therefore, for some entity types, even if there is no corresponding entity in the text, the model will still extract the wrong answer from the text. To address this problem, the present invention establishes the following Figure 7 The answer quantity detector shown uses the probability distribution of the start and end positions of the answers to calculate the weighted text representation and concatenates it with the [CLS] contextual representation vector obtained in step 2 to identify unanswerable questions:

[0075] R 1 =G·P start＝1

[0076] R 2 =G·P end＝1

[0077] z=W z ·[R 1 ; R 2 ]+b z

[0078] P answer =sigmoid(W a ·[H [cls] ; z]+b a )

[0079] Since the [CLS] tag encodes the global information of the sequence and integrates all the semantic information of the text and the question, and the predicted start and end positions contain relevant information about the number of answers, by effectively integrating the three parts and training the classifier, unanswered questions can be effectively identified.

[0080] Step 7: In order to make the model more suitable for the real results and reduce the gap between the model and the real results, the present invention designs the following loss function to enhance the accuracy of entity classification:

[0081] L start =CE(P start , Y start )

[0082] L end =CE(P end , Y end )

[0083] L answer =CE(P answer , Y answer )

[0084] L=α·L answer +β·L start +γ·L endα,β,γ∈[0,1]

[0085] The overall loss function of the model training is composed of the start and end position loss function and the answer number detector loss function. All three parts use the cross entropy loss function and are adjusted by three hyperparameters: α, β, and γ. By reducing the loss function of the model, the model can extract entities when there are entities of the corresponding type, and detect that there is no matching answer for the question of the entity category when there is no entity of the corresponding type.

[0086] Experimental equipment and required environment

[0087] The experiment of the present invention is based on Python language and its third-party library. The deep learning environment adopts the open source Pytorch computing framework and uses GPU for acceleration. The specific environment configuration is shown in Table 2.

[0088] Table 2 Experimental environment configuration

[0089]

[0090] Preprocessing and parameter setting

[0091] The data set of the experiment of the present invention comes from the evaluation task of CCKS2020. The officially provided annotated training data totals 1050 texts. Due to the relatively small number of data sets, the present invention integrates the relevant data in the CCKS2019 data set on the basis of the CCKS2020 data set. After converting the original data set into the MRC input form, a total of 9522 question-answering data sets are obtained. In order to better grasp the data set for model modeling, Tables 3 and 4 show the specific statistical data of the CCKS2020 data set used in the present invention.

[0092] Table 3 Statistics of CCKS2020 dataset

[0093]

[0094] Table 4 Statistics of whether or not there are answers in the MRC form CCKS2020 dataset

[0095]

[0096] The present invention transforms the named entity recognition task into a machine reading comprehension problem. According to the entity type to be recognized in the text, an entity query statement containing semantic prior information is constructed, and the data set is constructed into a form that conforms to the machine reading comprehension framework input through data preprocessing. The specific data processing flow is as follows: Figure 8 shown.

[0097] In the experiment, the Chinese NEZHA pre-trained language model was used for character embedding encoding. Compared with the BERT model, NEZHA used relative position encoding and introduced full-word masking technology, which learned more semantic information and boundary information than the BERT model. The present invention sets the dimension of the word vector to 768, the maximum length of the pre-trained language model to 128, the hidden layer dimension of the bidirectional long short-term memory network to 256, the dropout to 0.3, the warm up to 0.1, and the batch size to 128. The above parameters are set for the CCKS2020 corpus, and fine-tuning is required for the data of other corpora.

[0098] Experimental settings and evaluation indicators

[0099] The present invention uses precision, recall, and F1-measure as the evaluation criteria of the model to comprehensively evaluate the entity recognition results of medical texts, and the calculation is as follows:

[0100] 1) Accuracy:

[0101] 2) Recall:

[0102] 3) F1-measure:

[0103] Among them, TP represents the number of entities predicted as positive samples, TN represents the number of entities predicted as negative samples, FN represents the number of entities predicted as negative samples, and FP represents the number of entities predicted as positive samples.

[0104] Experimental results analysis

[0105] In order to verify the effect of the model proposed in this paper on the text-level named entity recognition task, NEZHA-CRF, NEZHA-IDCNN-CRF, BERT-BiLSTM-CRF, NEZHA-BiLSTM-CRF, NEZHA-MRC-baseline and the model MRC-NER proposed in this paper are compared on the CCKS2020 Chinese electronic medical record dataset. The basic information of each model is as follows:

[0106] (1) NEZHA-CRF: A model that uses the NEZHA pre-trained language model to extract features and combines CRF to perform sequence labeling on character vectors.

[0107] (2) NEZHA-IDCNN-CRF: A model that uses the NEZHA pre-trained language model and iterative dilated convolutional neural networks (IDCNN) to extract features and combines CRF to perform sequence labeling on the input character vector.

[0108] (3) BERT-BiLSTM-CRF: A model that uses the BERT pre-trained language model and BiLSTM to extract contextual features and combines CRF to perform sequence labeling on the input character vector.

[0109] (4) NEZHA-BiLSTM-CRF: A model that uses the NEZHA pre-trained language model and BiLSTM to extract contextual features and combines CRF to perform sequence labeling on the input character vector.

[0110] (5) NEZHA-MRC-baseline: The NEZHA pre-trained language model is used to extract features and combined with the model constructed by Li et al. to transform the named entity recognition task into a machine reading comprehension problem.

[0111] It can be seen from Table 5 that the MRC-NER (ours) method proposed in this invention is superior to other algorithms in terms of accuracy, recall and F1 value. From the analysis of the results, it can be obtained that:

[0112] (1) The NEZHA-BiLSTM-CRF algorithm, which uses the NEZHA pre-trained language model for global feature extraction, is better than the BERT-BiLSTM-CRF algorithm, indicating that the NEZHA pre-trained language model using relative position encoding and full-word masking technology has certain advantages in word vector encoding and can better learn the intrinsic correlation between different sentences in the text.

[0113] (2) NEZHA-BiLSTM-CRF, which uses BiLSTM for local feature extraction, is better than NEZHA-CRF and NEZHA-IDCNN-CRF algorithms. This is because compared with IDCNN, BiLSTM can capture longer-distance semantic dependencies in both directions and better extract the internal features of the text.

[0114] (3) By comparing the MRC-NER model with the NEZHA-MRC-baseline model proposed by Li et al., it can be found that the model constructed by the present invention can effectively capture the contextual information of the full text and has obvious advantages in processing text-level named entity recognition tasks.

[0115] (4) Comparing the MRC-NER model with the other four sequence labeling models, the results show that the entity recognition effect is enhanced because the prior knowledge of entity categories is encoded in the question using the machine reading comprehension framework.

[0116] Table 5 Evaluation results of CCKS2020 dataset

[0117]

[0118] In addition, in order to more clearly verify the different effects of various parts of the method of the present invention on the results, the present invention conducted the following ablation experiment, and the results are shown in Table 6:

[0119] (1) NEZHA-MRC-BiLSTM is based on the baseline model and only adds BiLSTM. By assigning larger weights to neighboring words, it alleviates the problem of the pre-trained language model's insufficient ability to capture local dependency information, thereby better capturing the local dependency information of the text, and the F1 value is improved by 2.25%.

[0120] (2) NEZHA-MRC-BiDAF introduces a bidirectional attention mechanism based on the baseline model, which enables the model to learn the interactive relationship between text and questions containing prior semantic information of entity categories, extract information that is more critical to entity recognition, and improve the F1 value by 0.07%.

[0121] (3) NEZHA-MRC-BiLSTM-BiDAF adds BiLSTM and bidirectional attention mechanism to the baseline model. Compared with the baseline model, the F1 value is improved by 2.77%.

[0122] (4) NEZHA-answer-border uses a gated boundary detector and an answer quantity detector based on the baseline model, and the F1 value is improved by 2.5%, indicating that strengthening the logical relationship between boundaries and identifying unanswered questions when predicting entities can effectively improve the named entity recognition effect.

[0123] Table 6 Evaluation results of ablation experiments on the CCKS2020 dataset

[0124]

[0125] At the same time, the present invention conducted a comparative experiment on the MRC-NER model proposed in the present invention and the baseline model NEZHA-MRC-baseline on the CMeEE Chinese Medical Named Entity Recognition dataset. As shown in Table 7, compared with the baseline model, the model proposed in the present invention has improved in all three evaluation indicators, thus verifying the effectiveness of the model proposed in the present invention on different datasets.

[0126] Table 7 Evaluation results of CMeEE dataset

[0127]

Claims

1. The BiLSTM-BiDAF named entity recognition method based on machine reading comprehension is characterized by: The method comprises the following steps: Step 1: Construct an entity category query statement containing semantic prior information based on the entity category to be identified in the text. Through data preprocessing, construct the data set into a form that conforms to the input of the machine reading comprehension framework. Each data piece contains the text, the entity category query statement and the start and end positions of the entity of the category in the text. Step 2: Extract features from the semantic information of the text and the prior information contained in the question; use the NEZHA pre-trained language model as the embedding layer to extract global features from the text and question to obtain the character embedding vectors of the text and question; and further extract local features from the character embedding vectors of the text and question through BiLSTM; Step 3: Use the bidirectional attention mechanism to learn the semantic association between text and entity categories; calculate the similarity matrix between text and question, and calculate the attention representation vector of text to question and the attention representation vector of question to text based on the similarity matrix, and fully integrate the semantic prior information of text and entity category and input them into the answer prediction layer together; Step 4: Predict the position of the entity in the text; use the semi-pointer and semi-annotation strategy to calculate the probability of whether the interactive fusion vector obtained in step 3 is the starting position of the entity position by position; considering the constraint of the entity starting position on the ending position, design a boundary detector based on the gating mechanism, dynamically fuse the starting position probability distribution and the interactive fusion vector, so that the model predicts the ending position more accurately; match the two classifier results that calculate the probability of the starting and ending positions of the entity in the nearest order through the index order, so as to mark the specific position of the entity; Step 5: In order to reduce the model’s extraction errors when the text does not contain certain entity categories, an answer quantity detector is established to identify unanswered questions. The weighted text representation is calculated using the predicted entity start and end position probabilities and the interaction fusion vector, and is concatenated with the [CLS] special tag in the pre-trained language model to train a classifier to effectively identify unanswered questions. In step 2, the NEZHA pre-trained language model is used as an embedding layer to perform global feature extraction on the semantic information of the text and the prior information contained in the question. In order to learn the intrinsic correlation between different sentences in the text and enhance the model's ability to represent the contextual semantic information of the text, the NEZHA pre-trained language model is used as an embedding layer to perform global feature extraction on the text and question, and obtain the character embedding vectors of the text and question. NEZHA pre-trained language model is used as an embedding layer to perform character embedding encoding on text and questions; let X = {c1, c2, …, c n } is a given piece of text, n is the length of the text, Q y ={q1,q2,…,q m } is an entity category query statement constructed based on the text content, where Y is the set of entity categories, Q y represents a predefined question query y∈Y, q i represents the i-th character, m is the length of the question; the query statement Q constructed from the text for entity recognition y ={q1, q2, ..., q m } and text X = {c1, c2, ..., c n } as input, using specific classification tags [CLS] and separation tags [SEP], the question and text are concatenated in sequence to form the following input sequence: I={[CLS];Q;[SEP];X;[SEP]} The final text vector representation X = {c1, c2, ..., c n }, the problem vector is Q = {q1, q2, ..., q m }.

2. The BiLSTM-BiDAF named entity recognition method based on machine reading comprehension according to claim 1, characterized in that: In step 1, the named entity recognition task is transformed into a machine reading comprehension problem; a query statement containing semantic prior information is constructed according to the entity type to be recognized in the text, and an input form that conforms to the machine reading comprehension framework is constructed through data preprocessing; When constructing entity category query statements, relevant descriptions and simple examples of entity types are integrated.

3. The BiLSTM-BiDAF named entity recognition method based on machine reading comprehension according to claim 1, characterized in that: In step 4, a bidirectional attention mechanism is used to learn the semantic association between text and entity categories; the similarity matrix between text and question is calculated, and the attention representation vector of text to question and the attention representation vector of question to text are calculated based on the similarity matrix, and the semantic prior information of text and entity category is fully integrated and input into the answer prediction layer together; Entity category query statements contain rich entity prior information. In order to fully learn the semantic association between text and entity categories, a bidirectional attention mechanism is introduced. By paying attention to the text on the question, the prior information contained in the text and the question is fully integrated. By paying attention to the question on the text, the question focuses on the key information related to the category in the text.

4. The BiLSTM-BiDAF named entity recognition method based on machine reading comprehension according to claim 1, characterized in that: In step 5, the position of the entity in the text is predicted; the starting position probability distribution predicted by the dynamic fusion model and the fusion vector obtained by the interaction layer fully consider the temporal relationship and logical relationship between the starting and ending positions of the entity, making the model's prediction of the ending position more accurate; First, the interaction vector G obtained in step 4 is used to calculate the probability distribution of whether it is the starting position of the answer position position by position using a half-pointer and half-labeling strategy; then, a boundary detector based on a gating mechanism is designed to control the organic combination of the starting position probability distribution and the interaction fusion vector, and then the probability distribution of each character as the ending position is calculated position by position, which is calculated by using the sigmoid activation function to calculate the prediction result of the starting position in the answer prediction layer, thereby controlling the degree of fusion between the starting position and the interaction vector G; the results of the two classifiers of the answer start and end positions are matched nearest to each other through the index order, so as to mark the specific position of the entity.

5. The BiLSTM-BiDAF named entity recognition method based on machine reading comprehension according to claim 1, characterized in that: In order to reduce the extraction error of the model when the text does not contain certain entity categories, an answer number detector is established to identify questions without answers. An answer number detector is established to calculate the weighted text representation using the probability distribution of the start and end positions of the answers, and concatenate it with the [CLS] contextual representation vector obtained in step 2 to identify questions that cannot be answered.

Images