Named entity recognition method based on conditional random field and deep learning

By combining conditional random field and deep learning in naming entity recognition in the medical field, integrating word categories, boundaries and dictionary features, and using self-built dictionary correction models, the problems of lack of labeling corpus and blurred boundaries in naming entity recognition in the medical field are solved. , achieving higher recognition accuracy.

CN119067116APending Publication Date: 2024-12-03SICHUAN UNIV +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310629638.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing naming entity recognition methods have problems in the medical field with lack of labeling corpus, long entity length and blurred boundaries, resulting in insufficient recognition accuracy.

Method used

Using a named entity recognition method based on conditional random field and deep learning, the word vector representation of word class features, entity boundary features and dictionary features is fused, combined with a two-way long and short-term memory network and a conditional random field model, and added to the self-built entity dictionary in medical field Correction of the model, using the method of labeling and dictionary correction.

Benefits of technology

It significantly improves the accuracy of naming entity recognition and improves the recognition performance of the model in the medical field.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004257992420000021
    Figure BDA0004257992420000021
  • Figure BDA0004257992420000036
    Figure BDA0004257992420000036
  • Figure BDA0004257992420000037
    Figure BDA0004257992420000037
Patent Text Reader

Abstract

The invention discloses a named entity recognition method based on a conditional random field and deep learning. Aiming at a named entity recognition task in the medical field, the named entity recognition method based on the conditional random field and the deep learning is provided, firstly, word class features, entity boundary features and dictionary features are fused into original word vector representation, so that the original word vector representation is richer; then realizing the research on the dependency between the tags by combining BiLSTM and CRF models; and meanwhile, a self-built medical field entity dictionary correction model is added, and a method of annotation and dictionary correction is adopted, so that the entity recognition accuracy is improved. The method has a very wide application prospect in the fields of information retrieval, business intelligence, public opinion monitoring and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention designs a named entity recognition method based on conditional random fields and deep learning, and relates to the technical field of natural language processing. Background Art

[0002] Named entity recognition is a basic task in the field of information extraction, which aims to extract entities from unstructured text and obtain entities with specific meanings. As one of the most important and popular research topics in the field of natural language processing (NLP), named entity recognition has been widely used in various application fields in recent years.

[0003] In the early days of named entity recognition, the most widely used method was the rule-based method. This method relies heavily on domain experts to write corresponding rules or design domain dictionaries from different perspectives such as vocabulary, syntax and semantics based on prior knowledge, and then perform entity recognition in text analysis based on rule matching. This type of system often has the characteristics of high accuracy and low recall rate because of its special domain rules and incomplete dictionary, but poor transferability.

[0004] Among the unsupervised learning methods, the most typical method is clustering. It can improve system performance by automatically determining the similarity between a word or phrase. The named entity recognition system based on clustering method extracts according to contextual similarity. Its core idea is to determine whether a named entity has been mentioned by using available vocabulary resources, vocabulary patterns and statistical data. In the machine learning-based method, after giving the labeled data samples, entity recognition can be regarded as a sequence labeling problem, and its features are carefully designed to indicate each training example. Then the model is learned using a machine learning algorithm. Based on these features, many machine learning algorithms have been applied to entity recognition, including: hidden Markov model, decision tree, maximum entropy model, support vector machine, etc. The machine learning-based method is to learn the features of the labeled training data fed into the model. Greenberg et al. proposed a feature induction method using conditional random fields (CRF). The named entity recognition method based on conditional random fields has been widely used in many aspects such as biomedical text, Twitter text and chemical text. Summary of the invention

[0005] Aiming at the named entity recognition task in the medical field, the present invention proposes a named entity recognition method based on conditional random fields and deep learning. The present invention firstly integrates word class features, entity boundary features and dictionary features into the original word vector representation to enrich the original word vector representation; then combines the bidirectional long short-term memory network (BiLSTM) with the conditional random field model to realize the study of the dependency between labels; at the same time, a self-built medical field entity dictionary correction model is added, and the method of labeling plus dictionary correction is adopted to improve the accuracy of entity recognition.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] The named entity recognition method based on conditional random fields and deep learning includes the following steps:

[0008] Step 1: First, preprocess the input text, split the input sentence into individual characters, then vectorize it, and then add word class features, entity boundary features, and dictionary features. The final vector representation is obtained by concatenating these features with the word vector;

[0009] Step 2: Use a bidirectional long short-term memory network to learn the final vector representation obtained in step 1, automatically extract the eigenvalues ​​corresponding to each character sequence, calculate the score of the classification label and output it, and then send it to the conditional random field layer to obtain the final optimal prediction sequence;

[0010] Step 3: Based on the self-built domain entity dictionary, further identify some entities missed by the model by correcting the text that has been fully annotated in step 2.

[0011] The role of the word vector feature in step 1 is as follows:

[0012] By adding word class features, entity boundary features, and dictionary features to the word vector representation, we can fully utilize syntactic information to enrich the representation of word vectors and help the model improve recognition task performance.

[0013] (1) Word class features: Word class features refer to the classification of words into verbs, adverbs, and adjectives based on their characteristics. When performing entity recognition tasks, they can help better determine the boundaries of entities. For example, in the sentence "do an imaging examination", by determining in advance that "do" and "check" are verbs, the left and right boundaries of the entity "imaging" can be determined.

[0014] (2) Entity boundary features: They are crucial for determining the position information of characters in each entity and are also an important basis for entity boundary recognition. The word boundary features of the input sequence use the BIO annotation mode, where B, I, and O represent the beginning of the entity, the rest of the entity, and the non-entity sequence, respectively. The boundary features and type encoding of the entity are shown in Table 1.

[0015] Table 1 Boundary features and entity type coding sets

[0016]

[0017] (3) Domain dictionary features: This chapter manually builds a medical domain dictionary. If an entity (or part of it) in the input sentence appears in the dictionary, it is marked using the BIO annotation mode and then combined with the dictionary type label to obtain the final dictionary features.

[0018] The role of the BiLSTM-CRF layer in step 2 is as follows:

[0019] The final vector sequence of the previous step is represented by X=(x1,x2,···,x n ) as the input of the BiLSTM layer, and the i-th word vector in the sequence is combined with and To splice, and Represents forward output and reverse output respectively. Represents the complete vector representation of character i, m is the number of hidden layer units, and then connected with the linear layer, the state of the hidden layer is mapped to l dimensions, l is the number of labels in the annotation set, and the automatically extracted features are recorded as matrix P. In the training stage, the eigenvalues ​​corresponding to each string on each node are calculated and correspond to the category to which the string belongs. P ij Represents x i The score of the jth label is classified and then connected to the Softmax layer to achieve l classification for each position. The BiLSTM layer outputs the final label for each word, but only uses the layer to mark each position and cannot use the labeled information. Therefore, a CRF layer is connected to constrain correction to obtain the final prediction sequence. The parameters of the CRF layer are represented by the transfer matrix A, and the probability of label i transferring to label j is recorded as A ij , and use the Viterbi algorithm for decoding. Thus, the best labeling sequence of X can be obtained as Y = (y1, y2, ... y n ) has a probability of .

[0020]

[0021]

[0022] The role of the dictionary correction layer in step 3 is as follows:

[0023] After using the BiLSTM-CRF model for entity recognition, some entities are still missed by the model. Therefore, we manually collected hospital electronic medical record lexicon, diagnostic lexicon (such as ICD-10 disease codes, SNOMEDCT, international standard surgical codes, etc.) from the Sogou lexicon and extracted entities from the training corpus to establish an entity dictionary in the medical field, which mainly covers five types of entities: body parts, examinations and tests, symptoms and signs, diseases and diagnosis and treatment. The source and size of the entity dictionary are shown in Table 2 below. In the dictionary, one entry corresponds to one entity. Table 3 shows the types and examples of characteristic words in the medical dictionary.

[0024] Table 2 Entity dictionary sources and sizes

[0025]

[0026] Table 3 Examples of feature dictionaries for medical terminology

[0027]

[0028]

[0029] According to the self-built dictionary, the results of the model annotation are corrected. The specific steps of the correction are as follows:

[0030] (1) Take out the string S consisting of n characters before the annotation statement n ;

[0031] (2) If the string S n If the length is 1, delete the leftmost character in the labeled statement X and return to step (1); otherwise, proceed to step (3);

[0032] (3) Traverse each entity i in the dictionary and determine whether entity i and string S n whether they are equal;

[0033] (4) If entity i and string S n If they are equal, then the string S n is the entity represented by i, then delete S from the left side of the annotation statement X n , and S n The type marked as i returns to step (1);

[0034] (5) If entity i and string S n If not, delete the leftmost character of the labeled statement X and return to step (2). BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is the architecture diagram of the named entity recognition model based on conditional random fields and deep learning proposed in the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the accompanying drawings:

[0037] Figure 1 This is the architecture diagram of the named entity recognition model based on conditional random fields and deep learning proposed by the present invention. In view of the shortcomings of traditional medical entity recognition technology, such as the lack of annotated corpus, the long entity length and the blurred boundary, for the input sentence text, we first merge the word class features, entity boundary features and dictionary features into the original word vector representation to make the original word vector representation richer; then combine the BiLSTM and CRF models to realize the study of the dependency between labels; at the same time, add a self-built medical field entity dictionary correction model, and use the method of annotation plus dictionary correction to improve the accuracy of entity recognition.

[0038] The present invention is tested on the self-built data set MEE to fully verify the effectiveness and reliability of the present invention. In order to verify the effectiveness of the named entity recognition method based on conditional random fields and deep learning, the method described in the present invention is compared with models such as BiLSTM, CRF, BERT-Softmax, BiLSTM-CRF, BERT-CRF, and the experimental results shown in Tables 4 and 5 are obtained. It can be concluded from the experimental results that the effect of using the method described in this patent to perform named entity recognition task experiments is significantly improved. The detailed experimental results are shown in Tables 4 and 5.

[0039] Table 4 Experimental results

[0040]

[0041] Table 5 shows the results of comparative experiments

[0042]

[0043] Table 6 shows the change in the named entity recognition performance of the model on the self-built dataset MEE before and after the word vector features and dictionary correction layer are introduced. From the experimental results, it can be seen that after the word vector features and dictionary correction layer are introduced, the performance of the model is significantly improved.

[0044] Table 6 Performance changes of the model before and after the introduction of word vector features and dictionary correction layer in the present invention

[0045]

Claims

1. A method for named entity recognition based on conditional random fields and deep learning, characterized in that The following steps are involved: Step 1: First, preprocess the input text, split the input sentence into individual characters, then vectorize it, and then add word class features, entity boundary features, and dictionary features. The final vector representation is obtained by concatenating these features with the word vector; Step 2: Use a bidirectional long short-term memory network to learn the final vector representation obtained in step 1, automatically extract the eigenvalues ​​corresponding to each character sequence, calculate the score of the classification label and output it, and then send it to the conditional random field layer to obtain the final optimal prediction sequence; Step 3: Based on the self-built domain entity dictionary, further identify some entities missed by the model by correcting the text that has been fully annotated in step 2.

2. The word vector feature according to claim 1 is characterized in that In step 1, word class features, entity boundary features, and dictionary features are added to the input word vector. Word class features refer to the classification of words into verbs, adverbs, adjectives, etc. based on their characteristics. They can help better determine the boundaries of entities when performing entity recognition tasks. Entity boundaries are crucial for determining the position information of characters in each entity, and are also an important basis for entity boundary recognition. They make full use of syntactic information to enrich the representation of word vectors and help the model improve recognition task performance.

3. The dictionary correction layer according to claim 1, characterized in that In step 3, the self-built dictionary is used to perform secondary correction on the entities not recognized by the model to identify the entities missed by the model. The specific operation steps of the correction are as follows: first, take out the string S consisting of n characters in front of the annotation sentence. n ; If the string S n If the length is 1, delete the leftmost character in the annotation statement X and re-take the string. Otherwise, traverse each entity i in the dictionary and judge whether the entity i and the string S n Are they equal? ​​If entity i is equal to string S n If they are equal, then the string S n is the entity represented by i, then delete S from the left side of the annotation statement X n , and S n The type marked as i returns to the beginning and re-takes the string; if entity i is the same as string S n If they are not equal, delete the leftmost character of the label statement X.

Citation Information

Cited By

  • Intelligent mobile phone model recommendation system for coprocessing large and small models

    CN119357755A