Named entity recognition method for Chinese electronic medical records based on attention mechanism

Abstract

The invention discloses a method for named entity recognition of Chinese electronic medical records based on an attention mechanism. Firstly, Chinese electronic medical record medical data is collected, a medical dictionary is constructed, a fine-tuned pre-trained language model RoBERTa‑wwm‑ext‑large is applied to perform feature extraction and input into a BiLSTM model to learn the contextual semantics of medical texts; then the medical dictionary is queried, a local attention mechanism based on medical concepts is designed, and local attention weights matching the medical concepts in the dictionary are calculated; according to the medical concepts to which each word belongs, a graph neural network is constructed, a graph aggregation representation of the word is calculated as the position embedding of the word, and the context representation and attention representation of each word are fused to obtain comprehensive semantic features; finally, a global pointer network is used to predict nested entities, thereby solving the problems of fuzzy entity boundaries and difficult-to-handle entity nesting in the task of named entity recognition of Chinese electronic medical records.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing in artificial intelligence, and specifically relates to a Chinese electronic medical record named entity recognition method based on an attention mechanism. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and big data technology, electronic medical records (EMR) have been widely used in the medical field. Electronic medical record data is the most important source of empirical data in biomedical research. It contains important medical information, such as clinical symptoms, diagnosis results, treatment plans, etc. It can be used as a treasure trove of medical knowledge and is widely used in the construction of medical knowledge graphs, medical question-answering systems, disease prediction and medical research. Electronic medical record named entity recognition is to automatically identify and extract medical entity information such as disease names, symptoms, diagnosis and treatment, and medication from electronic medical records. It is a basic task of natural language processing and information extraction, and a key step in the digitalization and intelligence of medical care.

[0003] Early named entity recognition methods can be summarized as rule-based methods and machine learning methods based on feature engineering. These methods rely heavily on domain experts to manually formulate rules and feature engineering, which is costly and inefficient. In recent years, with the development of deep learning technology and pre-trained language models, researchers have proposed many electronic medical record named entity recognition methods based on deep learning technology. In the prior art, an attention network based on medical dictionaries is proposed, and a Bert+BiLSTM+Attention+CRF model is designed to enhance the model's medical term segmentation and recognition capabilities, but the problem of entity embedding in Chinese electronic medical records is ignored, so the generalization of the model is low. In the prior art, the naming recognition problem is modeled as a machine reading comprehension problem, and multi-granularity embedding is used to represent medical text. Similar methods are used in the prior art, but the MRC task has the disadvantage of high complexity in solving multi-category NER problems. In the prior art, it is proposed to apply fine-grained medical text segmentation and establish a BiLSTM+CRF recognition model to improve the performance of entity recognition, but the algorithm has obvious defects in solving the problem of fuzzy entity boundaries. With the development of pre-trained language models, the application of the medical field knowledge training model Med-BERT in the existing technology has improved the recognition performance to a certain extent, but this type of pre-trained model cannot solve the entity nesting problem that is widely present in Chinese electronic medical records. The existing technology uses the graph attention network GAT to model Chinese medical texts, but because it does not consider the position representation of each character in the medical text, its performance is affected to a certain extent. With the gradual maturity of generative large model technologies such as ChatGPT, researchers have begun to try to apply large models in the medical field. For example, the existing technology uses large model medical dialogues to enhance the performance of Chinese medical diagnosis. Whether it is a pre-trained language model or a large model such as ChatGPT, its application in the medical and health field is still in the exploratory stage.

[0004] At present, although there have been a lot of research on named entity recognition in electronic medical records, the Chinese medical record named entity recognition task has problems such as long text sequences, nested medical entities, and fuzzy entity boundaries, which leads to some challenges and difficulties in this field. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the Chinese electronic medical record named entity recognition method based on attention mechanism provided by the present invention solves the problems of blurred entity boundaries and nested medical entities existing in the existing related Chinese electronic medical record named entity recognition methods.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for named entity recognition of Chinese electronic medical records based on attention mechanism, comprising the following steps:

[0007] S1. Collect Chinese electronic medical record text data and construct a medical dictionary:

[0008] S2, input the Chinese electronic medical record text data into the pre-trained language model RoBERTa-wwm-ext-large for feature extraction to obtain the corresponding medical word embedding;

[0009] S3. For each word in the Chinese electronic medical record text data, the medical concept embedding of the word is searched in the medical dictionary, and the medical concept embedding is added bit by bit to the medical word embedding of the word to obtain the final word embedding;

[0010] S4, capturing the contextual semantics of each word in the Chinese electronic medical record text data through the BiLSTM model, and querying the medical dictionary to generate a masked attention vector, and then calculating the attention weight that matches the medical concept in the medical dictionary;

[0011] S5. For each medical concept to which a word belongs, a graph neural network is constructed to calculate the graph aggregation representation of the word as the position embedding of the word;

[0012] S6. The medical word embedding obtained by the BiLSTM model is added to the position embedding of the word as the value in the self-attention mechanism, and then multiplied by the attention weight to calculate the attention representation of each word in the medical text; the word embedding obtained by the BiLSTM model is concatenated with the attention representation of the word, and after linear transformation, the comprehensive semantic feature representation of the word is obtained;

[0013] S7. Input the sentence feature representation composed of comprehensive semantic features into the global pointer network, calculate the named entity score, and obtain the named entity category.

[0014] Furthermore, in step S1, the data in the medical dictionary includes medical entity names and medical concept names in Chinese electronic medical records.

[0015] Furthermore, in step S2, the pre-trained language model RoBERTa-wwm-ext-large integrates the word segmenter LTP, and is a model that is fine-tuned based on Chinese electronic medical record text data.

[0016] Furthermore, in step S3, the medical concept embedding is the average of all medical word embeddings in the same medical concept to which each word in the Chinese electronic medical record text data belongs.

[0017] Furthermore, the step S4 is specifically as follows:

[0018] S41. The BiLSTM model is used to capture the contextual semantics of each word in the Chinese electronic medical record text data, forming a feature representation of each word in the medical text;

[0019] S42, query the medical dictionary, match the medical concepts of each word in the text, and generate a masked attention vector;

[0020] S43. Calculate the attention weight of each word based on the masked attention vector.

[0021] Furthermore, in step S42, in the generated masked attention vector, the mask of the position where the words appearing in the medical dictionary are located is 1, and the mask of the positions where the remaining words are located is 0.

[0022] Furthermore, in step S43, the formula for calculating the attention weight of each word in the text is:

[0023]

[0024] In the formula, represents the embedding representation obtained after the mask operation, O i and O j represents the embedding representation of the i-th and j-th words output by the BiLSTM model, T j Indicates O j The corresponding mask attention vector, represents the mask operation, W u represents the learnable parameters in the model, [;] represents the connection operation, α i Indicates O j Relative to O i The attention weight, tanh() represents the hyperbolic tangent function, which is a nonlinear activation function, u i and u j They represent the embedding representations of the i-th and j-th medical word embeddings after nonlinear transformation, and u ik u represents the kth word in the medical concept of the i-th word k , n is the number of words in the medical concept where the i-th word is located.

[0025] Furthermore, in step S5, the position is embedded The calculation formula is:

[0026]

[0027] In the formula, represents the aggregate representation of all neighbor nodes of the target node v at the kth layer, W pool , W k and b represent learnable parameters, Represents the neighbor node u of the target node v i The k-1th layer embedding representation of represents the k-th layer embedding representation of the target node v, represents the final embedding representation of the target node v after aggregation of all neighbor layers, that is, the position embedding of the medical word represented by the target node v in the medical concept, represents the k-1th layer embedding representation of the target node v, σ(·) represents the ReLU function, and u i represents neighbor nodes, Represents a set of neighbor nodes.

[0028] Furthermore, in step S6, the comprehensive semantic feature representation e i for:

[0029]

[0030] e i =tanh(W e [c i ;O i ])

[0031] In the formula, tanh(·) represents the hyperbolic tangent function, which is a nonlinear activation function, and W e represents the parameters that can be learned in the model, O i and O j represents the embedding representation of the i-th and j-th words output by the BiLSTM model, c i represents the attention representation of the i-th medical word relative to other words, α j Indicates O i Relative to O j The attention weight, Indicates O j The corresponding position is embedded.

[0032] Furthermore, the step S7 is specifically as follows:

[0033] S71. For each comprehensive semantic feature representation in the sentence feature representation, e i Transform according to the following formula to obtain the starting position representation q of the αth type of named entity: i,α and the end position indicates k i,α ;

[0034] q i,α =W q,α e i +b q,α

[0035] k i,α =W k,α e i +b k,α

[0036] Where W q,α and W k,αrepresents the learnable parameters in model training, b q,α and b k,α Represents the bias term of the model;

[0037] S72, indicating q according to the starting position i,α and the end position indicates k i,α , calculate the score s of a named entity of type α for the continuous segment from i to j α (i,j);

[0038]

[0039] In the formula, R i represents the rotational position encoding of the i-th word, R j represents the rotational position encoding of the jth word, R j-i It represents the relative rotation position encoding from the i-th word to the j-th word in the medical text, and the superscript T represents the matrix transposition operation.

[0040] The beneficial effects of the present invention are:

[0041] (1) In order to enhance the model's medical knowledge perception ability, a medical dictionary containing 105,706 medical entity names and commonly used terminology in Chinese electronic medical records was constructed. Medical concept information was introduced in feature extraction and local attention mechanism design, which enhanced the model's domain knowledge adaptability and improved the effect of medical entity recognition.

[0042] (2) Fusion of medical word embedding and medical concept embedding: When extracting features, not only the embedding representation of the character itself is considered, but also the concept-level embedding representation of the medical concept to which the character belongs. This enriches the semantic information of the character feature representation and makes it more adaptable to medical domain knowledge.

[0043] (3) Design of a local attention mechanism based on medical concepts: Different from the standard Attention mechanism, this paper focuses on the attention of medical words in medical concepts, and then designs a local attention mask based on medical concepts. When calculating the attention weight, characters in the electronic medical record text that are not related to medical concepts or medical terms are masked out, which can effectively solve the problem of blurred entity boundaries.

[0044] (4) Medical word position embedding representation based on graph neural network: Position information is crucial when processing text sequences. The present invention draws on the idea of ​​the graph neural network GraphSAGE algorithm to dynamically learn the position embedding of medical words in medical concepts through aggregation representation, thereby enhancing the feature representation ability of medical word sequences.

[0045] (5) An electronic medical record named entity recognition architecture based on RoBERTa+BiLSTM+Attention+GP was designed, which incorporated domain knowledge of medical terminology and medical concepts to improve the model's perception of domain knowledge and effectively solve the problems of fuzzy entity boundaries and entity nesting in the Chinese electronic medical record named entity recognition task. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Flowchart of the Chinese electronic medical record named entity recognition method based on attention mechanism provided by the present invention.

[0047] Figure 2 This is a framework diagram of the Chinese electronic medical record named entity recognition process provided by the present invention.

[0048] Figure 3 This is a schematic diagram of node aggregation provided by the present invention. DETAILED DESCRIPTION

[0049] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0050] The embodiment of the present invention provides a method for identifying named entities in Chinese electronic medical records based on an attention mechanism, such as Figure 1-2 As shown, the following steps are included:

[0051] S1. Collect Chinese electronic medical record text data and construct a medical dictionary:

[0052] S2, input the Chinese electronic medical record text data into the pre-trained language model RoBERTa-wwm-ext-large for feature extraction to obtain the corresponding medical word embedding;

[0053] S3. For each word in the Chinese electronic medical record text data, the medical concept embedding of the word is searched in the medical dictionary, and the medical concept embedding is added bit by bit to the medical word embedding of the word to obtain the final word embedding;

[0054] S4, capturing the contextual semantics of each word in the Chinese electronic medical record text data through the BiLSTM model, and querying the medical dictionary to generate a masked attention vector, and then calculating the attention weight that matches the medical concept in the medical dictionary;

[0055] S5. For each medical concept to which a word belongs, a graph neural network is constructed to calculate the graph aggregation representation of the word as the position embedding of the word;

[0056] S6. The medical word embedding obtained by the BiLSTM model is added to the position embedding of the word as the value in the self-attention mechanism, and then multiplied by the attention weight to calculate the attention representation of each word in the medical text; the word embedding obtained by the BiLSTM model is concatenated with the attention representation of the word, and after linear transformation, the comprehensive semantic feature representation of the word is obtained;

[0057] S7. Input the sentence feature representation composed of comprehensive semantic features into the global pointer network, calculate the named entity score, and obtain the named entity category.

[0058] In step S1 of the embodiment of the present invention, the general word segmentation model or word segmentation algorithm usually segments the text by words or subwords, and there are a large number of medical concepts and terms in the Chinese medical electronic medical record text. Subdividing these concepts or terms will cause the loss of medical semantics or even misunderstanding. Therefore, building a medical dictionary is very important for named entity recognition of Chinese electronic medical records.

[0059] In step S1 of this embodiment, the data in the medical dictionary includes medical entity names and medical concept names in Chinese electronic medical records.

[0060] Specifically, in this embodiment, 105,706 medical entity names and commonly used terminology names in Chinese electronic medical records are collected and sorted to construct a Chinese medical dictionary, of which 54,304 entities are from the International Classification of Diseases ICD library, 12,109 entities are from the Disease Diagnosis Code Library ICD-10, as well as 37,308 drug name terms and 1,985 common terms in Chinese electronic medical records. At the same time, Trie Tree is used to store the constructed medical dictionary, and the Aho-Corasick algorithm is used to search and match words in Chinese medical concepts in the dictionary. Trie Tree is a multi-branch tree structure, which can be used with the Aho-Corasic algorithm to well implement the storage and search of medical dictionaries.

[0061] The medical dictionary in this embodiment has two functions. First, it is used to calculate the concept embedding of each word in the medical text, so that the model is more suitable for the task of medical entity recognition and enriches the semantics of feature extraction of medical text. Second, it is used to calculate the local attention based on medical concepts, that is, to generate a masked attention vector.

[0062] In step S2 of the embodiment of the present invention, the pre-trained language model RoBERTa-wwm-ext-large integrates the word segmenter LTP, and is a model that is fine-tuned based on Chinese electronic medical record text data.

[0063] Specifically, the pre-trained language model RoBERTa-wwm-ext-large in this embodiment is used as a feature extraction model and trained in the MLM manner using the whole-word masking strategy, which preserves the integrity of the entire phrase or expression, helps to solve the problem of information loss in Chinese electronic medical records, and enables the model to better understand the context information of the text. Since the pre-trained language model RoBERTa-wwm-ext-large is trained on general data corpora such as wikipedia, Book-Corpus, CC-NEWS, OPENWEBTEXT, STORIES, etc., there is a mismatch with data in specific domains such as Chinese electronic medical records. Therefore, in this embodiment, before applying RoBERTa-wwm-ext-large for feature extraction, a large amount of Chinese electronic medical record data is collected to fine-tune the RoBERTa-wwm-ext-large model.

[0064] In step S2 of this embodiment, the Chinese electronic medical record text data is input into the fine-tuned pre-trained language model RoBERTa-wwm-ext-large for feature extraction to obtain medical word embeddings.

[0065] In step S3 of the embodiment of the present invention, the medical concept embedding is the mean of all medical word embeddings in the same medical concept to which each word in the Chinese electronic medical record text data belongs.

[0066] Exemplarily, the final word-level embedding representation of the word "branch" in "bronchitis" is the bitwise sum of the mean of the embeddings of the four words "branch", "qi", "tube", "inflammation" and the embedding representation of the word "branch" itself. Combining the semantics of the medical concept with the semantics of the word itself in this way can enhance the semantic expression ability of medical characters.

[0067] Step S4 of the embodiment of the present invention is specifically as follows:

[0068] S41. Capture the context semantics of each word in the Chinese electronic medical record text data through a BiLSTM model to form the feature representation of each word in the medical text;

[0069] S42. Query the medical dictionary to match the medical concepts of each word in the text and generate a masked attention vector;

[0070] S43. Calculate the attention weights of each word based on the masked attention vector.

[0071] In step S41 of this embodiment, the final word embedding representation is input into the BiLSTM model for sequence encoding, and the electronic medical record text is selectively forgotten or transferred through the BiLSTM model gating mechanism to capture the contextual semantics of the text and form a feature representation of each word in the text.

[0072] In step S42 of this embodiment, in the generated masked attention vector, the mask of the position where the words appearing in the medical dictionary are located is 1, and the mask of the positions where the other words are located is 0. The masked attention vector in this embodiment can help the model pay more attention to the words related to medical concepts in the text, effectively reducing the noise of named entity recognition.

[0073] Specifically, the medical dictionary is queried to obtain the medical concepts of each word (i.e., token) in the text and form a masked attention vector.

[0074] For example, the masked attention vector corresponding to the text "The patient has cough and sputum, and is suspected of having bronchitis" is T = [0,0,0,0,1,1,1,1,0,0,0,0,0,1,1,1,1], because the two medical concepts of "cough and sputum" and "bronchitis" are in the constructed medical dictionary, so the mask at the corresponding position is 1, and the masks at the other positions are 0.

[0075] In step S43 of this embodiment, the formula for calculating the attention weight of each word in the text is:

[0076]

[0077] In the formula, represents the embedding representation obtained after the mask operation, O i and O j represents the embedding representation of the i-th and j-th words output by the BiLSTM model, T j Indicates O j The corresponding mask attention vector, represents the mask operation, W u represents the learnable parameters in the model, [;] represents the connection operation, α i Indicates O j Relative to O i The attention weight, tanh() represents the hyperbolic tangent function, which is a nonlinear activation function, u i and u j They represent the embedding representations of the i-th and j-th medical word embeddings after nonlinear transformation, and u ik u represents the kth word in the medical concept of the i-th word k , n is the number of words in the medical concept where the i-th word is located.

[0078] Among them, for the masking operation it means that the words corresponding to "1" in the masked attention vector participate in the calculation of the attention weight, and the words with "0" are masked, and finally

[0079] In step S5 of the embodiment of the present invention, the medical concept or medical term is a string, such as "bronchitis". When learning the embedding representation of each word, the sequential position relationship of the words in the string is crucial. Obtaining the position information of the word in the medical concept can make up for the defect of ignoring the position information when calculating the attention weight. This embodiment only targets the words after word segmentation of the electronic medical record text, and the word belongs to a certain medical concept in the medical dictionary. Drawing on the idea of the graph neural network GraphSAGE, the embedding representation of the word in the medical concept is obtained through multi-layer aggregation as the position embedding of the word in the medical concept. As Figure 3 shown, according to the idea of aggregating neighbors towards the central word, the position embedding of "zhi" in "bronchitis" is obtained through three-layer neighbor aggregation, while "qi" only needs two-layer aggregation.

[0080] Based on this, the position embedding in step S5 of the embodiment is

[0081]

[0082]

[0083] In the formula, represents the aggregated representation of all neighbor nodes of the target node v at the k-th layer, W pool , W k and b represent learnable parameters, represents the (k-1)-th layer embedding representation of the neighbor node u i of the target node v, represents the k-th layer embedding representation of the target node v, represents the final embedding representation of the target node v after aggregation through all neighbor layers, that is, the position embedding of the medical word represented by the target node v in the medical concept, represents the (k-1)-th layer embedding representation of the target node v, σ(·) represents the Relu function, u i represents the neighbor node, represents the neighbor node set.

[0084] Specifically, in this embodiment, a non-linear transformation is first performed on the (k-1)-th layer embedding representation of the neighbor nodes of the target node, and then a max-pooling operation is performed to obtain the aggregated representation of all neighbor nodes of the target node v at the k-th layer and the target node’s embedding representation at layer k-1 After concatenation, a nonlinear transformation is performed to obtain the k-th layer embedding representation of the target node. The final aggregate representation is the position embedding of the medical word represented by the target node v in the medical concept

[0085] In step S6 of the embodiment of the present invention, the comprehensive semantic feature representation e i for:

[0086]

[0087] e i =tanh(W e [c i ;O i ])

[0088] In the formula, tanh(·) represents the hyperbolic tangent function, which is a nonlinear activation function, and W e represents the parameters that can be learned in the model, O i and O j represents the embedding representation of the i-th and j-th words output by the BiLSTM model, c i represents the attention representation of the i-th medical word relative to other words, α i Indicates O i Relative to O j The attention weight, Indicates O j The corresponding position is embedded.

[0089] In this embodiment, based on the above processing, the obtained comprehensive semantic feature representation is fused to form a sentence feature representation [e 1 ,e 2 ,…,e n ] as the input of the global pointer network GP.

[0090] Step S7 of the embodiment of the present invention is specifically:

[0091] S71. For each comprehensive semantic feature representation in the sentence feature representation, e i Transform according to the following formula to obtain the starting position representation q of the αth type of named entity: i,α and the end position indicates k i,α ;

[0092] q i,α =W q,α e i +b q,α

[0093] k i,α =Wk,α e i +b k,α

[0094] Where W q,α and W k,α represents the learnable parameters in model training, b q,α and b k,α Represents the bias term of the model;

[0095] S72, indicating q according to the starting position i,α and the end position indicates k i,α , calculate the score s of a named entity of type α for the continuous segment from i to j α (i,j);

[0096]

[0097] In the formula, R i represents the rotational position encoding of the i-th word, R j represents the rotational position encoding of the jth word, R j-i It represents the relative rotation position encoding from the i-th word to the j-th word in the medical text, and the superscript T represents the matrix transposition operation.

[0098] Specifically, in this embodiment, the global pointer network GP is used to model the Chinese electronic medical record named entity recognition problem as a multi-label classification problem, and the sentence feature representation [e 1 ,e 2 ,…,e n ]Process in the above manner to obtain the scores of each type of named entity.

[0099] In this embodiment, in order to make full use of the boundary information of the electronic command named entity, the rotation position encoding technology is introduced into the entity score, and the characteristics of the transformation matrix are used to Get the score s containing the relative position information from i to j α (i, j). In this way, identifying entities of a specific class α becomes a multi-label classification problem with a total of n(n+1) / 2 categories, where the number of labels is the number of entity categories in the electronic medical record text. Finally, the entity categories are obtained: B-dis (disease start), I-dis (disease middle), B-sym (symptom start), I-sym (symptom middle), B-dru (drug start), I-dru (drug middle), etc., and then combined to obtain the medical entities in the sentence: disease, symptom, drug, etc.

[0100] In the embodiment of the present invention, the Global Pointer Network (GP) is used as a classifier for named entity recognition, which effectively solves the problem of entity nesting in Chinese electronic medical records. GP considers the starting and ending positions of the entity at the same time, and uses the feature information of the head and tail to predict the nested entity. GP uses the idea of ​​global normalization for named entity recognition, is compatible with nested entities and non-nested entities, and can effectively solve the problem of entity nesting; at the same time, compared with the commonly used entity classifier conditional random field CRF, GP can run in parallel, and the efficiency is greatly improved.

[0101] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0102] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A Chinese electronic medical record named entity recognition method based on attention mechanism, characterized by: The following steps are involved: S1. Collect Chinese electronic medical record text data and construct a medical dictionary: S2, input the Chinese electronic medical record text data into the pre-trained language model RoBERTa-wwm-ext-large for feature extraction to obtain the corresponding medical word embedding; S3. For each word in the Chinese electronic medical record text data, the medical concept embedding of the word is searched in the medical dictionary, and the medical concept embedding is added bit by bit to the medical word embedding of the word to obtain the final word embedding; S4, capturing the contextual semantics of each word in the Chinese electronic medical record text data through the BiLSTM model, and querying the medical dictionary to generate a masked attention vector, and then calculating the attention weight that matches the medical concept in the medical dictionary; S5. For each medical concept to which a word belongs, a graph neural network is constructed to calculate the graph aggregation representation of the word as the position embedding of the word; S6. The medical word embedding obtained by the BiLSTM model is added to the position embedding of the word as the value in the self-attention mechanism, and then multiplied by the attention weight to calculate the attention representation of each word in the medical text; The word embedding obtained by the BiLSTM model is concatenated with the attention representation of the word, and after linear transformation, the comprehensive semantic feature representation of the word is obtained; S7, inputting the sentence feature representation composed of comprehensive semantic features into the global pointer network, calculating the named entity score, and obtaining the named entity category; In step S5, the position is embedded The calculation formula is: In the formula, represents the aggregate representation of all neighbor nodes of the target node v at the kth layer, W pool , W k and b represent learnable parameters, Represents the neighbor node u of the target node v i The k-1th layer embedding representation of represents the k-th layer embedding representation of the target node v, represents the final embedding representation of the target node v after aggregation of all neighbor layers, that is, the position embedding of the medical word represented by the target node v in the medical concept, represents the k-1th layer embedding representation of the target node v, σ(·) represents the ReLU function, and u i represents neighbor nodes, Represents a set of neighbor nodes; The step S7 is specifically as follows: S71. For each comprehensive semantic feature representation in the sentence feature representation, e i Transform according to the following formula to obtain the starting position representation q of the αth type of named entity: i,α and the end position indicates k i,α ; q i,α =W q,α e i +b q,α k i,α =W k,α e i +B k,α Where W q,α and W k,α represents the learnable parameters in model training, b q,α and b k,α Represents the bias term of the model; S72, indicating q according to the starting position i,α and the end position indicates k i,α , calculate the score s of a named entity of type α for the continuous segment from i to j α (i,j); In the formula, R i represents the rotational position encoding of the i-th word, R j represents the rotational position encoding of the jth word, R j-i It represents the relative rotation position encoding from the i-th word to the j-th word in the medical text, and the superscript T represents the matrix transposition operation.

2. According to the attention mechanism-based Chinese electronic medical record named entity recognition method of claim 1, it is characterized in that: In step S1, the data in the medical dictionary includes medical entity names and medical concept names in Chinese electronic medical records.

3. According to the attention mechanism-based Chinese electronic medical record named entity recognition method of claim 1, it is characterized in that: In step S2, the pre-trained language model RoBERTa-wwm-ext-large integrates the word segmenter LTP, and is a model that is fine-tuned based on Chinese electronic medical record text data.

4. The method for named entity recognition of Chinese electronic medical records based on attention mechanism according to claim 1 is characterized in that: In step S3, the medical concept embedding is the average of all medical word embeddings in the same medical concept to which each word in the Chinese electronic medical record text data belongs.

5. The method for named entity recognition of Chinese electronic medical records based on attention mechanism according to claim 1 is characterized in that: The step S4 is specifically as follows: S41. The BiLSTM model is used to capture the contextual semantics of each word in the Chinese electronic medical record text data, forming a feature representation of each word in the medical text; S42, query the medical dictionary, match the medical concepts of each word in the text, and generate a masked attention vector; S43. Calculate the attention weight of each word based on the masked attention vector.

6. The method for named entity recognition of Chinese electronic medical records based on attention mechanism according to claim 5 is characterized in that: In step S42, in the generated masked attention vector, the mask of the position where the words appearing in the medical dictionary are located is 1, and the mask of the positions where the other words are located is 0.

7. The method for named entity recognition of Chinese electronic medical records based on attention mechanism according to claim 5 is characterized in that: In step S43, the formula for calculating the attention weight of each word in the text is: In the formula, represents the embedded representation after mask operation, O i and O j represents the embedding representation of the i-th and j-th words output by the BiLSTM model, T j Indicates O j The corresponding mask attention vector, represents the mask operation, W u represents the learnable parameters in the model, [;] represents the connection operation, α i Indicates O j Relative to O i The attention weight, tanh() represents the hyperbolic tangent function, which is a nonlinear activation function, u i and u j They represent the embedding representations of the i-th and j-th medical word embeddings after nonlinear transformation, and u ik u represents the kth word in the medical concept of the i-th word k , n is the number of words in the medical concept where the i-th word is located.

8. The method for named entity recognition of Chinese electronic medical records based on attention mechanism according to claim 1 is characterized in that: In step S6, the comprehensive semantic feature representation e i for: e i =tanh(W e [c i ;O i ]) In the formula, tanh(·) represents the hyperbolic tangent function, which is a nonlinear activation function, and W e represents the parameters that can be learned in the model, O i and O j represents the embedding representation of the i-th and j-th words output by the BiLSTM model, c i represents the attention representation of the i-th medical word relative to other words, α j Indicates O i Relative to O j The attention weight, Indicates O j The corresponding position is embedded.

Images