A method for labeling medical entity parts of speech in Chinese electronic medical records

Abstract

The present invention discloses a method for labeling medical entity word classes of Chinese electronic medical records. The method includes two parts. The first part is the formulation of fine-grained Chinese electronic medical record medical entity category classification rules and the construction of a corpus data set; the second part is a method for labeling medical entity word classes of Chinese electronic medical records based on a gated attention mechanism; a fine-grained Chinese electronic medical record medical entity labeling rule is formulated according to the characteristics of Chinese electronic medical record texts, and a Chinese electronic medical record medical entity corpus data set is constructed according to this rule, and a GATTLCN model combined with a gated attention mechanism is invented based on the characteristics of Chinese electronic medical record texts, and the context elements that need to be focused on are dynamically selected through the gated attention network to enhance text focus and improve the model effect. The invention solves the deficiencies in the fine-grained medical entity word class labeling of Chinese electronic medical records.

Description

Technical Field

[0001] The present invention relates to the field of medical entity word class labeling, specifically including the construction of medical entity category labeling rules and data sets for Chinese electronic medical records, and a method for labeling medical entity word classes for Chinese electronic medical records using a Bi-LSTM+CRF joint network model (GATTLCN model) combined with a gated attention mechanism. Background Art

[0002] In the context of the rapid development of science and technology, with the continuous development of "Internet + medical" technology, more and more research has begun to focus on the intersection of computers and medical information. In the era of rapid development of medical informationization, major hospitals have accumulated a large amount of clinical electronic medical record data. Electronic Medical Record (EMR) is also called computer-based patient records. It is a digital storage, transmission, reproduction and management of clinical medical records generated by patients during medical treatment through electronic devices (computers, electronic cards, etc.) to replace handwritten paper medical records. The content of the electronic medical record includes all the information of the original paper medical record. It refers to the digital information such as text, graphics, symbols, images, etc. generated by medical staff using the information system of the medical institution during the medical activities, and can realize the storage, transmission, reproduction and management of medical records.

[0003] At present, the electronic medical record system has been widely used in various medical institutions in China, and researchers have a better understanding of electronic medical records. It not only contains the patient's clinical diagnosis and treatment information, such as symptom descriptions, test results, disease diagnosis, treatment measures, etc., but also contains a large amount of medical entity information. However, in order to realize the electronicization of electronic medical record medical entity information and further explore its potential medical value on this basis, it is particularly important to structure the medical entities in the electronic medical record text. Electronic medical record medical entity word class annotation is the first step in building a medical knowledge graph and medical decision assistance. The effect of medical entity annotation directly determines the effect and quality of medical decision assistance.

[0004] At present, the part-of-speech tagging technology in the general text field is becoming increasingly mature. Part-of-speech tagging tools such as THULAC and Jieba have achieved an accuracy rate of more than 90% on public data sets in the general field. However, in the special field of medical entity part-of-speech tagging in Chinese electronic medical records, due to the lack of a unified medical entity classification standard and the unique grammatical structure and text style of electronic medical records, the existing annotation toolkits cannot accurately tag medical entities in Chinese electronic medical records.

[0005] The essence of tagging medical entities in Chinese electronic medical records is to perform named entity recognition (NER) on Chinese electronic medical records. Medical entities are the main carriers of relevant medical knowledge in electronic medical records, and clinical named entity recognition (CNER) is also one of the most basic tasks in the analysis and processing of Chinese electronic medical records. In recent years, researchers have conducted extensive research on named entity recognition in electronic medical records and proposed many high-accuracy methods. These methods can be roughly divided into three types: methods based on rules and word matching, methods based on machine learning, and methods based on deep learning.

[0006] In addition, the attention mechanism has achieved good results in combination with various models, but the traditional attention mechanism focuses on all the words in the input sentence, aggregates information from the entire hidden state sequence, calculates the attention weight of the hidden vector at each position in the input source, and then obtains the text focus vector. However, in the Chinese electronic medical record medical entity word class labeling task, the input is based on each medical record, and it is not necessary to assign attention weights to each word during the labeling process, that is, not all attention is required, especially in long sequences of electronic medical record texts. Therefore, in order to better extract text focus semantics, the present invention adopts a gated attention mechanism to improve model performance, dynamically selects the set of words that need to be paid attention to through an auxiliary gate structure, and then calculates the text focus vector, which avoids calculating all elements and allows the model to dynamically focus on important parts of the sequence. Summary of the invention

[0007] The present invention aims to solve the shortcomings of the existing Chinese electronic medical record medical entity word class annotation. According to the characteristics of Chinese electronic medical record texts, a fine-grained Chinese electronic medical record medical entity annotation rule is formulated. At the same time, a Chinese electronic medical record medical entity corpus dataset is constructed according to this rule. Based on the characteristics of Chinese electronic medical record writing, a Bi-LSTM+CRF joint network model (GATTLCN model) combined with a gated attention mechanism is invented. The gated attention network dynamically selects contextual elements that need to be focused on to enhance text focus and improve the model effect.

[0008] To achieve the above object, the present invention adopts the following technical solution, which is mainly divided into two parts.

[0009] The first part of the fine-grained Chinese electronic medical record medical entity classification rule formulation and corpus dataset construction includes the following steps:

[0010] Step 1: Establish rules for classifying medical entities in Chinese electronic medical records.

[0011] The present invention refers to the definition of medical entity categories in the UMLS semantic type, and combines the research of many scholars at home and abroad on similar topics and the suggestions given by professionals. The Chinese electronic medical record medical entities are divided into six basic entity categories: body parts, symptoms, drugs, treatments, diseases and tests. At the same time, ten extended categories are formulated based on the basic categories: time points, time periods, specific scenarios, medical indicators, medical history, non-patients, negative words, high degrees, low degrees and institutions. Finally, "no specific category" is set to mark entities that do not belong to the above categories.

[0012] Step 2: Manual annotation, constructing a Chinese electronic medical record medical entity annotation corpus dataset.

[0013] Preferably, the labeling work in step 2 should be carried out in full accordance with the defined classification of medical entities. When there is ambiguity or ambiguity in the labeling, the labeling should be suspended and discussed, and the advice of medical experts should be referred to to reach a consensus.

[0014] As a preference, the labeling work in step 2 is divided into two stages. The first stage is carried out under the full guidance of professionals. The initial stage is mainly to allow the labelers to adapt to the labeling mode, conduct cross-checks while familiarizing themselves with the rules, and keep the consistency of the labeling results. The later stage is the main labeling process. In the second stage, professionals conduct unified sampling inspection and correction of the labeling results.

[0015] Part II: A method for labeling medical entity word classes in Chinese electronic medical records based on a gated attention mechanism, including the following steps:

[0016] Step 1: Obtain the word vector of the Chinese electronic medical record. The present invention uses the skip-gram model of word2vec to train the Chinese electronic medical record text after word segmentation to obtain the low-dimensional dense vector of the Chinese electronic medical record.

[0017] Step 2: Obtain the contextual semantic information of the Chinese electronic medical record. Use the word vector obtained in step 1 as the input vector and input it into the bidirectional long short-term memory network to extract the contextual semantic information of the input text and capture the sequence information and medium- and long-range dependency information of the Chinese electronic medical record text.

[0018] Step 3: Use the gated attention layer to extract the semantic focus vector of the Chinese electronic medical record text, and use the latent vector output from step 2 as the input vector to input into the gated attention network, focusing on the information that is more critical to the current task from the overall perspective of the text, and filtering out information that is irrelevant to the current task. By applying the gated attention mechanism to the model, the model can dynamically calculate the weights of words in the electronic medical record text, and further derive the importance of each word to different medical entities, thereby filtering out irrelevant information and enhancing the model's dynamic semantic focus capability.

[0019] Step 4: Use conditional random fields to fully consider the constraints and dependencies between medical entity labels. Use the text semantic focus vector output in step 3 as the input vector and input it into the CRF layer to predict the best medical entity word class label based on the contextual medical entity sentence-level label information.

[0020] Step 5: Use the trained GATTLCN joint network model to label the medical entity word classes of the test dataset.

[0021] Preferably, in step 1, the word vector of the Chinese electronic medical record that has been segmented is used to train the word vector. The present invention adopts the skip-gram model of word2vec to obtain the low-dimensional dense word vector of the Chinese electronic medical record, and the word vector dimension is set to 100 dimensions. Each word vector is obtained as: X = {x 1 ,x 2 ,……,x 100}, the text vector of each sentence is: S = {X 1 ,X 2 ,……,X n}, where n is the length of each sentence and the dimension of each word vector is 100.

[0022] As a preference, the Bi-LSTM layer in step 2 uses the word vector X output in step 1 t As input. The LSTM neural network unit has a clever gate structure design for the hidden layer, which uses three logic gates: input gate, output gate, and forget gate to control the proportion of input data transferred to the memory unit and the proportion of forgetting the previous state, so as to extract the contextual semantic information of the medical record text.

[0023] In addition, the previous and next context information is of great value for the annotation of medical entity word classes in Chinese electronic medical records. In order to simultaneously consider the contextual semantic information of electronic medical records, the present invention adopts bidirectional LSTM to learn the hidden representation of sentences from the global context information.

[0024] Preferably, the gated attention layer in step 3 uses the latent vector h output from step 2 t As input. The present invention adds a gated attention mechanism to the model, so that the encoding modeling of the text sequence relationship breaks through the long-distance limitation of the electronic medical record text sequence, focuses on the important semantic information from the overall text, and filters out the unimportant information. Applying the gated attention mechanism to the model allows the model to dynamically calculate the weights of words in the electronic medical record text, and further derive the importance of each word to different medical entities, thereby filtering out irrelevant information.

[0025] The gated attention network uses an auxiliary gate structure to dynamically select a subset of elements that need to be focused on, calculates their attention weights, and then calculates the weighted average of the corresponding element latent vectors based on their attention weights. The gated attention network contains a series of auxiliary gate structures and a backbone attention network. The auxiliary gate structure generates a set of input-related binary gates to determine whether each word needs to be paid attention to in the current task. The backbone network is a conventional attention network that aggregates the latent vectors of the corresponding words by calculating the attention weight values ​​of the words in the selected important medical record text, and then calculates the cumulative sum of the selected latent vectors to obtain the text focus vector.

[0026] Preferably, in order to solve the problem that the binary gate function discretizes the variables and causes the function to be non-differentiable in the process of the gated attention layer using the auxiliary gate structure to dynamically select the subset of attention elements in step three, the present invention adopts the Gumbel Softmax method to continuousize the discrete variable sampling process, so that the normal back propagation algorithm can be used to optimize the model.

[0027] Preferably, the CRF layer in step 4 uses the text focus vector C output in step 3 t As input. The conditional random field can describe the interdependence between output labels through the transfer matrix, fully considering the constraints and dependencies between medical entity labels. When predicting the optimal medical entity class, based on the Viterbi algorithm, the maximum probability label sequence of each word is calculated as the result of the medical entity category recognition of the word, and the medical entity labeling is completed.

[0028] Preferably, in step five, a negative log-likelihood function is used as a loss function during the model training process, and the parameters are optimized by minimizing the loss function.

[0029] Based on the proposed fine-grained Chinese electronic medical record medical entity category classification rules, the present invention constructs a Chinese electronic medical record medical entity corpus dataset based on 1,200 real medical records, and trains the GATTLCN model on this dataset. After the multi-layer Bi-LSTM network trains the contextual semantic features, the present invention further uses the auxiliary gate structure through the gated attention network to dynamically select the subset of elements that need to be focused on, thereby enhancing the model's text focusing ability. The introduction of the gated attention mechanism not only improves the annotation accuracy of the model, but also has a certain enhancement effect on the annotation of unregistered words. This invention solves the shortcomings in the annotation of fine-grained medical entity word classes in Chinese electronic medical records.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] 1. A fine-grained classification rule for medical entities in Chinese electronic medical records is proposed.

[0032] 2. A Chinese electronic medical record medical entity corpus dataset was constructed based on 1,200 real medical records.

[0033] 3. No external medical dictionary assistance is required, reducing the work of building medical entity dictionaries.

[0034] 4. The gated attention mechanism is integrated into the model to improve the model's dynamic semantic focusing ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Flow chart of the method of the present invention.

[0036] Figure 2 Flowchart for constructing a dataset for labeling rules of the present invention.

[0037] Figure 3 This is a diagram of the Chinese electronic medical record medical entity dataset.

[0038] Figure 4 This is a structural diagram of the combined neural network model of the present invention.

[0039] Figure 5 Graph showing the dynamic selection process of the gated attention mechanism of the present invention. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below in conjunction with the embodiments and accompanying drawings. The following description covers many specific details in order to provide a comprehensive understanding of the present invention. The present invention is by no means limited to any specific configuration and algorithm proposed below, but covers any modification, replacement and improvement of related elements, components and algorithms without departing from the spirit of the present invention.

[0041] like Figures 1 to 5 As shown, the Chinese electronic medical record medical entity word category recognition method provided by the present invention mainly constructs a data set based on the fine-grained Chinese electronic medical record medical entity category division rules, and combines the characteristics of the large gap between the writing style of Chinese electronic medical records and the text in the development field, and adds a gated attention mechanism to the deep neural network. In the data set construction stage, the real electronic medical record data is manually annotated based on the proposed fine-grained Chinese electronic medical record medical entity category division rules to construct a data set. In the data processing stage, the self-constructed data set is trained through word2vec to obtain the text word vector. The word vector is input into the Bi-LSTM network, and its contextual semantic feature information is extracted from the front and back directions of the medical record text. Then, the vector with contextual semantic information is input into the gated attention network, and the auxiliary gate structure is used to dynamically select the element subset that needs to be focused on and assign different attention weights to it to calculate the text focus vector, improve the model focus effect, and strengthen the connection between contexts. Finally, the annotation result is output through the CRF layer.

[0042] like Figure 1 Shown is a flow chart of the method of the present invention, and the specific steps of the present invention are as follows:

[0043] Step 1: Develop fine-grained rules for classifying medical entities in Chinese electronic medical records.

[0044] 1.1) First, six basic entity categories are formulated based on the content of the medical record text: body parts, symptoms, drugs, treatments, examinations, and diseases. The entities in the basic categories are directly related to medical information and can directly cover the most important information in the medical record.

[0045] 1.2) Eleven extended categories are formulated based on the basic categories: time point, time period, specific scenario, medical indicator, medical history, non-patient, negative word, high degree, low degree, institution and no category. Although the extended categories do not have direct medical meanings, they can provide a detailed description of the patient's health information in time and space.

[0046] Step 2: Manually annotate real electronic medical records and construct a Chinese electronic medical record medical entity category annotated corpus dataset.

[0047] The process of constructing the Chinese electronic medical record medical entity category corpus dataset is as follows: Figure 2 As shown in the figure, after formulating the classification rules for medical entities, labeling is mainly divided into two stages: pre-labeling and formal labeling.

[0048] 2.1) The labelers are divided into two groups for pre-labeling. After the labeling is completed, the two groups of labeling results are compared to calculate their consistency and analyze the inconsistent labeling. Then, the medical entity classification rules are updated according to the comparison results, and then another round of pre-labeling is carried out.

[0049] 2.2) After completing two rounds of pre-annotation and becoming familiar with the annotation process and the medical entity classification rules, the annotators will conduct formal annotation. After the annotation is completed, the Chinese electronic medical record medical entity classification annotated corpus dataset is constructed as follows: Figure 3 shown.

[0050] Step 3: Build a Bi-LSTM+CRF joint network model combined with gated attention mechanism.

[0051] 3.1) The self-constructed dataset is trained with word2vec to obtain the low-dimensional dense word vector of the Chinese electronic medical record, and the word vector dimension is set to 100 dimensions. Each word vector is obtained as: X = {x 1 ,x 2 ,……,x 100}, the text vector of each sentence is: S = {X 1 ,X 2 ,……,X n}, where n is the length of each sentence and the dimension of each word vector is 100.

[0052] 3.2) Input the medical record text word vector X into the LSTM layer, and use the LSTM neural network to extract the contextual semantic information of the medical record text. LSTM has a clever gate mechanism design for its neural unit, which uses three logic gates to control the ratio of input data to memory units and the ratio of forgetting previous states. LSTM controls the flow of information by introducing three gate structures: input gate, output gate, and forget gate in each storage unit. The calculation formulas for the four states in the LSTM neural network unit are as follows:

[0053] f t =σ(W f ·[h t-1 ,x t ]+b f ) (1)

[0054] i t =σ(W i ·[h t-1 ,x t ]+b i ) (2)

[0055]

[0056]

[0057] o t =σ(W o ·[h t-1 ,x t ]+b o ) (5)

[0058] h t =o t *tanh(c t ) (6)

[0059] Where σ and tanh are the Sigmoid function and the hyperbolic tangent function respectively. t-1 、c t-1 is the hidden vector and unit vector output by the previous neural network unit, x t 、h t are the input and output vectors of the current network unit, i t 、f t , o t are the vectors of input method, forget gate, and output gate respectively. c t is a unit vector, W and b are trainable parameters.

[0060] In addition, in order to simultaneously extract the contextual information before and after the medical record text, the present invention uses a bidirectional LSTM to learn the hidden representation of the sentence from the global contextual information. Indicates that the text is scanned from left to right. It is represented as scanning the text from right to left. Then the hidden vector learned by the bidirectional LSTM network can be expressed as The hidden representation of the i-th character is and connection, that is The output of the Bi-LSTM layer is h = [h 1 ,h 2 ,…,h n ].

[0061] 3.3) The bidirectional LSTM network extracts the medical record text context semantic information h = [h 1 ,h 2 ,…h n ] and then input it into the gated attention layer. Different from the traditional attention network that directly calculates the output attention weight, the gated attention mechanism network has an additional binary gate structure G = {g 1 ,g 2 ,…,g t} t ∈{0,1}, the binary gate is used to control the selection of the current unit hidden vector inflow. t = 1, the tth binary gate is opened, and the current unit hidden vector flows into the attention calculation. t = 0, the tth binary is closed, erasing the current unit hidden vector information. The auxiliary binary gate in the gated attention network is used to dynamically select a subset of elements that need to be focused on, and calculate their attention weights. Then, the weighted average of the hidden vectors of the corresponding elements is calculated based on their attention weights. The calculation of attention weights in the gated attention mechanism network is as follows:

[0062] e t,i =score(h t ,h i ) (7)

[0063] p i =sigmoid(score(h t ,h i )) (8)

[0064]

[0065]

[0066] Where N is the number of words in the electronic medical record text, and β is the threshold for whether the binary gate is open or not. The score function is an attention scoring function, which can be a function of any comparison vector, such as dot product, parameter matrix, scaled dot product, etc. The dot product method is used in the present invention. t,i It refers to the attention weight of the i-th word pair to predict the t-th word label.

[0067] Calculate the text focus vector C through gated attention t When using g t = 1, the binary gate selects the hidden vector of the unit for calculation. t = 0, the hidden vector of the unit corresponding to is not included in the calculation. Finally, the hidden vectors contained in the set M are selected by calculating the weighted sum, and the text focus vector c t The calculation is as follows:

[0068]

[0069] Where M is the number of words that need to be focused on after being screened by the auxiliary gate structure. The gated attention mechanism dynamically selects important elements. Figure 4 shown.

[0070] At the same time, for the binary gate function g t To solve the problem that discretization of variables leads to non-differentiable functions, this paper uses the Gumbel Softmax method proposed by Jang et al. (2017) to make the discrete variable sampling process continuous, so that the normal back propagation algorithm can be used to optimize the model. In order to make the auxiliary binary gate structure differentiable during the model training process, the Gumbel Softmax distribution is used to approximate g in the retraining process. t Gate function, approximated The gate function is as follows.

[0071]

[0072] Among them G i,j are two independent Gumbel (0,1) random noises, τ∈(0,∞) is the temperature parameter, when it is close to 0, the Gumbel Softmax distribution is approximately g t Gated output. Used during training. Function is used to replace the gating mechanism function to optimize β and subsequent parameters.

[0073] 3.4) After the semantic focus vector of the medical record text is extracted in the gated attention layer, it is input into the CRF layer. The conditional random field can obtain the dependency between labels through the state transfer matrix, and obtain the optimal medical record text medical entity category label sequence y=[y 1 ,y 2 ,…,y n ]. The score function Score(x, y) of the sequence label y is calculated as follows:

[0074]

[0075] Where T is the state transition probability matrix, T yi,yi+1 Indicates that the current entity is marked as y i label and its next entity is labeled y i+1 The probability of the label. C is the score matrix output by the gated attention layer, C xi,yi Indicates that the i-th word x in the sentence is labeled y i The probability of the label.

[0076] After calculating the scores of all possible label sequences, the Softmax function is used to normalize the probability of the word class label sequence y to obtain its conditional probability. The calculation formula is as follows:

[0077]

[0078] During the model training process, the negative log-likelihood function is used as the loss function of the model. The parameters are optimized by minimizing the loss function. The negative log-likelihood function is as follows:

[0079]

[0080] Step 4: Prediction of medical entity category annotations in Chinese electronic medical records.

[0081] 4.1) The medical record text that needs to be labeled with medical entity categories and has been segmented (such as: physical examination found gallstones...) is trained with word2vec to obtain the word vector X = {x 1 ,x 2 ,……,x n}.

[0082] 4.2) Input the word vector obtained in 4.1 into the trained GATTLCN model to predict the entity category label of the Chinese electronic medical record data to be extracted. * The largest predicted sequence is taken as the final labeling result, y * The calculation formula is as follows:

[0083]

[0084] in Represents the set of all possible label sequences.

[0085] The present invention is not limited to the specific configurations and processes described above and shown in the figures. Moreover, for the sake of brevity, a detailed description of known methods and techniques is omitted here. In the above embodiments, several specific steps are described and shown as examples, but not every implementation includes only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should take the specification as a whole, and the technical solutions in each implementation can also be appropriately combined and implemented according to the understanding of those skilled in the art. The present invention can be implemented in other specific forms without departing from its spirit and essential characteristics. For example, the algorithm described in a specific embodiment can be modified, and the system architecture does not depart from the basic spirit of the present invention.

[0086] The series of detailed descriptions listed above are only specific descriptions of feasible implementation modes of the present invention. They are not used to limit the protection scope of the present invention. All equivalent implementation modes or changes that do not deviate from the spirit of the invention should be included in the protection scope of the present invention.

Claims

1. A method for labeling medical entity word classes in Chinese electronic medical records, characterized by: Implementing this annotation method mainly includes two parts: The first part is the formulation of fine-grained Chinese electronic medical record medical entity classification rules and the construction of corpus datasets, including the following steps: Step 1: Formulate the classification rules for medical entities in Chinese electronic medical records; Referring to the definition of medical entity categories in UMLS semantic types, Chinese electronic medical record medical entities are divided into six basic entity categories: human body parts, symptoms, drugs, treatments, diseases and tests; ten extended categories are formulated based on the basic entity categories: time point, time period, specific scenario, medical indicator, medical history, non-patient, negative word, high degree, low degree and institution; finally, "no specific category" is set to mark entities that do not belong to the above categories; Step 2: Manual annotation to build a Chinese electronic medical record medical entity annotation corpus dataset; The labeling work in step 2 is carried out completely according to the defined classification of medical entities. The labeling work in step 2 is divided into two stages. The first stage is carried out under the full guidance of professionals. The second stage is for professionals to conduct unified sampling inspection and correction of the labeling results. The second part is a method for labeling medical entity word classes in Chinese electronic medical records based on the gated attention mechanism, which includes the following steps: Step 1: Obtain the word vector of Chinese electronic medical record. Use the skip-gram model of word2vec to train the Chinese electronic medical record text after word segmentation to obtain the low-dimensional dense vector of Chinese electronic medical record. Step 2: Obtain the contextual semantic information of the Chinese electronic medical record. Use the word vector obtained in step 1 as the input vector and input it into the bidirectional long short-term memory network to extract the contextual semantic information of the input text and capture the sequence information and medium- and long-range dependency information of the Chinese electronic medical record text. Step 3: Use the gated attention layer to extract the semantic focus vector of the Chinese electronic medical record text. The latent vector output from step 2 is used as the input vector and input into the gated attention network to focus on the information that is more critical to the current task from the overall perspective of the text and filter out information that is irrelevant to the current task. By applying the gated attention mechanism to the model, the model dynamically calculates the weights of words in the electronic medical record text, derives the importance of each word to different medical entities, filters out irrelevant information, and enhances the dynamic semantic focus capability of the model. Step 4: Use conditional random fields to fully consider the constraints and dependencies between medical entity labels, use the text semantic focus vector output in step 3 as the input vector, and input it into the CRF layer to predict the best medical entity word class label based on the contextual medical entity sentence level label information; Step 5: Use the trained GATTLCN joint network model to label the medical entity word classes of the test dataset; In step 2 of the second part, the Bi-LSTM layer uses the word vector X output in step 1 t As input; the LSTM neural network unit has a clever gate structure design for the hidden layer. It uses three logic gates, input gate, output gate, and forget gate to control the proportion of input data transferred to the memory unit and the proportion of forgetting the previous state, so as to extract the contextual semantic information of the medical record text; In order to simultaneously consider the contextual semantic information of the electronic medical record, a bidirectional LSTM is used to learn the hidden representation of the sentence from the global context information; In step 3 of part 2, the gated attention layer uses the latent vector h output from step 2 t As input; a gated attention mechanism is added to the model, so that the encoding modeling of text sequence relationships breaks through the long-distance limitation of electronic medical record text sequences, focuses on important semantic information from the overall perspective of the text, and filters out unimportant information; the gated attention mechanism is applied to the model, so that the model dynamically calculates the weights of words in the electronic medical record text, derives the importance of each word to different medical entities, and filters out irrelevant information; The gated attention network uses an auxiliary gate structure to dynamically select a subset of elements that need to be focused on, calculates their attention weights, and then calculates the weighted average of the corresponding element latent vectors based on their attention weights; the gated attention network contains a series of auxiliary gate structures and a backbone attention network; the auxiliary gate structure generates a set of input-related binary gates to determine whether each word needs to be paid attention to in the current task; the backbone network is an attention network, which aggregates the latent vectors of the corresponding words by calculating the attention weight values ​​of the words in the selected important medical record text, and then calculates the cumulative sum of the selected latent vectors to obtain the text focus vector.

2. A method for labeling medical entity word classes in Chinese electronic medical records according to claim 1, characterized in that: In step 1 of the second part, we use the Chinese electronic medical records that have been segmented to train word vectors, and use the skip-gram model of word2vec to obtain low-dimensional dense word vectors of Chinese electronic medical records. The word vector dimension is set to 100 dimensions; each word vector is obtained as: X = {x1, x2, ..., x 100 }, the text vector of each sentence is: S = {X1,X2,……,X n }, where n is the length of each sentence and the dimension of each word vector is 100.

3. A method for labeling medical entity word classes in Chinese electronic medical records according to claim 1, characterized in that: In step 3 of the second part, the gated attention layer uses an auxiliary gate structure to dynamically select a subset of elements to focus on. In order to solve the problem that the binary gate function discretizes the variables and causes the function to be non-differentiable, the Gumbel Softmax method is used to make the discrete variable sampling process continuous, so that the normal back-propagation algorithm can be used to optimize the model.

4. The method for labeling medical entity word classes in Chinese electronic medical records according to claim 1, characterized in that: In step 4 of the second part, the CRF layer uses the text focus vector C output from step 3 t as input; the conditional random field can describe the mutual dependence between output labels through the transfer matrix, fully considering the constraints and dependencies between medical entity labels; when predicting the optimal medical entity class, based on the Viterbi algorithm, the maximum probability label sequence of each word is calculated as the result of the medical entity category recognition of the word, and the medical entity labeling is completed.

5. The method for labeling medical entity word classes in Chinese electronic medical records according to claim 1, characterized in that: In step 5 of the second part, the negative log-likelihood function is used as the loss function during the model training process, and the parameters are optimized by minimizing the loss function.

Images