Traditional Chinese medicine medical record named entity recognition method and system based on multi-head attention mechanism

Abstract

The present disclosure provides a traditional Chinese medical record named entity recognition method based on a multi-head attention mechanism, comprising the following steps: obtaining text data of a traditional Chinese medical record; combining character vectors and word vectors in the obtained text data and sending them into a Bi-GRU neural network for feature extraction to obtain global features; using a multi-head attention mechanism to provide potential semantic information for the character vectors and extract local features; inputting the global features and the local features into a conditional random field layer to obtain a named entity sequence labeling result of the text data; the present disclosure does not require a word segmentation operation, combines the features of characters and words to form a joint feature, controls the weights of the characters and the words using a hyperparameter, inputs the joint feature into an embedding layer, and adds spatial attention in the Bi-GRU layer to make up for the deficiency in extracting effective features, greatly improving the accuracy of entity recognition.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of text data processing, in particular to a traditional Chinese medical record named entity recognition method and system based on a multi-head attention mechanism. BACKGROUND

[0002] The statements in this section merely provide background technology related to the present disclosure and do not necessarily constitute the prior art.

[0003] With the rapid development of traditional Chinese medicine informatization, the application of traditional Chinese medicine in major hospitals is also becoming more and more popular. Unlike other medical species, the most important thing in traditional Chinese medicine is traditional Chinese medical records. Traditional Chinese medical records are the carriers of the clinical practice experience and academic thoughts of generations of physicians. As the record of the clinical diagnosis and treatment experience of generations of traditional Chinese physicians, it contains the medical thoughts and experience of dialectical treatment of generations of physicians, and carries the inheritance and development of the traditional Chinese medicine knowledge system. Traditional Chinese medical records record a series of important information closely related to the health status of patients, such as diseases, symptoms and signs, and treatment methods. These information is an important experience and method in traditional Chinese medicine, and how to use traditional Chinese medical records to mine useful information is an important part of the development of traditional Chinese medicine. The text of traditional Chinese medical records is unstructured data and lacks a unified expression standard, which is a major factor affecting the secondary use of traditional Chinese medical records. Accurate recognition of entities in unstructured traditional Chinese medical record texts can provide effective decision support for patient diagnosis and treatment. In order to achieve this purpose, the task of traditional Chinese medical record text named entity recognition is used to automatically identify various named entities expressing independent meanings in traditional Chinese medical record texts, which includes determining entity boundaries and determining entity types.

[0004] By using computer technology to understand traditional Chinese medical record text information in an unstructured form, a latest named entity recognition model is designed using a deep learning-based method. For example, researchers have used attention mechanisms to conduct named entity recognition research on Chinese electronic medical records based on the combination of characters and words, and researchers have integrated language models and reading control gates in BLSTM-CRF for biomedical named entity recognition research. Compared with English texts, Chinese texts have the disadvantage of natural word segmentation, and incorrect Chinese word segmentation will lead to incorrect downstream work. Unlike processing English texts, most models that use word-level information must face the problem of correct Chinese text segmentation. Incorrect Chinese word segmentation results will lead to incorrect named entity recognition results. In addition, character-based models cannot fully utilize potential word and word sequence information. Therefore, multi-granularity information is needed to improve the performance of character-based models.

[0005] The inventors of the present disclosure have found that traditional methods based on machine learning include classification methods and sequence labeling methods. The sequence labeling method can jointly consider the labeling results of adjacent words, so it has received more attention. For example, some researchers have applied conditional random fields based on feature template sets to conduct named entity recognition research on traditional Chinese medicine medical records, and some researchers have used semi-Markov models to serialize and annotate named entities in English EMRs. In addition, maximum entropy models, support vector machines (SVMs) and other mainstream methods can also be used. However, the above methods require detailed and complex manual functions and are easily affected by sparse data. Summary of the Invention

[0006] In order to address the shortcomings of the existing technology, the present invention provides a method and system for named entity recognition in traditional Chinese medicine medical records based on a multi-head attention mechanism. Without word segmentation operation, a joint feature is formed by combining the features of characters and words, and the weights of characters and words are controlled using hyperparameters. The joint feature is input into the embedding layer, and spatial attention is added in the Bi-GRU layer, which makes up for its shortcomings in extracting effective features and greatly improves the accuracy of entity recognition.

[0007] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0008] The first aspect of the present disclosure provides a method for identifying named entities in traditional Chinese medicine medical records based on a multi-head attention mechanism.

[0009] A method for named entity recognition in traditional Chinese medicine medical records based on a multi-head attention mechanism includes the following steps:

[0010] Obtain text data of traditional Chinese medicine medical records;

[0011] The character vectors and word vectors in the acquired text data are combined and fed into the Bi-GRU neural network for feature extraction to obtain global features;

[0012] Utilize the multi-head attention mechanism to provide potential semantic information for character vectors and extract local features;

[0013] The global features and local features are input into the conditional random field layer to obtain the named entity sequence labeling results of the text data.

[0014] The second aspect of the present disclosure provides a named entity recognition system for traditional Chinese medicine medical records based on a multi-head attention mechanism.

[0015] A named entity recognition system for TCM medical records based on a multi-head attention mechanism, including:

[0016] The data acquisition module is configured to: acquire text data of traditional Chinese medicine medical records;

[0017] The global feature acquisition module is configured to combine the character vector and the word vector in the obtained text data and input into a Bi-GRU neural network for feature extraction to obtain global features.

[0018] The local feature acquisition module is configured to use a multi-head attention mechanism to provide potential semantic information for the character vector and extract local features.

[0019] The entity sequence recognition module is configured to input the global features and the local features into a conditional random field layer to obtain a named entity sequence labeling result of the text data.

[0020] The third aspect of the present disclosure provides a computer-readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps in the method for Chinese medical record named entity recognition based on a multi-head attention mechanism according to the first aspect of the present disclosure.

[0021] The fourth aspect of the present disclosure provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, the processor implementing the steps in the method for Chinese medical record named entity recognition based on a multi-head attention mechanism according to the first aspect of the present disclosure when executing the program.

[0022] Compared with the prior art, the present disclosure has the following beneficial effects:

[0023] 1. The method, system, medium or electronic device according to the present disclosure does not require a word segmentation operation, combines the features of characters and words to form a joint feature, controls the weights of characters and words using hyperparameters, inputs the joint feature into an embedding layer, and adds spatial attention in the Bi-GRU layer to make up for the shortcomings in extracting effective features, greatly improving the accuracy of entity recognition.

[0024] 2. The method, system, medium or electronic device according to the present disclosure uses a combination of characters and words, assigns different weights to each character, which can be regarded as an extension of the character-based model, integrates a word-based unit, controls information weights, and runs through the entire information flow, avoiding the influence of training data segmentation errors.

[0025] 3. The method, system, medium or electronic device according to the present disclosure applies a conditional random field layer to the hidden state output by the Bi-GRU to make predictions and obtain a globally optimal output sequence. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which form a part of the present disclosure, are intended to provide further understanding of the present disclosure, and the illustrative embodiments of the present disclosure and their descriptions serve to explain the present disclosure, and do not constitute improper limitations on the present disclosure.

[0027] Figure 1 A flowchart of a Chinese medical record named entity recognition method based on a multi-head attention mechanism is provided for Embodiment 1 of the present disclosure.

[0028] Figure 2 A joint feature diagram combining characters and words is provided for Embodiment 1 of the present disclosure. DETAILED DESCRIPTION

[0029] The present disclosure is further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0031] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of the features, steps, operations, devices, components, and / or combinations thereof.

[0032] The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0033] Embodiment 1

[0034] Embodiment 1 of the present disclosure provides a Chinese medical record named entity recognition method based on a multi-head attention mechanism, specifically provides a Bi-GRU-CRF model structure that can be used for Chinese medical record text named entity recognition, which combines characters and words and adds an attention mechanism, as shown in Figure 1 .

[0035] The essence of the model structure proposed herein is as follows:

[0036] S1: The feature vector combining character vectors and word vectors is input into the Bi-GRU;

[0037] S2: The recurrent neural network of the Bi-GRU unit is used to extract features from the input sequence information, and finally the GRU results in two directions are connected to form a global feature output;

[0038] S3: A multi-head attention mechanism is used to provide potential semantic information for characters and extract local features;

[0039] S4: Global features and local features are sent to the CRF layer, which is the output layer of the model, to generate the sequence labeling results of the text.

[0040] In detail, the following steps are included:

[0041] In S1, the character vector and the word vector are combined to form a feature vector.

[0042] In the traditional Chinese medical record text: "for the treatment of coronary heart disease", "coronary heart disease" is a named entity, and the association between each character in the entity is closer, while the connection between "Yu" before the entity and "de" after the entity is relatively weak. Because the influence of each character on the entity is different, different weights need to be assigned to each character. This paper adopts the method of combining characters and words, as shown in Figure 2 .

[0043] The above combination method can be regarded as an extension of the character-based model, integrating word-based units and controlling information weights throughout the entire information flow, avoiding the impact of training data segmentation errors. When combining characters and words, this paper uses hyperparameters (0, 1) to control the combination ratio between characters and words, as shown in equations (1) and (2):

[0044]

[0045]

[0046] where o c represents the characters embedded in the general field table, represents the word vector, represents the character vector corresponding to c i , x i represents the feature vector combined by characters and words, and a and b represent hyperparameters.

[0047] The feature vector combined by the character vector and the word vector is input to the Bi-GRU layer for global feature extraction.

[0048] In S2, the Bi-GRU layer extracts sentence context features.

[0049] Since the bidirectional information in the sentence helps sequence modeling, the Bi-GRU network, which can capture the bidirectional information of the text, is used to extract the sentence context features.

[0050]

[0051]

[0052]

[0053] h s =[h1,h2,…,h m ] (6)

[0054] Among them, a <t-1>< / t-1> , a <t+1> Represents the hidden layer state of the current storage unit, h t and Represent the hidden states of the forward and backward storage networks at the i-th character position, y t Represents the combination of the hidden states in both directions of Bi-GRU. s is the result of Bi-GRU encoding the input sentence s, and m represents the total number of characters contained in the sentence s. In Bi-GRU, more semantic context features can be captured to generate global features.

[0055] However, when Bi-GRU extracts feature dimensions from text vectors, the number of feature dimensions becomes twice that of the hidden layer dimensions, and not all captured feature dimensions are useful. Therefore, it is difficult to extract more effective information using only the Bi-GRU network. This embodiment combines the attention mechanism to compensate for the insufficient local feature acquisition of the Bi-GRU neural network.

[0056] In S3, multi-head attention mechanism.

[0057] The attention mechanism selects information that is more critical to the current named entity recognition task from a large amount of information, and after recognition, devotes more attention resources to the target area that needs to be focused on.

[0058] By combining the Bi-GRU neural network with the attention mechanism, different weights are assigned to different words in the TCM medical case text, and then using context-based semantic association information to make up for the shortcomings of the Bi-GRU neural network in obtaining local features.

[0059] Traditional Chinese medicine medical case texts contain a large number of professional Chinese medicine terms with diverse word structures. This rich information can be parsed using a multi-head attention mechanism, which helps to analyze the complex content of traditional Chinese medicine medical case texts.

[0060] For the input TCM medical case text X=(x1,…,x t ,…x m ), we need to use formula (6) to calculate the output H=(h1,…,h t ,…h m ), for the current word x t , h t It is the output of the Bi-GRU neural network, which contains the information after neural encoding.

[0061] The hidden state single-head attention weight of the current input Chinese medicine text is trained, as shown in formula (7).

[0062]

[0063] wherein w Q , w k and w v are weight parameters to be trained, is a smoothing term, d is the dimension number of H t , and parameters are not shared between each single-head attention output.

[0064] The multi-head attention mechanism is obtained by connecting the single-head attention, as shown in formula (8).

[0065]

[0066] wherein w m is a training parameter, and h is the splicing number, which is set to 6 in the embodiment.

[0067] After obtaining the multi-head attention, it needs to be connected with the current hidden state output, and an activation layer is added for training, as shown in formula (9).

[0068]

[0069] wherein P=[e1,…,e t ,…,e m ] is the output of the multi-head attention combined Bi-GRU neural network, each column of P is composed of e t In order to calculate the dependency relationship of the output sequence on the global sequence, CRF is introduced to calculate the transition probability between each label.

[0070] In S4, named entity recognition is usually regarded as a sequence labeling task, and there is a strong dependency relationship between the output labels. In this paper, B, I, O, E, and S are used to label the Chinese medical record text, which respectively represent the beginning / middle / outside / end of a word and a character word.

[0071] If the label is “I-XX”, the label before the label must be “B-XX” or “I-XX”, and a conditional random field layer is applied on the hidden state h of the Bi-GRU output to make a prediction and obtain the globally optimal output sequence.

[0072] For the output of the Bi-GRU neural network, the calculation formula of the score s(X,x) of the medical record text X output label sequence x is defined as formula (10):

[0073]

[0074] In formula (10), A is a transition matrix, representing the probability of transitioning from all states to the next state, P is the output of the Bi-GRU neural network, P i,j represents the score of calculating the i-th word as the probability of the j label, m represents the number of label sequences x, x represents the predicted label of each word, x i represents the i-th word predicted label, X x represents a set of all possible label sequences of x.

[0075] Specifically, A is a transition matrix, used to represent the transition from label i to label j, the starting label is x1 and the ending label is x m , and the label category is denoted as k, then A can be represented as: A ∈ R (k+2)*(k+2) .

[0076] The sequence with the highest total score is predicted as the optimal sequence, as shown in formula (11).

[0077]

[0078] wherein, X x represents all possible label sequences of x.

[0079] The algorithm for recognizing the named entity of the traditional Chinese medical record text designed in this embodiment is shown as algorithm 1.

[0080] Algorithm 1: Traditional Chinese medical record named entity recognition method based on multi-head attention mechanism

[0081] Input: Traditional Chinese medical record text txt;

[0082] Output: Optimal output sequence label tagged_txt;

[0083] (1) Load the input traditional Chinese medical record text;

[0084] (2) txt = load(txt);

[0085] (3) Combine the character vector and the word vector to form a feature vector;

[0086] (4) Input to the Bi-GRU neural network;

[0087] (5) txt_words = Bi-GRU(wordsVector);

[0088] (6) Calculate the weight list by multi-head attention;

[0089] (7) MHead(att_words);

[0090] (8) obtain the output matrix;

[0091] (9) obtain the transition matrix;

[0092] (10) return the optimal output sequence label tagged_txt.

[0093] Embodiment 2:

[0094] Embodiment 2 of the present disclosure provides a traditional Chinese medical record named entity recognition system based on a multi-head attention mechanism, comprising:

[0095] A data acquisition module configured to acquire text data of a traditional Chinese medical record;

[0096] A global feature acquisition module configured to combine the character vectors and word vectors in the acquired text data and send them into a Bi-GRU neural network for feature extraction to obtain global features;

[0097] A local feature acquisition module configured to use a multi-head attention mechanism to provide potential semantic information for the character vectors and extract local features;

[0098] An entity sequence recognition module configured to input the global features and local features into a conditional random field layer to obtain a named entity sequence labeling result of the text data.

[0099] The working method of the system is the same as the traditional Chinese medical record named entity recognition method based on the multi-head attention mechanism provided in Embodiment 1.

[0100] Embodiment 3:

[0101] Embodiment 3 of the present disclosure provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the traditional Chinese medical record named entity recognition method based on the multi-head attention mechanism as described in Embodiment 1 of the present disclosure, the steps being:

[0102] Acquire text data of a traditional Chinese medical record;

[0103] Combine the character vectors and word vectors in the acquired text data and send them into a Bi-GRU neural network for feature extraction to obtain global features;

[0104] Use a multi-head attention mechanism to provide potential semantic information for the character vectors and extract local features;

[0105] Input the global features and local features into a conditional random field layer to obtain a named entity sequence labeling result of the text data.

[0106] The detailed steps are the same as the method for identifying Chinese medical record named entities based on the multi-head attention mechanism provided in Embodiment 1.

[0107] Embodiment 4

[0108] Embodiment 4 of the present disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the method for identifying Chinese medical record named entities based on the multi-head attention mechanism as described in Embodiment 1 of the present disclosure, and the steps are as follows:

[0109] obtaining text data of a Chinese medical record;

[0110] combining the character vector and the word vector in the obtained text data, and inputting the combined vector into a Bi-GRU neural network to perform feature extraction and obtain global features;

[0111] providing potential semantic information for the character vector by using the multi-head attention mechanism, and extracting local features;

[0112] inputting the global features and the local features into a conditional random field layer to obtain named entity sequence labeling results of the text data.

[0113] The detailed steps are the same as the method for identifying Chinese medical record named entities based on the multi-head attention mechanism provided in Embodiment 1.

[0114] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0115] The present disclosure is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one block or multiple blocks.

[0116] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flow or block Figure 1 one or more blocks or blocks specified in the flow.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flow or block Figure 1 one or more blocks or blocks specified in the flow.

[0118] Those of ordinary skill in the art can understand that all or part of the flow of the above-mentioned embodiment method can be completed by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the flow of the above-mentioned embodiment method. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0119] The above only describes the preferred embodiments of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art can make various modifications and changes to the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for named entity recognition in traditional Chinese medicine medical records based on a multi-head attention mechanism, characterized in that: The following steps are involved: Obtain text data of traditional Chinese medicine medical records; The character vectors and word vectors in the acquired text data are combined and fed into the Bi-GRU recurrent neural network to extract features from the input feature vectors. The GRU results in the two directions are connected to obtain the global feature output, which is specifically: Use the Bi-GRU network that captures bidirectional information of the text to extract sentence context features: h s =[h1,h2,…,h m ] Among them, a <t-1>< / t-1> , a <t+1> Represents the hidden layer state of the current storage unit, h t and Represent the hidden states of the forward and backward storage networks at the i-th character position, y t Represents the combination of hidden states in two directions of Bi-GRU; h s is the result of Bi-GRU encoding the input sentence s, where m represents the total number of characters in the sentence s. More semantic context features are captured in the Bi-GRU to generate global features. Character and word features are combined to form a joint feature, and hyperparameters are used to control the weights of characters and words. The joint feature is input to the embedding layer, and spatial attention is added in the Bi-GRU layer. A multi-head attention mechanism is used to provide potential semantic information for character vectors and extract local features. Specifically, the multi-head attention mechanism is as follows: The hidden state single-head attention weight of the currently input TCM text is trained; as shown in the formula: Among them, w Q 、w k and w v They are the weight parameters that need to be trained, is the smoothing term, d is H t The number of dimensions, H t is the output of the Bi-GRU neural network; the output of each single-head attention does not share parameters; The multi-head attention mechanism is obtained by connecting the single-head attention; as shown in the formula: Among them, w m is the training parameter, h is the number of splicing; Connect the obtained multi-head attention with the current hidden state output, add an activation layer for training, and obtain the output of the multi-head attention combined with the Bi-GRU neural network; Input global features and local features into the conditional random field layer to obtain the named entity sequence annotation results of the text data; The algorithm for named entity recognition in TCM medical records based on the multi-head attention mechanism is as follows: Input: TCM medical records text txt; Output: optimal output sequence label tagged_txt; (1) Load and input TCM medical records; (2) txt = load (txt); (3) Combine the character vector and word vector to form a feature vector; (4) Input to Bi-GRU neural network; (5)txt_words=Bi-GRU(wordsVector); (6) Multi-head attention calculates the weight list; (7)MHead(att_words); (8) Obtain the output matrix; (9) Obtain the transfer matrix; (10) Return the optimal output sequence tag tagged_txt.

2. The method for named entity recognition of TCM medical records based on a multi-head attention mechanism as claimed in claim 1, characterized in that: In the conditional random field layer, the sequence score is calculated based on the probability score of each word being any label and the transition matrix, and the sequence with the highest score is taken as the optimal sequence.

3. The method for named entity recognition of TCM medical records based on a multi-head attention mechanism as claimed in claim 1, characterized in that: When combining character vectors and word vectors, hyperparameters are used to control the ratio of characters to words.

4. The method for named entity recognition of TCM medical records based on a multi-head attention mechanism as claimed in claim 1, characterized in that: The first character, second character, third character, fourth character and fifth character are used to mark the TCM medical case text, which respectively represent the beginning, middle, outside, end and a single-character word.

5. A named entity recognition system for traditional Chinese medicine medical records based on a multi-head attention mechanism, characterized by: include: The data acquisition module is configured to: acquire text data of traditional Chinese medicine medical records; The global feature acquisition module is configured to combine the character vectors and word vectors in the acquired text data and feed them into the Bi-GRU recurrent neural network, perform feature extraction on the input feature vectors, and concatenate the GRU results in both directions to obtain the global feature output, specifically: Use the Bi-GRU network that captures bidirectional information of the text to extract sentence context features: h s =[h1,h2,…,h m ] Among them, a ＜t-1> , a <t+1> Represents the hidden layer state of the current storage unit, h t and Represent the hidden states of the forward and backward storage networks at the i-th character position, y t Represents the combination of hidden states in two directions of Bi-GRU; h s is the result of Bi-GRU encoding the input sentence s, where m represents the total number of characters in the sentence s. More semantic context features are captured in the Bi-GRU to generate global features. Character and word features are combined to form a joint feature, and hyperparameters are used to control the weights of characters and words. The joint feature is input to the embedding layer, and spatial attention is added in the Bi-GRU layer. The local feature acquisition module is configured to: use the multi-head attention mechanism to provide latent semantic information for the character vector and extract local features; The multi-head attention mechanism is specifically: The hidden state single-head attention weight of the currently input TCM text is trained; as shown in the formula: Among them, w Q 、w k and w v They are the weight parameters that need to be trained, is the smoothing term, d is H t The number of dimensions, H t is the output of the Bi-GRU neural network; the output of each single-head attention does not share parameters; The multi-head attention mechanism is obtained by connecting the single-head attention; as shown in the formula: Among them, w m is the training parameter, h is the number of splicing; Connect the obtained multi-head attention with the current hidden state output, add an activation layer for training, and obtain the output of the multi-head attention combined with the Bi-GRU neural network; The entity sequence recognition module is configured to: input global features and local features into the conditional random field layer to obtain the named entity sequence annotation results of the text data; The algorithm for named entity recognition in TCM medical records based on the multi-head attention mechanism is as follows: Input: TCM medical records text txt; Output: optimal output sequence label tagged_txt; (1) Load and input TCM medical records; (2) txt = load (txt); (3) Combine the character vector and word vector to form a feature vector; (4) Input to Bi-GRU neural network; (5)txt_words=Bi-GRU(wordsVector); (6) Multi-head attention calculates the weight list; (7)MHead(att_words); (8) Obtain the output matrix; (9) Obtain the transfer matrix; (10) Return the optimal output sequence tag tagged_txt.

6. A medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for named entity recognition of traditional Chinese medicine medical records based on a multi-head attention mechanism are implemented as described in any one of claims 1 to 4.

7. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps in the method for named entity recognition of traditional Chinese medicine medical records based on a multi-head attention mechanism are implemented as described in any one of claims 1 to 4.

Images