Medical named entity recognition system and method based on RWLSTM model fusion

Abstract

The present invention relates to a medical named entity recognition system and method based on RWLSTM model fusion, which comprises a data preprocessing module for word segmentation, entity annotation, and dictionary construction of user questions; a text encoding module for converting text into understandable data types and calculation units; a model building module for constructing a model framework according to a task; an entity extraction module for extracting and classifying entities from information after operation and feature extraction through model construction; and a dictionary construction module for constructing a named entity corpus dictionary of medical records. The electronic medical record text is subjected to data preprocessing, the Chinese word segmentation module is used to segment the text, the entity annotation module is used to annotate the text, and then some erroneous and useless data are eliminated; the electronic medical record is cleaned by the data preprocessing module, which effectively reduces the time cost of model training; and the problem of named entity recognition in the field of electronic medical records in the medical field is solved.

Description

Technical Field

[0001] The present invention relates to a medical named entity recognition system and method based on RWLSTM model fusion. Background Art

[0002] At present, artificial intelligence technology is developing rapidly. It is urgent to use existing technologies to truly solve the problems in real life of human society in order to truly benefit from the technology. Electronic medical records are generated in the clinical treatment process. Named entities and entity relationships reflect the health status of patients and contain a large amount of medical knowledge closely related to the health status of patients. Therefore, their recognition and extraction are an important extension of information extraction and natural language processing research in the medical field. Judging whether a named entity is correctly recognized includes two aspects: whether the boundary of the entity is correct and whether the type of the entity is correctly labeled. However, the underlying structure of the language content is not necessarily continuous and may be complex. This leads to the inapplicability of the chain model in processing language structure. In the task of Chinese named entity recognition, word ambiguity still exists. In the case of failure to extract contextual information, it is more challenging to identify named entities with overlapping ambiguous strings or polysemy. Since almost all previous models on named entity recognition are language-specific, we are committed to building a model that can process longer text sequences by exploring the wired mode of neural networks. Compared with English, the Chinese named entity recognition task is more complicated and is more difficult due to factors such as word segmentation. In addition, current named entities, whether in English or Chinese contexts, are performed on general corpora and have little application in some professional fields, such as the medical field.

[0003] The main technical methods of named entity recognition are divided into: rule-based and dictionary-based methods, statistical-based methods, and a hybrid of the two. Rule-based methods mostly use linguistic experts to manually construct rule templates, and select features including statistical information, punctuation marks, keywords, demonstrative words and directional words, positional words (such as suffixes), central words, etc., with pattern and string matching as the main means. Most of these systems rely on the establishment of dictionaries and dictionaries; rule-based and dictionary-based methods are the earliest methods used in named entity recognition.

[0004] The statistical method uses manually annotated corpus for training. The annotation of corpus does not require extensive linguistic knowledge and can be completed in a relatively short time. The statistical machine learning based methods mainly include: Hidden Markov Model (HMM), Maximum Entropy (ME), Support Vector Machine (SVM), Conditional Random Fields (CRF), etc. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a medical named entity recognition system and method based on RWLSTM model fusion.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] The medical named entity recognition system based on RWLSTM model fusion is characterized by: including data preprocessing module, text encoding module, model building module, entity extraction module and dictionary construction module;

[0008] The data preprocessing module performs word segmentation, entity annotation, and dictionary construction on user questions;

[0009] The text encoding module converts the text into understandable data types and calculation units;

[0010] The model building module constructs the framework of the model according to the task;

[0011] The entity extraction module extracts entities and classifies the information after the operation of building the model and feature extraction;

[0012] The dictionary building module builds a named entity corpus dictionary of medical records.

[0013] Furthermore, in the above-mentioned medical named entity recognition system based on RWLSTM model fusion, the data preprocessing module includes a Chinese word segmentation module, an entity labeling module and a data cleaning module. The Chinese word segmentation module is used for Chinese word segmentation of text in electronic medical records, and the words in the text are segmented, and a word segmentation tool is used to obtain a string of vocabulary sequences; the entity labeling module is used for entity labeling, and the segmented words are labeled with entity labels using an entity labeling tool, so as to judge the type of entity according to the entity label when classifying the entity; the data cleaning module rechecks the correctness and validity of the labeled data after entity labeling, and eliminates data with incorrect entity labeling.

[0014] Furthermore, in the above-mentioned medical named entity recognition system based on RWLSTM model fusion, the text encoding module uses BiLSTM to encode the text data processed by the data preprocessing module to provide rich word boundary information; then, the forward and backward embeddings are connected as character-level word representations; finally, the word representation is combined with the dictionary to generate a comprehensive word representation vector.

[0015] Furthermore, in the above-mentioned medical named entity recognition system based on RWLSTM model fusion, the model modeling module includes a pre-training module and a feature extraction module; in the pre-training module, before the word vector enters the feature extraction, the character-level word vector sequence is combined with the dictionary to generate a comprehensive word representation vector, and the RWLSTM pre-training model is used to train and generate context-related word vectors; in the feature extraction module, the context-related word vectors generated by the pre-training model are operated to extract the features of entity categories and entity boundaries from the word vectors.

[0016] Furthermore, in the above-mentioned medical named entity recognition system based on RWLSTM model fusion, the entity extraction module performs entity extraction on the data after the pre-training module and the feature extraction module, and uses the conditional random field to map a sequence into another sequence through calculation. In named entity recognition, the sequence obtained through the conditional random field is the label of the named entity, and the label contains entity boundary and entity category information.

[0017] Furthermore, in the above-mentioned medical named entity recognition system based on RWLSTM model fusion, the dictionary construction module includes public data sets and data that meets the requirements crawled from the web, and the dictionary construction maintains a relative balance in the number of entity categories.

[0018] The present invention provides a medical named entity recognition method based on RWLSTM model fusion, comprising the following steps:

[0019] First, the text of the electronic medical record is subjected to data preprocessing;

[0020] Then, the text data after data preprocessing is BiLSTM encoded, and the generated character-level word vector sequence is combined with the dictionary. After the comprehensive word representation vector is entered into the RWLSTM pre-training model module for training, the context-related word vector is generated;

[0021] After that, the word vector is input into the feature extraction module for feature extraction, and entity boundary features and category features are extracted. The mathematical representation of features is a high-dimensional vector.

[0022] Secondly, the feature vector is input into the entity extraction module to extract the entities of the text; the entity extraction module inputs the probability of each entity category, and the category with the highest probability is the final output result;

[0023] Finally, the entities and entity categories are output as the final output and serve as the basis for the doctor's diagnosis.

[0024] Furthermore, in the above-mentioned medical named entity recognition method based on RWLSTM model fusion, the data preprocessing module performs Chinese word segmentation and named entity labeling on the electronic medical record text; the text encoding module uses BiLSTM encoding on the text data; the model building module inputs the word vector into the pre-training module to form a context-related word vector, and then enters the feature extraction module to extract the features of the word vector; the entity extraction module converts the extracted features into entity sequences; the dictionary construction module crawls relevant information and related public data sets from the Internet to provide the initial electronic medical record text data.

[0025] Furthermore, in the above-mentioned medical named entity recognition method based on RWLSTM model fusion, the Chinese word segmentation module of the data preprocessing module performs Chinese word segmentation on the text in the electronic medical record, segments the words in the text, and uses a word segmentation tool to obtain a string of vocabulary sequences; the entity labeling module of the data preprocessing module labels the entities, and uses an entity labeling tool to label the segmented words with entity labels, so that the type of the entity can be determined according to the entity label when the entity is classified;

[0026] The text encoding module uses BiLSTM encoding to process text data into understandable and processable encoding sequences. It uses memory gates and forget gates to calculate word vectors and capture the information and features of the previous and next words in the text sequence to learn the boundaries of entities and long-distance dependencies. The calculation of memory gates and forget gates is as follows:

[0027] Calculate the forget gate. To reduce the amount of calculation, some information needs to be forgotten. Its input is the word vector h at the previous moment. t-1 , the input word vector at the current moment is x t , w f is the weight matrix of this layer used to remove some redundant information, b f is the bias matrix of this layer, σ is the activation function, and the final output is the value of the forget gate f t , which is the word vector after forgetting some information, substitute it into formula (4);

[0028] f t =σ(w f ·[h t-1 , x t +b f ]) (1)

[0029] Calculate the memory gate. In addition to the forget gate, important information needs to be retained. The input is the word vector h at the previous moment. t-1 , the input word vector h at the current moment t-1 , W i is the weight matrix of this layer, and then outputs the value i of the memory gate t, temporary cell state It stores the word vector that needs to be remembered at the current moment, and then substitutes it into formula (4);

[0030] i t =σ(W i ·[h t-1 ,x t ]+b i ) (2)

[0031]

[0032] Calculate the word vector stored in the temporary state at the current moment, and input the word vector i of the memory gate t and the value of the forget gate f t As coefficients, the word vectors of the temporary cell state The word vector C stored in the cell at the last moment t-1 , and perform matrix multiplication, the output is the word vector in the current cell state is C t ;

[0033]

[0034] Calculate the output gate and the word vector at the current moment, and input the word vector h at the previous moment t-1 , the input word x at the current moment t , the word vector C at the current moment t , W o is the weight matrix of this layer, b o is the bias matrix of this layer, and then the tanh function is used for calculation; then there is the word vector value O of the output gate t , the hidden layer word vector is h t ;

[0035] O t =σ(W o ·[h t-1 , x t +b o ]) (5)

[0036] h t =O t *tanh(C t ) (6)

[0037] Calculate n rounds to get a word vector sequence {h0, h1, h2...h t-1}; Finally, the word vector sequence is combined with the dictionary to generate a comprehensive word representation vector;

[0038] The pre-training module of the model building module combines the character-level word vector sequence with the dictionary to generate a comprehensive word representation vector, and uses the RWLSTM pre-training model to train and generate context-related word vectors; the feature extraction module of the model building module operates on the context-related word vectors generated by the pre-training model, and extracts the features of entity categories and entity boundaries from the word vectors;

[0039] The feature extraction module in the model modeling module uses Global Attention to perform an attention mechanism on the representation to find global information; the attention mechanism in the transformer model is a function that maps important and relevant words from the input sentence and assigns higher weights to the words; its calculation formula is (7);

[0040]

[0041] V is a vector representing input features, Q and K are feature vectors for calculating Attention weights, which are obtained from input features; Attention (Q, K, V) is to multiply V by the corresponding weights according to the degree of attention. Q, K, and V in the Attention mechanism calculate the similarity between the current Query and all Keys, and pass the similarity values ​​through the Softmax layer to obtain a set of weights. The Value value under Attention is obtained by summing the product of the weights and the corresponding Values; the word vector sequence obtained is then input into the entity extraction module for entity extraction;

[0042] The entity extraction module extracts entities from the data after being trained by the model building module, including entity boundaries and entity categories, and uses the conditional random field algorithm to extract entities;

[0043] The dictionary building module includes public electronic medical record datasets and data collected and processed by web crawlers.

[0044] Compared with the prior art, the present invention has significant advantages and beneficial effects, which are specifically embodied in the following aspects:

[0045] ① Based on the medical named entity recognition system fused with the RWLSTM model, the electronic medical record text is preprocessed. The Chinese word segmentation module segments the text, and the entity annotation module annotates the text. Then, some errors and useless data are eliminated. The data preprocessing module cleans the electronic medical record data, effectively reducing the time cost of model training.

[0046] ② A new model fusion mode replaces the traditional method based on dictionary and statistical method. This invention innovatively solves the problem of named entity recognition in the field of electronic medical records in the medical field, and has a higher entity recognition accuracy rate on general data sets than the existing technology.

[0047] ③ Carry out practical research in the medical field with extremely high application value and improve the electronic medical record corpus, which is pioneering and improves the accuracy of entity recognition.

[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by practicing the specific embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 : Schematic diagram of the architecture principle of the system of the present invention;

[0051] Figure 2 : Schematic diagram of the process of the present invention;

[0052] Figure 3 : Schematic diagram of the architecture principle of the data preprocessing module;

[0053] Figure 4 : Schematic diagram of the architecture principle of the text encoding module;

[0054] Figure 5 : Schematic diagram of the architecture principle of the model building module;

[0055] Figure 6 :Schematic diagram of RWLSTM generation process;

[0056] Figure 7 :RWLSTM algorithm;

[0057] Figure 8 :Schematic diagram of comparison of related model effects. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0059] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, directional terms and order terms are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0060] like Figure 1 As shown, the medical named entity recognition system based on RWLSTM model fusion is characterized by: comprising a data preprocessing module 1, a text encoding module 2, a model building module 3, an entity extraction module 4 and a dictionary construction module 5;

[0061] Data preprocessing module 1: word segmentation, entity annotation, and dictionary construction of user questions;

[0062] Text encoding module 2, converting text into understandable data types and calculation units;

[0063] Model building module 3, builds the framework of the model according to the task;

[0064] Entity extraction module 4, extracts entities and classifies the information after the operation of building the model and feature extraction;

[0065] The dictionary construction module 5 constructs a named entity corpus dictionary of medical records.

[0066] like Figure 3 The data preprocessing module 1 includes a Chinese word segmentation module 101, an entity labeling module 102 and a data cleaning module 103. The Chinese word segmentation module 101 is used for Chinese word segmentation of the text in the electronic medical record, and the words in the text are segmented, and a word segmentation tool is used to obtain a string of vocabulary sequences; the entity labeling module 102 is used for entity labeling, and the entity labeling tool is used to label the segmented words with entity labels, so that the type of entity can be determined according to the entity label during entity classification; the data cleaning module 103 rechecks the correctness and validity of the labeled data after entity labeling, and eliminates the data with incorrect entity labeling.

[0067] like Figure 4 ,Text encoding module 2, uses BiLSTM to encode the text data processed by data preprocessing module 1 to provide rich word boundary information; then, the forward and backward embeddings are connected as character-level word representation; finally, the word representation is combined with the dictionary to generate a comprehensive word representation vector.

[0068] like Figure 5 The model building module includes a pre-training module and a feature extraction module; in the pre-training module, before entering the feature extraction, the word vector is encoded with the text and combined with the comprehensive word representation vector generated by the dictionary module to use the RWLSTM pre-training model to train and generate context-related word vectors; in the feature extraction module, the context-related word vectors generated by the pre-training model are calculated to extract the features of entity categories and entity boundaries from the word vectors.

[0069] The entity extraction module 4 performs entity extraction on the data after the pre-training module 301 and the feature extraction module 302, and uses the conditional random field to map a sequence into another sequence through calculation. In named entity recognition, the sequence obtained through the conditional random field is the label of the named entity, and the label contains entity boundary and entity category information.

[0070] The dictionary construction module 5 contains public data sets and data that meets the requirements from web crawling. The dictionary construction maintains a relative balance in the number of entity categories.

[0071] The present invention provides a medical named entity recognition method based on RWLSTM model fusion, comprising the following steps:

[0072] First, the text of the electronic medical record is subjected to data preprocessing;

[0073] Then, the text data after data preprocessing is BiLSTM encoded, and the generated character-level word vector sequence is combined with the dictionary. After the comprehensive word representation vector is entered into the RWLSTM pre-training model module for training, the context-related word vector is generated;

[0074] After that, the word vector is input into the feature extraction module for feature extraction, and entity boundary features and category features are extracted. The mathematical representation of features is a high-dimensional vector.

[0075] Secondly, the feature vector is input into the entity extraction module to extract the entities of the text; the entity extraction module inputs the probability of each entity category, and the category with the highest probability is the final output result;

[0076] Finally, the entities and entity categories are output as the final output and serve as the basis for the doctor's diagnosis.

[0077] like Figure 2 The data preprocessing module 1 performs Chinese word segmentation and named entity annotation on the electronic medical record text; the text encoding module 2 uses BiLSTM encoding on the text sequence; the model building module 3 inputs the word vector into the pre-training module to form a context-related word vector, and then enters the feature extraction module to extract the features of the word vector; the entity extraction module 4 converts the extracted features into entity sequences; the dictionary construction module 5 crawls relevant information and related public data sets from the Internet to provide the initial electronic medical record text data.

[0078] The Chinese word segmentation module 101 of the data preprocessing module 1 performs Chinese word segmentation on the text in the electronic medical record, segments the words in the text, and uses a word segmentation tool to obtain a string of vocabulary sequences; the entity tagging module 102 of the data preprocessing module 1 tags the entities, and uses an entity tagging tool to tag the segmented words with entity tags, so that the type of the entity can be determined according to the entity tag during entity classification;

[0079] Text encoding module 2 uses BiLSTM to process text encoding into understandable and processable encoding sequences. It uses memory gates and forget gates to calculate word vectors and capture the information and features of the previous and next words in the text sequence to learn the boundaries of entities and long-distance dependencies. The calculation of memory gates and forget gates is as follows:

[0080] Calculate the forget gate. To reduce the amount of calculation, some information needs to be forgotten. Its input is the word vector h at the previous moment. t-1 , the input word vector at the current moment is x t , W f is the weight matrix of this layer used to remove some redundant information, b f is the bias matrix of this layer, σ is the activation function, and the final output is the value of the forget gate f t , which is the word vector after forgetting some information, substitute it into formula (4);

[0081] f t =σ(w f ·[h t-1 , x t +b f ]) (1)

[0082] Calculate the memory gate. In addition to the forget gate, important information needs to be retained. The input is the word vector h at the previous moment. t-1 , the input word vector h at the current moment t-1 , W i is the weight matrix of this layer, and then outputs the value i of the memory gate t , temporary cell state It stores the word vector that needs to be remembered at the current moment, and then substitutes it into formula (4);

[0083] i t =σ(W i ·[h t-1 ,x t ]+b i ) (2)

[0084]

[0085] Calculate the word vector stored in the temporary state at the current moment, and input the word vector i of the memory gate t and the value of the forget gate f t As coefficients, the word vectors of the temporary cell state The word vector C stored in the cell at the last moment t-1 , and perform matrix multiplication, the output is the word vector in the current cell state is C t ;

[0086]

[0087] Calculate the output gate and the word vector at the current moment, and input the word vector h at the previous moment t-1 , the input word x at the current moment t , the word vector C at the current moment t , W o is the weight matrix of this layer, b o is the bias matrix of this layer, and then the tanh function is used for calculation; then there is the word vector value O of the output gate t , the hidden layer word vector is h t ;

[0088] O t =σ(W o ·[h t-1 , x t +b o ]) (5)

[0089] h t =O t *tanh(C t ) (6)

[0090] After n rounds of calculation, we get a word vector sequence {h0, h1, h2...h n-1}; Finally, the word vector sequence is combined with the dictionary to generate a comprehensive word representation vector;

[0091] The pre-training module 301 of the model building module 3 combines the character-level word vector sequence with the dictionary to generate a comprehensive word representation vector, and uses the RWLSTM pre-training model to train and generate context-related word vectors; the feature extraction module 302 of the model building module 3 performs operations based on the context-related word vectors generated by the pre-training model, and extracts the features of entity categories and entity boundaries from the word vectors;

[0092] The feature extraction module 302 in the model building module 3 uses Global Attention to perform an attention mechanism on the representation (after calculation by the RWLSTM model) to find global information. The attention mechanism in the transformer model is a function that maps important and relevant words from the input sentence and assigns higher weights to these words; its calculation formula is (7);

[0093]

[0094] V is a vector representing input features, Q and K are feature vectors for calculating Attention weights, all of which are obtained from input features; Attention (Q, K, V) is to multiply V by the corresponding weight according to the degree of attention; Q, K, V in the Attention mechanism calculates the similarity between the current Query and all Keys, and this similarity value is passed through the Softmax layer to obtain a set of weights, and the Value value under Attention is obtained by summing the product of this set of weights and the corresponding Value; Since the RWLSTM process enhances this function, the function has been maximized (such as context information and boundary information), resulting in the amplification of some useless information, so this mechanism is used to enhance important features and reduce redundancy; The final word vector sequence is then input into the entity extraction module for entity extraction;

[0095] Entity extraction module 4, extracts entities from the data after being trained by the model building module, including entity boundaries and entity categories, and uses a conditional random field algorithm to extract entities;

[0096] The dictionary building module 5 includes the public electronic medical record data set and the data collected and processed by the web crawler.

[0097] like Figure 6, RWLSTM generation process, RWLSTM is a bidirectional LSTM is an atypical block-level mode, composed of a forward LSTM layer and a backward LSTM layer. However, due to the hierarchical structure of the language, the bidirectional LSTM is not enough to extract deep context information. The Watts Strogatz (WS) model is a random graph generation model, which is defined to generate a small "world graph". This model uses it to generate the lines between LSTM nodes, and these nodes constitute a directed acyclic graph (DAG). The present invention proposes a block-level random connection pattern generated based on the Random graph model (WS) model to adapt to named entity recognition and define the operations between message exchange nodes and edges, which can extract and enhance features. The edges between nodes are randomly generated, and the wiring pattern of the neural network is mainly modeled using a graph structure. The process of word representation is explained by taking four nodes a, b, c and d as an example. In the input node, the same word representation is copied three times and passed to three adjacent nodes (including nodes a and b). Then, the "LSTM node" performs a message exchange process (message exchange includes node and edge operations) according to the characteristics of the previous adjacent nodes. Finally, all original output nodes are concatenated and a unique node output is created;

[0098] like Figure 7 , RWLSTM algorithm, (aggregation) aggregation → (update) update → (aggregation) aggregation → ..., is an effective graph message passing architecture. The message exchange between the nodes of the graph is forward and efficient, and can combine the features of the same text sequence in different states between nodes. The (aggregation) aggregation operation combines the input data (from one or more edges) into one node through a weighted sum, and the state of the target node For adjacent nodes Connect to previous state Its calculation formula is (8); on this basis, the transformation operation is added, and its calculation formula is (9); where h represents the characteristics of the node, and the subscripts v and u are the indexes of the nodes. The superscript k is the number of layers, σ is the activation function, and w k and B k is the training matrix, N(v) is the set of adjacent nodes, AGG(.) is the aggregation operation, and Trans(.) is the transformation operation.

[0099]

[0100] like Figure 8For example, the case sample is: "Patient Zhang Feng had pain in the lower left chest in July 2021, without fever, skin itching, fatigue and other symptoms. After not paying attention, the tumor in the lower left chest wall gradually increased. The patient visited this hospital in January 2022." The disease and diagnosis results finally output by the traditional CNN model and BERT model are not accurate and the factual results are quite different, and the results are unsatisfactory. The RWLSTM model of the present invention has high output accuracy, which greatly facilitates doctors to diagnose the disease.

[0101] In specific applications, the text of the electronic medical record is preprocessed, the text data is segmented, and the entities are labeled after segmentation, and then the data is cleaned to remove some erroneous and useless data; the text is input into the text encoding module for BiLSTM encoding and the generated character-level word vector sequence is combined with the dictionary, and the comprehensive word representation vector is input into the RWLSTM pre-training model for training to generate context-related word vectors; secondly, the word vector is input into the feature extraction module, namely Global Attention, for feature extraction to obtain entity boundary features and category features; finally, the features are input into the entity extraction model, namely the conditional random field, to extract the entities of the text (diseases and diagnoses, drugs, etc.), and the entities are classified according to the categories to which the entities belong; and the entities and entity categories are output as the final output to the doctors as a basis for diagnosis.

[0102] In summary, the present invention proposes a medical named entity recognition system based on the RWLSTM model fusion based on the pre-training model, performs data preprocessing on the electronic medical record text, the Chinese word segmentation module performs word segmentation on the text, the entity annotation module annotates the text, and then some erroneous and useless data are eliminated; the electronic medical record data is cleaned by the data preprocessing module, which effectively reduces the time cost of model training; the new model fusion mode replaces the traditional method based solely on dictionaries and statistical methods; the present invention innovatively solves the problem of named entity recognition in the field of electronic medical records in the medical field, and has a higher entity recognition accuracy on general data sets than the existing technology; it is pioneering to carry out practical research in the medical field with extremely high application value and improve the electronic medical record corpus, which improves the accuracy of entity recognition.

[0103] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of rights of the present invention. At the same time, the above description should be understandable and implementable for professionals in the relevant technical field. Therefore, other equivalent changes or modifications that do not deviate from the spirit disclosed by the present invention should be included in the scope of the patent application.

Claims

1. A medical named entity recognition system based on RWLSTM model fusion, characterized by: It includes a data preprocessing module (1), a text encoding module (2), a model building module (3), an entity extraction module (4) and a dictionary building module (5); The data preprocessing module (1) performs word segmentation, entity annotation, and dictionary construction on user questions; The text encoding module (2) converts the text into understandable data types and calculation units; The model building module (3) constructs a model framework according to the task; The entity extraction module (4) extracts entities and classifies the information after the operation of building the model and extracting the features; The dictionary building module (5) builds a named entity corpus dictionary of medical records; The model building module (3) comprises a pre-training module (301) and a feature extraction module (302); the pre-training module (301) encodes the text before the word vector enters the feature extraction module, and then uses the RWLSTM pre-training model to train the comprehensive word representation vector generated by the dictionary module to generate context-related word vectors; the feature extraction module (302) performs operations on the context-related word vectors generated by the pre-training model to extract the features of entity categories and entity boundaries from the word vectors; The RWLSTM pre-training model uses a random connection algorithm in graph theory to connect multiple LSTM blocks to each other; uses the Watts Strogatz model to generate connections between LSTM nodes, and the nodes form a directed acyclic graph DAG; in the input nodes, the same word representation is copied and passed to adjacent nodes; the LSTM nodes perform a message exchange process according to the characteristics of the previous adjacent nodes; all original output nodes are connected in series and a unique node output is created.

2. The medical named entity recognition system based on RWLSTM model fusion according to claim 1, characterized in that: The data preprocessing module (1) comprises a Chinese word segmentation module (101), an entity labeling module (102) and a data cleaning module (103). The Chinese word segmentation module (101) is used for Chinese word segmentation of text in electronic medical records, segmenting words in the text, and using a word segmentation tool to obtain a string of vocabulary sequences; the entity labeling module (102) is used for entity labeling, and uses an entity labeling tool to label the segmented words with entity labels so that the type of entity can be determined according to the entity label during entity classification; the data cleaning module (103) rechecks the correctness and validity of the labeled data after entity labeling, and removes data with incorrect entity labeling.

3. The medical named entity recognition system based on RWLSTM model fusion according to claim 1, characterized in that: The text encoding module (2) uses BiLSTM to encode the text data processed by the data preprocessing module (1) to provide rich word boundary information; then, the forward and backward embeddings are connected as character-level word vector representations; finally, the character-level word vector sequence is combined with a dictionary to generate a comprehensive word representation vector.

4. The medical named entity recognition system based on RWLSTM model fusion according to claim 1, characterized in that: The entity extraction module (4) performs entity extraction on the data after the pre-training module (301) and the feature extraction module (302), and uses a conditional random field to map a sequence into another sequence through calculation. In named entity recognition, the sequence obtained through the conditional random field is the label of the named entity, and the label contains entity boundary and entity category information.

5. The medical named entity recognition system based on RWLSTM model fusion according to claim 1, characterized in that: The dictionary construction module (5) includes a public data set and data that meets the requirements obtained by web crawling. The dictionary construction maintains a relative balance in the number of entity categories.

6. A medical named entity recognition method based on RWLSTM model fusion, characterized by: The following steps are involved: First, the text of the electronic medical record is subjected to data preprocessing; Then, the text data after data preprocessing is BiLSTM encoded, and the generated character-level word vector sequence is combined with the dictionary. After the comprehensive word representation vector is entered into the RWLSTM pre-training model module for training, the context-related word vector is generated; After that, the word vector is input into the feature extraction module for feature extraction, and entity boundary features and category features are extracted. The mathematical representation of features is a high-dimensional vector. Secondly, the feature vector is input into the entity extraction module to extract the entities of the text; the entity extraction module inputs the probability of each entity category, and the category with the highest probability is the final output result; Finally, the entities and entity categories are output as the final output, which serves as the basis for the doctor’s diagnosis; The RWLSTM pre-training model uses a random connection algorithm in graph theory to connect multiple LSTM blocks to each other; uses the Watts Strogatz model to generate connections between LSTM nodes, and the nodes form a directed acyclic graph DAG; in the input nodes, the same word representation is copied and passed to adjacent nodes; the LSTM nodes perform a message exchange process according to the characteristics of the previous adjacent nodes; all original output nodes are connected in series and a unique node output is created.

7. The medical named entity recognition method based on RWLSTM model fusion according to claim 6, characterized in that: The data preprocessing module (1) performs Chinese word segmentation and named entity annotation on the electronic medical record text; the text encoding module (2) uses BiLSTM encoding on the text sequence; the model building module (3) inputs the word vector into the pre-training module to form a context-related word vector, and then the feature extraction module extracts the features of the word vector; the entity extraction module (4) converts the extracted features into entity sequences; and the dictionary construction module (5) crawls relevant information and related public data sets from the Internet to provide the initial electronic medical record text data.

8. The medical named entity recognition method based on RWLSTM model fusion according to claim 6, characterized in that: The Chinese word segmentation module (101) of the data preprocessing module (1) performs Chinese word segmentation on the text in the electronic medical record, segments the words in the text, and uses a word segmentation tool to obtain a string of vocabulary sequences; the entity labeling module (102) of the data preprocessing module (1) labels the entities, and uses an entity labeling tool to label the segmented words with entity labels, so that the type of the entity can be determined according to the entity label when the entity is classified; The text encoding module (2) uses BiLSTM encoding to process the text data into an understandable and processable encoding sequence; it uses memory gates and forget gates to calculate word vectors and capture the information and features of the previous and next words in the text sequence to learn the boundaries of entities and long-distance dependencies. The calculation of the memory gate and forget gate is as follows: Calculate the forget gate. To reduce the amount of calculation, some information needs to be forgotten. Its input is the word vector h at the previous moment. t-1 , the input word vector at the current moment is x t , W f is the weight matrix of this layer used to remove some redundant information, b f is the bias matrix of this layer, σ is the activation function, and the final output is the value of the forget gate f t , which is the word vector after forgetting some information, substitute it into formula (4); f t =σ(W f ·[h t-1 ,x t +b f ]) (1) Calculate the memory gate. In addition to the forget gate, important information needs to be retained. The input is the word vector h at the previous moment. t-1 , the input word vector h at the current moment t-1 , W i is the weight matrix of this layer, and then outputs the value i of the memory gate t , temporary cell state It stores the word vector that needs to be remembered at the current moment, and then substitutes it into formula (4); i t =σ(W i ·[h t-1 ,x t ]+b i ) (2) Calculate the word vector stored in the temporary state at the current moment, and input the word vector i of the memory gate t and the value of the forget gate f t As coefficients, the word vectors of the temporary cell state The word vector c stored in the cell at the last moment t-1 , and perform matrix multiplication, the output is the word vector in the current cell state is c t ; Calculate the output gate and the word vector at the current moment, and input the word vector h at the previous moment t-1 , the input word x at the current moment t , the word vector c at the current moment t , W o is the weight matrix of this layer, b o is the bias matrix of this layer, and then the tanh function is used for calculation; then there is the word vector value O of the output gate t , the hidden layer word vector is h t ; The t =σ(W o ·[h t-1 ,x t +b o ]) (5) h t =O t *tanh(C t ) (6) Calculate n rounds to get a word vector sequence {h0,h1,h2...h n-1 }; Finally, the word vector sequence is combined with the dictionary to generate a comprehensive word representation vector; The pre-training module (301) of the model building module (3) combines the character-level word vector sequence with the dictionary to generate a comprehensive word representation vector, and uses the RWLSTM pre-training model to train and generate context-related word vectors; the feature extraction module (302) of the model building module (3) performs operations based on the context-related word vectors generated by the pre-training model, and extracts the features of entity categories and entity boundaries from the word vectors; The feature extraction module (302) in the model building module (3) uses GlobalAttention to perform an attention mechanism on the representation to find global information; The attention mechanism in the transformer model is a function that maps important and relevant words from the input sentence and gives higher weights to the words; The calculation formula is (7); V is a vector representing input features, Q and K are feature vectors for calculating Attention weights, which are obtained from input features; Attention (Q, K, V) is to multiply V by the corresponding weights according to the degree of attention. Q, K, and V in the Attention mechanism calculate the similarity between the current Query and all Keys, and pass the similarity values ​​through the Softmax layer to obtain a set of weights. The Value value under Attention is obtained by summing the product of the weights and the corresponding Values; the word vector sequence obtained is then input into the entity extraction module for entity extraction; An entity extraction module (4) extracts entities from the data after being trained by the model building module, including entity boundaries and entity categories, and uses a conditional random field algorithm to extract entities; The dictionary building module (5) includes the public electronic medical record dataset and the data collected and processed by web crawlers.

Images