Word segmentation method and system for Chinese electronic medical records

Abstract

The present invention discloses a word segmentation method and system for Chinese electronic medical records. The main body of the method is a Chinese electronic medical record word segmentation model based on GNN-BiLSTM-CRF, which includes the following steps: data set construction and data processing. The Chinese electronic medical record data from the cooperative hospital are manually annotated to complete the construction of the Chinese electronic medical record word segmentation data set. The data in the constructed data set is converted into CoNLL format. The Chinese electronic medical record word segmentation model is trained. The Chinese electronic medical record model proposed in the present invention makes full use of the advantages of GNN and LSTM, and captures local features such as the word formation rules of medical terms and context sequence information and long-term features through the interaction of graph structure and chain structure to improve the quality of word segmentation. The domain dictionary is efficiently integrated into the word segmentation model through the graph neural network, and the application value of the dictionary is expanded by learning the word formation rules of terms, reducing the model's dependence on the data set and alleviating the word segmentation problem caused by the large number and increasing number of medical terms.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and is a deep learning-based word segmentation method for Chinese electronic medical records and a system constructed thereby; specifically, a GNN-BiLSTM-CRF-based model is used to segment Chinese electronic medical records. Background Art

[0002] The task of electronic medical record text word segmentation is an important part of medical natural language processing. It is the first step in building knowledge graphs for clinical auxiliary diagnosis and treatment and personal health information management. It has important application value. The quality of word segmentation affects the quality of the knowledge graph and ultimately affects the value of the data.

[0003] Research on Chinese word segmentation methods has been going on since the 1980s. There are roughly three types of methods: dictionary-based matching methods, statistical word segmentation methods based on statistics and language models, and deep learning methods based on neural networks.

[0004] The dictionary-based matching method was first proposed by Liu Yuan et al. in 1986. Subsequently, various matching methods, such as those based on size and direction, were developed. Dictionary matching offers fast and targeted word segmentation, requiring no training. However, it relies heavily on dictionaries, struggles with disambiguation, and performs poorly on unregistered words. Currently, it is rarely used solely for word segmentation; it is typically incorporated into a model to improve segmentation performance.

[0005] There are numerous word segmentation methods based on statistics and language models, including the mutual information model proposed by Sproat et al. in the 1990s, the maximum entropy model pioneered by Xue et al. in 2003, and the conditional random field model proposed by Peng et al. in 2004. These methods have achieved promising results in word segmentation, but they rely heavily on feature engineering, and the quality of feature definition directly affects the results.

[0006] Since Zheng et al. applied neural networks to Chinese text word segmentation in 2013, many scholars have begun researching Chinese word segmentation methods based on deep learning. In 2015, Chen et al. used LSTM, a variant of RNN, to solve the Chinese word segmentation problem, using the model's memory units to learn long-term dependencies in text information. Huang et al. combined neural networks with statistical models to propose the BI-LSTM-CRF model. These deep learning models have achieved promising results in word segmentation, but their drawbacks include a high dependence on datasets and limited adaptability across different fields.

[0007] Currently, Chinese word segmentation technology for general-purpose text has matured, achieving extremely high segmentation standards. For example, word segmentation tools like Jieba and PKUSEG have achieved precision and recall rates exceeding 90% on open-source datasets for general-purpose text. However, when processing specialized fields like electronic medical records, these tools, trained on general-purpose datasets, often produce incorrect segmentation errors due to the large number of specialized terms, the increasing number of new terms, and the unique text style.

[0008] Many scholars have also conducted research on the problem of word segmentation in Chinese electronic medical records and have achieved certain results. For example, Zhang Libang uses a semi-supervised learning-based approach to segment Chinese electronic medical records. He first uses a domain dictionary to segment the electronic medical records, then introduces a probability model to estimate the probability of word occurrence, and finally uses information entropy to transform the recognition of unregistered words into an optimization problem. The problem of goodness failure of traditional unsupervised word segmentation methods when processing electronic medical records is solved using a dynamic programming algorithm. Xia uses a dictionary to generate partially labeled data for Chinese clinical practice, and then trains a semi-supervised conditional random field model together with the annotated data to solve the problem that the scarcity of annotated data affects the quality of the model. Junjie Xing proposes minimizing the distribution distance of hidden representations between the source and target domains through a multi-task learning framework of adaptive multi-task transfer learning, adapting to each other and obtaining domain-invariant features. This idea of transfer learning is used to improve the word segmentation effect, effectively improving the problem of the model's heavy reliance on annotated data.

[0009] While these studies have achieved promising results, these models generally make limited use of dictionaries and rely heavily on datasets. Furthermore, electronic medical records are subject to privacy and specialized domains, leading to high costs for annotating datasets. Domain dictionaries are less expensive than annotating datasets, as medical terminology generally follows specific patterns in word formation. Therefore, efficiently learning these patterns through domain dictionaries would facilitate the recognition of new words and reduce reliance on datasets. Summary of the Invention

[0010] Based on the above analysis and combining the structural style of Chinese electronic medical record corpus and the characteristics of medical terminology, this paper proposes a Chinese electronic medical record word segmentation model based on GNN-BiLSTM-CRF. By learning domain dictionary knowledge and mining text information features, this model solves the problems in electronic medical record word segmentation and improves the quality of word segmentation. A Chinese electronic medical record word segmentation system is constructed based on this model, which is easy to use.

[0011] The present invention mainly includes two parts:

[0012] The Chinese electronic medical record word segmentation method is based on the GNN-BiLSTM-CRF Chinese electronic medical record word segmentation model, which includes the following steps:

[0013] Step 1: Dataset construction and data processing.

[0014] Manually annotate the Chinese electronic medical record data from the partner hospital to complete the construction of the Chinese electronic medical record word segmentation dataset. Convert the data in the constructed dataset into CoNLL format.

[0015] Step 2: Train the Chinese electronic medical record word segmentation model.

[0016] In step 2.1, extract part of the data processed in step 1 to generate a training set. Input the training set and dictionary into the embedding unit, use Word2vec to embed the characters and words into corresponding vectors, and input them into the next layer of the model's graph neural network (GNN) unit for calculation.

[0017] In step 2.2, a graph is constructed using the embedded character and word vectors, with the characters serving as nodes and the words as edges. This information transfer mechanism, based on a multi-head attention mechanism and gating, is then used to iteratively aggregate and update information to learn local features such as the morphological patterns of medical terminology. The node and edge features of the graph are updated with each iteration, and the feature vector of each node in the graph after the last iteration is input into the next layer of the model.

[0018] In step 2.3, the features of each node in the graph are input into the bidirectional long short-term memory network (LSTM) unit to model the contextual information of each word, capture sequence information and long-range dependency features, and mine the text information features of the Chinese electronic medical record corpus.

[0019] In step 2.4, the conditional random field (CRF) unit is used to jointly decode the output of the LSTM to predict the best label sequence, and finally the model predicts the label for each word.

[0020] In step 2.5, the negative log-likelihood function is used as the loss function to evaluate the degree of difference between the model's predicted value and the true value. The model is trained by minimizing the loss function to obtain the Chinese electronic medical record word segmentation model.

[0021] Based on this model, an online graphical system for Chinese electronic medical record word segmentation based on B / S architecture was constructed. The main functional modules of the system are as follows:

[0022] The front-end interface module allows access to the system's front-end interface through a browser. Input raw Chinese electronic medical record data via the corresponding input text boxes or file upload function. The word segmentation function in the interface transmits the input data to the system's data processing module for word segmentation preprocessing. The segmentation results are displayed in the interface and a download function is available.

[0023] The data processing module converts the front-end input data into the CoNLL format required by the Chinese electronic medical record word segmentation module and inputs it into the Chinese electronic medical record word segmentation module. The original input Chinese electronic medical record text is segmented according to the predicted label of each word output by the Chinese electronic medical record word segmentation module, and the segmentation results are returned to the front-end interface.

[0024] The Chinese electronic medical record word segmentation module inputs the data processed by the data processing module into the trained Chinese electronic medical record word segmentation model, and performs operations through the embedding unit, graph neural network unit, bidirectional long short-term memory network unit, and conditional random field unit in turn, and finally outputs the predicted label of each word, and then returns the result to the data processing module.

[0025] The present invention has the following advantages:

[0026] 1. The Chinese electronic medical record model proposed in this paper fully utilizes the advantages of GNN and LSTM, and captures local features such as the word formation rules of medical terms, context sequence information, and long-term features through the interaction of graph structure and chain structure, thereby improving the quality of word segmentation.

[0027] 2. Through graph neural networks, domain dictionaries are efficiently integrated into the word segmentation model. By learning the word formation rules of terms, the application value of the dictionary is expanded, the model's dependence on the dataset is reduced, and the word segmentation problem caused by the large number and increasing number of medical terms is alleviated.

[0028] 3. Use attention and gating mechanisms to help the model filter noise and improve the model's performance in Chinese electronic medical record word segmentation.

[0029] 4. Use the trained model to build a system. When faced with the task of Chinese electronic medical record word segmentation in the future, you can directly use the system to input the electronic medical record data to be processed into the system to obtain the word segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the structure of the Chinese electronic medical record word segmentation model based on GNN-BiLSTM-CRF

[0031] Figure 2 Schematic diagram of system functional modules

[0032] Figure 3 Schematic diagram of the system flow

[0033] Figure 4 Example diagram for the construction portion of the diagram in the model DETAILED DESCRIPTION

[0034] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings.

[0035] The schematic diagram of the Chinese electronic medical record word segmentation model structure in the present invention is as follows Figure 1 The specific implementation steps are as follows:

[0036] (1) Dataset construction and data processing

[0037] The dataset used in this study primarily originated from partner hospitals. To ensure the validity of the dataset, its annotation rules were based on the Chinese word segmentation standards developed by the Institute of Computational Linguistics at Peking University. Considering the subsequent structural requirements of knowledge graph construction, the dataset underwent five revisions by data mining professionals and hospital experts before being used for electronic medical record annotation. The annotation process was completed by field professionals, who were trained and educated on the established annotation rules before annotation. After annotation, repeated cross-validation was performed to resolve disagreements among annotators. The annotation process was based on the work of Cohen et al.

[0038] The original Chinese electronic medical record data segmentation is actually a sequence annotation at the word level in the model. The present invention uses the BMES sequence annotation method. The basic principle is that each word in the text is annotated with one of the four letters "B", "M", "E", and "S". Among them, "B" represents the beginning position of the word (begin), "M" represents the middle position of the word (middle), "E" represents the end part of the word (end), and "S" represents a single word (single). In this annotation method, from "B" to "E" represents a word, and "S" represents a word. In this way, by annotating each word, the text can be segmented. The goal of the text segmentation method research is to find the optimal annotation sequence. Therefore, the data needs to be converted into a suitable input format before being input into the model. This model uses the CoNLL format, each word and its mark are one line, separated by a tab in the middle, and sentences are separated by blank lines.

[0039] (2) Model construction

[0040] The model structure is as follows Figure 1 As shown, from bottom to top:

[0041] a) The embedding layer maps the text to be segmented into vectors and inputs them into the model. Word2vec is used for character embedding and word embedding, and fine-tuning is performed.

[0042] b) The GNN layer first constructs the graph, then uses iterative aggregation and update to extract and transmit feature information, and the final node feature information is output to the next layer model.

[0043] Graph construction

[0044] Each word in the text corresponds to a node in the graph. The edges in the graph represent the words in the dictionary that match the text, that is, the words in the dictionary that exist in the text. For example, if the input text is "nitroglycerin", and the dictionary matches it with the words "nitroglycerin", "glycerol", and "nitroglycerin", then the graph constructed based on this dictionary is as follows: Figure 4 shown.

[0045] The four nodes in the graph are: c1 corresponds to "nitre", c2 corresponds to "acid", c3 corresponds to "sweet", and c4 corresponds to "oil". The edges in the graph are: 1,2 Corresponding to "nitric acid", e 1,4 Corresponding to "nitroglycerin", e 3,4 Corresponding to "glycerin".

[0046] Aggregation of graph neural networks

[0047] The graph neural network aggregation operation uses a multi-head attention mechanism, where the node feature set and edge feature sets e ij For all the words in the dictionary that match the text, that is, all the edges in the graph.

[0048] Each node c o Aggregate calculations go through the following three steps:

[0049] The first step is to determine node c o Feature information that needs to be aggregated Aggregate edge features for each node when clustering nodes and the characteristics of the corresponding connected nodes The boundary information of words can be effectively learned by extracting features of words and word beginnings and endings. The calculation is shown in formula (1):

[0050]

[0051] Among them: "[||]" represents the splicing operation, The feature dimension after splicing is 2F; r∈N o , N o Represents all with c o The node subscript set with edge connections, "|" represents or, that is, considering c at the same time o As the beginning and end of a word.

[0052] The second step is to calculate the node characteristics With aggregate features The attention coefficient α ro :

[0053]

[0054] Where: Wc and W g are all trainable weight matrices, where Perform linear transformation on the features; W o is the trainable weight matrix, LR stands for nonlinear activation using the LeakyReLU function;

[0055] The third step uses the multi-head mechanism to calculate the aggregated features

[0056]

[0057] Where: K is the number of heads in the multi-head attention mechanism; represents the attention coefficient of the normalized node feature and the corresponding aggregate feature calculated by the k-th attention mechanism; W k is a trainable weight matrix; perform weighted averaging on all heads to obtain the final output features

[0058] The final output feature set of node aggregation is

[0059] Edge ij Aggregation also goes through three steps:

[0060] The first step is to determine the edge e ij Feature information that needs to be aggregated When edge aggregation is performed, feature information of all nodes on each edge is aggregated, v∈N ij , N ij Represents the subscript set of all nodes from node i to node j.

[0061] The second step is to calculate the edge features With aggregate features The attention coefficient α iv :

[0062]

[0063] Where: W e and W v are all trainable weight matrices, where W e , W i is the trainable weight matrix,

[0064] The third step uses the multi-head mechanism to calculate the aggregated features

[0065]

[0066] Where: K is the number of heads in the multi-head attention mechanism; represents the attention coefficient of the normalized edge feature and the corresponding aggregate feature calculated by the k-th attention mechanism; W e is the trainable weight matrix.

[0067] The final output feature set of edge aggregation is

[0068] Updates on graph neural networks

[0069] The model uses a gating mechanism to control the flow of aggregated features to nodes and edges, thereby learning the local feature information obtained through aggregation. The following describes node updates and edge updates respectively.

[0070] Node Update:

[0071] Input node feature set at time t and node aggregation feature set Output time t+1

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Where W and V are both trainable parameter matrices, b is a trainable parameter vector; ⊙ is the Hadamard Product, the corresponding elements in the matrix are multiplied, and the two multiplied matrices are required to be of the same type; σ is the activation function sigmoid; Softmax normalizes the two gates; tanh is the activation function, which performs nonlinear changes; is the node feature of node o in the t+1 round of iteration.

[0078] First, calculate the two gates and And the feature information to be learned All by node features and aggregate features Calculated. Then use To control the forgetting feature Part of the information in Controlling learning features Finally, the node features of the next round are output

[0079] Side update:

[0080] Input edge feature set at time t The feature set obtained by edge aggregation Output edge feature set at time t+1

[0081]

[0082]

[0083]

[0084]

[0085]

[0086] Where W and V are trainable parameter matrices, and b is a trainable parameter vector; ⊙, σ, Softmax, and tanh are updated at the same node; For edge e ij Edge features in iteration t+1.

[0087] First, calculate the two gates and And the characteristic information to be transmitted Through edge features and aggregate features Calculated. Then use Control characteristics use Control characteristics Finally output the edge features of the next round

[0088] Each iteration of the entire graph neural network goes through a complete node and edge aggregation, as well as node and edge updates. After m rounds of iteration, the final output node feature set is As the output of the graph neural network.

[0089] c) The LSTM layer uses a bidirectional LSTM to capture past and future information respectively, concatenating the two results as the final output. Its input comes from the output of the graph neural network, and its specific calculation process is as follows:

[0090] First calculate the three gates And the characteristic information to be transmitted All by and output Calculated.

[0091]

[0092]

[0093]

[0094]

[0095] Where W and V are trainable parameter matrices, and b is a trainable parameter vector.

[0096] Gating Used to control the forgetting of memory features passed from the previous unit Part of the information in the gate Used to control learning features Part of the information in generates the current unit memory feature

[0097] And pass it to the next unit, as shown in formula (9):

[0098]

[0099] Will After using the tanh function for nonlinear activation, the gate Forget some of the information and get the output of the current unit And pass it to the next unit, as shown in formula (10):

[0100]

[0101] h in LSTM t Used to obtain feature information transmitted by nodes closer to the current node, C t It is used to record remote sequence information, one near and one far to efficiently capture context sequence information and remote dependencies.

[0102] d) The CRF layer models the label sequence of the entire sentence text through CRF, and predicts the corresponding state sequence based on the corresponding observation sequence. The score s(y|x) of the label sequence y is calculated as follows:

[0103]

[0104] Where: T is the matrix obtained by linear transformation of the encoding layer output, where the matrix elements Represents the current character x i The corresponding label is y iThe score when Z represents the transfer matrix, which is learned through training. The matrix elements Represents the previous label y i-1 Move to the current tab y i score.

[0105] After calculating the scores of all possible label sequences, we normalize them using the softmax function to obtain the conditional probability P(y|x) of the label sequence y:

[0106]

[0107] Where: Y(x) represents all possible label sequences.

[0108] (3) Model training

[0109] The predicted results are compared with the actual situation, and the negative log-likelihood function is used as the loss function, as shown in formula (13):

[0110]

[0111] Training is performed by minimizing the loss function.

[0112] During the prediction process, we look for the label sequence y with the highest conditional probability * :

[0113]

[0114] Use the Viterbi algorithm to solve. * That is, the label sequence that the model finally predicts the output.

[0115] (4) Model segmentation experimental results

[0116] a) Experimental conditions

[0117] Windows 10, Python, Pytorch framework, GeForce GTX 2080Ti graphics card.

[0118] b) Experimental parameters

[0119] In the experiment, the dimensions of character embedding and word embedding are both set to 50, the number of heads and dimension size of the multi-head attention in the GNN aggregation process are both set to 10, the number of GNN iterations is 2, the dimension of the LSTM layer is set to 128, the batch size is set to 8, the learning rate is set to 2e-4, and the Adam optimization algorithm is used. In order to reduce overfitting, dropout random deactivation is used, the embedding layer ratio is 0.5, the aggregation layer ratio is 0.2, and the LSTM layer ratio is 0.2.

[0120] c) Performance comparison

[0121] We used the most popular word segmentation tools, Jieba and PKUSEG, for comparison. PKUSEG supports loading pre-trained word segmentation models. Since the dataset is in the medical field, PKUSEG was loaded with models from the medicine domain. Since both tools support loading dictionaries, we conducted four comparative experiments using these two tools.

[0122] At the same time, the most mainstream sequence labeling model BI-LSTM-CRF and the latest technical model AMTTL proposed by Xing et al. for Chinese medical text segmentation are used as baseline models.

[0123] Experiments were conducted using the Chinese electronic medical record word segmentation dataset EMRs constructed in this patent. The results are shown in Table 1. It can be seen that this method is superior to the best electronic medical record word segmentation model and the most mainstream word segmentation tool in the past.

[0124] Table 1 Main experimental results of the EMRs dataset

[0125]

[0126]

[0127] To better verify the effectiveness of the model, this method was applied to the dataset Forum from Good Doctor Online, a Chinese medical consultation forum, published in the paper by Xing et al. It can be seen that in both datasets, the performance of this patented model is better than other word segmentation models and tools.

[0128] Table 2 Main experimental results of the Forum dataset

[0129]

[0130] The flowchart of the Chinese electronic medical record word segmentation system of the present invention is as follows Figure 3 As shown, the system is built using language frameworks such as Java, SpringBoot, Bootstrap, Python, and PyTorch. It is based on a B / S architecture, i.e., a browser / server architecture. The browser is the interface for using the system. Through the browser interface, data can be transmitted or requests can be made to the server, and the results returned by the server can be processed and displayed. The server is used to process data and return results. Using a B / S architecture makes the system more versatile, less dependent on the environment, and easier to use. The main steps of the system are as follows:

[0131] (1) In the browser, you can input the original electronic medical record data through the input text box in the interface or the file upload function, make a request to the server, and perform word segmentation on the input data.

[0132] (2) The server converts the input data transmitted from the browser into data in the CoNLL format (BMES annotation) that conforms to the model input.

[0133] (3) Input the format-converted data into the trained model and use the Viterbi algorithm to predict its optimal labeling sequence.

[0134] (4) Segment the text according to the optimal annotation sequence output by the model

[0135] (5) The server returns the segmented electronic medical record data, displays it in the browser's output text box, and can be exported and downloaded

[0136] In summary, this paper proposes a model and system for Chinese electronic medical record word segmentation. This model leverages the strengths of different networks by integrating a graph neural network (GNN), a long short-term memory network (LSTM), and a conditional random field (CRF) based on an attention mechanism and a gating mechanism. This effectively solves the problem of Chinese electronic medical record word segmentation and demonstrates high accuracy through experiments. By encapsulating the trained model into a BS-based system, it can be used more conveniently and efficiently for Chinese electronic medical record word segmentation tasks. Word segmentation can be performed simply by inputting data in a browser, making it easy to use.

Claims

1. A word segmentation method for Chinese electronic medical records, characterized by: The main body of this method is a Chinese electronic medical record word segmentation model based on GNN-BiLSTM-CRF, which includes the following steps: Step 1: Dataset construction and data processing; Manually annotate the Chinese electronic medical record data from the cooperative hospital to complete the construction of the Chinese electronic medical record word segmentation dataset; convert the data in the constructed dataset into CoNLL format; Step 2: Train the Chinese electronic medical record word segmentation model; Step 2.1: Extract the data processed in step 1 to generate a training set. Input the training set and dictionary into the embedding unit. Use Word2vec to embed the characters and words into corresponding vectors and input them into the next layer of the model's graph neural network (GNN) unit for calculation. In step 2.2, a graph is constructed using the embedded character and word vectors, with characters serving as nodes and words as edges. This information transfer mechanism, based on a multi-head attention mechanism and gating, is then used to iteratively aggregate and update information to learn the local features of medical terminology. The node and edge features of the graph are updated in each iteration, and the feature vector of each node in the graph after the last iteration is input into the next layer of the model. In step 2.3, each node feature in the graph is input into a bidirectional long short-term memory (LSTM) unit to model the contextual information of each word, capture sequence information and long-range dependency features, and mine the text information features of the Chinese electronic medical record corpus. Step 2.4: Use the conditional random field (CRF) unit to jointly decode the LSTM output to predict the best label sequence, and finally output the model's predicted label for each word; In step 2.5, the negative log-likelihood function is used as the loss function to evaluate the difference between the model's predicted value and the true value. The training is performed by minimizing the loss function to obtain the Chinese electronic medical record word segmentation model. The graph neural network aggregation operation uses a multi-head attention mechanism, where the node feature set and edge feature sets e ij For all words that match the dictionary and the text, that is, all edges in the graph; Each node c o Aggregate calculations go through the following three steps: The first step is to determine node c o Feature information that needs to be aggregated Aggregate edge features for each node when clustering nodes and the characteristics of the corresponding connected nodes The boundary information of words is learned by extracting the features of words and word beginnings and endings; The calculation is shown in formula (1): Among them: "[||]" represents the splicing operation, The feature dimension after splicing is 2F; r∈N o , N o Represents all with c o The node subscript set with edge connections, "|" represents or, that is, considering c at the same time o As the beginning and end of a word; The second step is to calculate node features With aggregate features The attention coefficient α ro : Where: W c and W g are all trainable weight matrices, where Perform linear transformation on the features; W o is the trainable weight matrix, LR stands for nonlinear activation using the LeakyReLU function; The third step is to use the multi-head mechanism to calculate the aggregated features Where: K is the number of heads of the multi-head attention mechanism; represents the attention coefficient of the normalized node feature and the corresponding aggregate feature calculated by the k-th attention mechanism; W k is a trainable weight matrix; perform weighted averaging on all heads to obtain the final output features The final output feature set of node aggregation is Edge ij The polymerization process is done in three steps: The first step is to determine the edge e ij Feature information that needs to be aggregated When edge aggregation is performed, feature information of all nodes on each edge is aggregated, v∈N ij , N ij Represents the subscript set of all nodes from node i to node j; The second step is to calculate the edge features With aggregate features The attention coefficient α iv : Among them, W e and W v are all trainable weight matrices, where W i is the trainable weight matrix, The third step is to use the multi-head mechanism to calculate the aggregated features Among them, K is the number of heads of the multi-head attention mechanism; represents the attention coefficient of the normalized edge feature and the corresponding aggregate feature calculated by the k-th attention mechanism; W e is the trainable weight matrix; The final output feature set of edge aggregation is Use a gating mechanism to control the flow of aggregated features to nodes and edges, so that they can learn the local feature information obtained by aggregation; Input node feature set at time t and node aggregation feature set Output time t+1 Where W and V are both trainable parameter matrices, b is a trainable parameter vector; ⊙ is the Hadamard Product, the corresponding elements in the matrix are multiplied, and the two multiplied matrices must be of the same type; σ is the activation function sigmoid; Softmax normalizes the two gates; tanh is the activation function, which performs nonlinear changes; is the node feature of node o in the t+1 round of iteration; Calculate two gates and And the feature information to be learned All by node features and aggregate features calculated; use To control the forgetting feature Part of the information in Controlling learning features Part of the information in; finally output the node features of the next round Side update: Input edge feature set at time t The feature set obtained by edge aggregation Output edge feature set at time t+1 Among them, W and V are trainable parameter matrices, b is a trainable parameter vector; ⊙, σ, Softmax, and tanh are updated at the same node; For edge e ij Edge features in the t+1 iteration; Calculate two gates and And the characteristic information to be transmitted Through edge features and aggregate features calculated; used Control characteristics use Control characteristics Finally output the edge features of the next round Each iteration of the entire graph neural network goes through a complete node and edge aggregation, as well as node and edge updates. After m rounds of iteration, the final output node feature set is As the output of the graph neural network; (a) The LSTM layer uses a bidirectional LSTM to capture past and future information respectively, and concatenates the two results as the final output. Its input comes from the output of the graph neural network. The specific calculation process is as follows: First calculate the three gates And the characteristic information to be transmitted All by and output Calculated; Among them, W and V are trainable parameter matrices, and b is a trainable parameter vector; Gating Used to control the forgetting of memory features passed from the previous unit Part of the information in the gate Used to control learning features Part of the information in generates the current unit memory feature And pass it to the next unit, as shown in formula (9): Will After using the tanh function for nonlinear activation, the gate Forget some of the information and get the output of the current unit And pass it to the next unit, as shown in formula (10): h in LSTM t Used to obtain feature information transmitted by nodes closer to the current node, C t It is used to record remote sequence information; (b) The CRF layer models the label sequence of the entire sentence text through CRF, and predicts the corresponding state sequence based on the corresponding observation sequence. The score s(y|x) of the label sequence y is calculated as follows: Where: T is the matrix obtained by linear transformation of the encoding layer output, where the matrix elements Represents the current character x i The corresponding label is y i The score when Z represents the transfer matrix, which is learned through training; the matrix elements Represents the previous label y i-1 Move to the current tab y i score; After calculating the scores of all possible label sequences, we normalize them using the softmax function to obtain the conditional probability P(y|x) of the label sequence y: Where: Y(x) represents all possible label sequences.

2. The Chinese electronic medical record word segmentation method according to claim 1, characterized in that: Based on the GNN-BiLSTM-CRF Chinese electronic medical record word segmentation model, an online graphical system for Chinese electronic medical record word segmentation based on B / S architecture is constructed. The functional modules of the system are as follows: The front-end interface module accesses the front-end interface of the system through a browser; inputs the original Chinese electronic medical record data through the corresponding input text box or file upload function in the interface; transmits the input data to the system's data processing module for word segmentation preprocessing through the word segmentation function in the interface; displays the word segmentation results in the interface and provides a download function; The data processing module converts the data input from the front end into the CoNLL format required by the Chinese electronic medical record word segmentation module and inputs it into the Chinese electronic medical record word segmentation module; it segments the original input Chinese electronic medical record text according to the predicted label of each word output by the Chinese electronic medical record word segmentation module and returns the segmentation results to the front end interface; The Chinese electronic medical record word segmentation module inputs the data processed by the data processing module into the trained Chinese electronic medical record word segmentation model, and performs operations through the embedding unit, graph neural network unit, bidirectional long short-term memory network unit, and conditional random field unit in turn, and finally outputs the predicted label of each word, and then returns the result to the data processing module.

Images