A method for extracting structured information from Chinese electronic medical records

Abstract

This invention discloses a method for extracting structured information from Chinese electronic medical records. First, the BERT model is used to classify sentences containing different symptom entity modifiers. Then, existing named entity recognition models are used to label symptom entities in each type of sentence according to different categories, achieving the goal of identifying entity modifiers. A rule-based feature extraction method for symptom entity attribute values ​​in Chinese electronic medical records is proposed. By summarizing the textual rules of symptom semantic segments in medical record texts, an attribute value extraction template is formed, and the extracted attribute values ​​are stored in a database. Based on existing medical record text extraction technologies, this invention deeply analyzes the textual rules of medical record texts and performs entity modification recognition and feature extraction on symptom semantic segments that have already been segmented and labeled, providing structured data support for subsequent upper-level applications.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and specifically to a method for extracting structured information from Chinese electronic medical records based on deep learning and rules. Background Technology

[0002] In recent years, the rapid development of information technology has driven the informatization of hospitals, with an increasing number of Chinese Electronic Medical Records (EMRs) replacing traditional paper medical records. As a crucial clinical information resource generated during medical activities, EMRs contain a vast amount of medical knowledge closely related to patients' health conditions. Extracting this knowledge from unstructured medical record text is a primary task of clinical medical information processing technology and a key aspect of medical record text information extraction. Structured information extraction technology for medical record texts uses natural language processing techniques, combined with the unique characteristics of medical record texts, to extract entities and their attributes such as symptoms, examinations, treatments, and diseases from the text. This technology can provide data support for higher-level applications such as medical knowledge graph construction and assisted diagnosis, thereby promoting the development of my country's medical industry.

[0003] Currently, research on structured extraction from Chinese electronic medical records mostly focuses on named entity recognition, entity relation extraction, and named entity modification recognition. Named entity recognition methods mainly include rule-based and dictionary-based methods, traditional machine learning methods, and deep learning methods. Each method has its advantages and disadvantages. The most mainstream method is the deep learning-based BiLSTM+CRF sequence labeling model, which has powerful capabilities in automatically extracting sequence features and capturing bidirectional semantic information, significantly improving the accuracy of named entity recognition. Entity relation extraction methods mainly include co-occurrence analysis, traditional machine learning methods, and deep learning methods. Among them, RNN-based deep learning methods incorporating attention mechanisms have achieved good rankings in recent evaluation tasks. However, the above two information extraction techniques only achieve the labeling of medical entities or the classification of relationships between entities, but cannot extract various attribute values ​​of entities, such as the start time, duration, severity, accompanying symptoms, and asymptomatic descriptions of symptom entities. In the area of ​​named entity modification recognition, there is relatively little research on Chinese electronic medical records compared to English electronic medical records. Zhang et al. (Medicalassertion classification in Chinese EMRs using attention enhanced neural network, 2019, Mathematical Biosciences and Engineering, 1966-1977) combined CNN with GRU to obtain character-level and sentence-level representations, achieving some results, but there is still considerable room for optimization. Therefore, more research is urgently needed in the area of ​​named entity modification recognition.

[0004] It can be seen that existing Chinese electronic medical record information extraction technologies focus more on text processing aspects such as named entity recognition and entity relation extraction, while having certain shortcomings in entity modification identification and feature extraction. Summary of the Invention:

[0005] To address the problem that existing information extraction technologies cannot fully realize the feature extraction of Chinese medical record text, this invention conducts in-depth research on the writing characteristics and rules of Chinese electronic medical records, and proposes a post-structured information extraction method for Chinese electronic medical records based on deep learning and rules. This method identifies the symptom entity modification components of symptom semantic segments that have already been segmented and annotated with entities, and extracts various attributes of symptom entities to achieve a structured expression of symptom entities.

[0006] This invention comprises two parts:

[0007] The first invention proposes a method for identifying the modifiers of symptom entities in Chinese electronic medical records. First, the BERT model is used to classify sentences containing different modifiers of symptom entities. Then, an existing named entity recognition model is used to label the symptom entities in each type of sentence with different categories, thereby achieving the goal of identifying entity modifiers.

[0008] The second invention proposes a rule-based method for feature extraction of symptom entity attribute values ​​in Chinese electronic medical records. A symptom entity has six attributes: current symptom, start time, duration, severity, accompanying symptoms, and denied symptoms. By summarizing the textual rules governing the semantic segments of symptoms in medical record texts, an attribute value extraction template is formed, and the extracted attribute values ​​are stored in a database.

[0009] The specific steps of the method of the present invention are as follows:

[0010] Step 1: Categorize sentences

[0011] The BERT (Bidirectional Encoder Representation from Transformers) model was used to classify sentences, resulting in three categories: sentences containing main symptom entities, accompanying symptom entities, and denial symptom entities.

[0012] Step 2: Word segmentation and named entity tagging

[0013] We utilize existing Chinese electronic medical record word segmentation and named entity recognition systems to perform word segmentation and entity annotation on medical record texts.

[0014] Step 3: Write rules to correct the annotation results

[0015] We summarize the location distribution and expression patterns of attribute values ​​of symptom entities in medical record text, and design rule templates to correct the annotation results of step two, thereby improving the accuracy of attribute extraction.

[0016] Step 4: Named Entity Modification Recognition

[0017] Based on the sentence classification results obtained in step one and the entity annotation results obtained in step three, the labels of symptom entities in different types of sentences are replaced to achieve the recognition of symptom entity modifications.

[0018] Step 5: Divide each medical record text into blocks using "start time" as the dividing point;

[0019] Step 6: Extract attributes and store them in the database.

[0020] Preferably, in step one, BERT is a pre-trained language representation model. Its main structure is a bidirectional Transformer encoder. The model input is obtained by summing the word embeddings and positional encodings of the original sequence. The input sequence first passes through a multi-headed self-attention mechanism layer (12 heads), enabling each word in the sequence to learn its multiple semantics in the current context. Then, the output of the attention mechanism layer is processed by residual connections and layer normalization before being input into the feedforward neural network. Finally, the output of the feedforward network undergoes another residual connection and layer normalization to obtain the output sequence. This completes the operation of one Transformer encoder. Stacking 12 Transformer encoders layer by layer constitutes the BERT model. The input of each encoder is the output of the previous encoder, and the output of the last encoder is the text feature representation output by the BERT model.

[0021] As a preferred approach, step three designs a series of rule templates to correct the annotation results of attribute values ​​in medical record texts. The rules are mainly divided into three categories—the first category: merging rules, which are mainly designed to solve the problem of excessive word segmentation and annotation of complete semantic segments in the original annotation results. The rules are listed in Table 1.

[0022] Table 1. Contents and Examples of Merging Rules

[0023]

[0024] The second category is classification rules. These rules are mainly designed to address the issue of identical labels for "start time" and "duration" in the original annotation results. The rules are listed in Table 2.

[0025] Table 2: Classification Rules and Examples

[0026] Rule Number Rules Regex6 Continuous.([\u4e00-\u9fa7]U)*[0-9]*-? [0-9]*[\u4e00-\u9fa7]+P Regex7 [0-9]*-? [0-9]*[\u4e00-\u9fa7]+P([\u4e00-\u9fa7]+U)*Relieve

[0027] The third category is negation rules. These rules are mainly designed to address the issue of "body parts" and "symptoms" being separated by negation words in medical record texts. The rules are listed in Table 3.

[0028] Table 3. Contents and Examples of Negation Rules

[0029]

[0030] As a preferred step, step four replaces the symptom entity label S in sentences labeled 1 and 2 with DS and AS respectively, based on the sentence classification results obtained in step one and the entity annotation results obtained in step three, thereby classifying the symptom entities into three categories: primary symptoms, accompanying symptoms, and denial symptoms.

[0031] As a preferred option, step five divides each medical record text into blocks based on the "start time". If a patient has multiple symptoms that appear at different times, this step will result in multiple text blocks. Each text block will be extracted into a record in the next step, corresponding to a symptom attribute at a specific time.

[0032] Preferably, step six uses the annotation results obtained in the above steps to extract various attributes of the symptom entities and saves the extraction results to the database. The database table has a total of 8 fields: id, patientId, symptom, startTime, duration, degree, accSymptom, and denSymptom. Among them, id is the primary key of the database table, patientId is used to identify whether each record belongs to the same patient, the symptom field stores the semantic fragments in the medical record text labeled with S, the startTime field stores the semantic fragments in the medical record text labeled with P, TP, or SS, and the following 4 fields store the semantic fragments in the medical record text labeled with DP, HL, LL, AS, and DS, respectively.

[0033] Based on existing medical record text extraction technologies, this invention deeply analyzes the writing rules of medical record texts and performs entity modification recognition and feature-based attribute value extraction on symptom semantic segments that have been segmented and annotated. This provides structured data support for subsequent upper-level applications. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method for extracting structured information from Chinese electronic medical records according to an embodiment of the present invention;

[0035] Figure 2a This is a model structure diagram of BERT, an embodiment of the present invention;

[0036] Figure 2b This is a model structure diagram of a Transformer encoder according to an embodiment of the present invention. Detailed Implementation

[0037] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. The following description covers numerous specific details in order to provide a comprehensive understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a clearer understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific configurations and algorithms presented below, but covers any modifications, substitutions, and improvements to related elements, components, and algorithms without departing from the spirit of the invention.

[0038] Given that existing information extraction techniques cannot fully achieve feature extraction from medical record text, this invention delves into the writing characteristics and rules of Chinese electronic medical records, proposing a post-structured information extraction method for Chinese electronic medical records based on deep learning and rules. The following section combines... Figure 1 This section illustrates an example of a method for extracting structured information from Chinese electronic medical records according to an embodiment of the present invention.

[0039] Figure 1 A flowchart illustrating a method for extracting structured information from Chinese electronic medical records according to an embodiment of the present invention is shown. Figure 1 As shown, a method for extracting structured information from Chinese electronic medical records according to an embodiment of the present invention includes a text processing module, a labeling result correction module, an entity modification recognition module, and an attribute extraction module. Their functions are as follows: The text processing module performs sentence classification using the BERT model and utilizes existing Chinese electronic medical record word segmentation and named entity recognition systems to segment and label the medical record text. The model structure diagrams of the BERT model and its main structure, the Transformer encoder, are shown below. Figure 2a , Figure 2b As shown, the annotation result correction module corrects the original annotation results using three types of rule templates; the entity modification recognition module uses the sentence classification results and the corrected annotation results to obtain three types of symptom entities with different modification components by replacing the symptom entity labels; the attribute extraction module first divides the medical record text into blocks according to the different symptom start times, and then combines the corrected annotation results and the entity modification recognition results to extract the attributes of the symptom entities.

[0040] A method for extracting structured information from Chinese electronic medical records according to an embodiment of the present invention includes the following steps:

[0041] Step 1: Categorize sentences

[0042] The BERT (Bidirectional Encoder Representation from Transformers) model was used to classify sentences, resulting in three categories: sentences containing main symptom entities, accompanying symptom entities, and denial symptom entities.

[0043] Step 2: Word segmentation and named entity tagging

[0044] We utilize existing Chinese electronic medical record word segmentation and named entity recognition systems to perform word segmentation and entity annotation on medical record texts.

[0045] Step 3: Write rules to correct the annotation results

[0046] We summarize the location distribution and expression patterns of attribute values ​​of symptom entities in medical record text, and design rule templates to correct the annotation results of step two, thereby improving the accuracy of attribute extraction.

[0047] Step 4: Symptom Entity Modification Recognition

[0048] Based on the sentence classification results obtained in step one and the entity annotation results obtained in step three, the annotation labels of symptom entities in different types of sentences are replaced to achieve the recognition of symptom entity modifications.

[0049] Step 5: Divide each medical record text into blocks using "start time" as the dividing point;

[0050] Step 6: Extract attributes and store them in the database.

[0051] Preferably, in step one, BERT is a pre-trained language representation model. Its main structure is a bidirectional Transformer encoder. The model input is obtained by summing the word embeddings and positional encodings of the original sequence. The input sequence first passes through a multi-headed self-attention mechanism layer (12 heads), enabling each word in the sequence to learn its multiple semantics in the current context. Then, the output of the attention mechanism layer is processed by residual connections and layer normalization before being input into the feedforward neural network. Finally, the output of the feedforward network is subjected to another residual connection and layer normalization to obtain the output sequence. This completes the operation of one Transformer encoder. Stacking 12 Transformer encoders layer by layer constitutes the BERT model. The input of each encoder is the output of the previous encoder, and the output of the last encoder is the text feature representation output by the BERT model.

[0052] The sentence classification task using the BERT model is achieved through two processes: pre-training and fine-tuning. The pre-training process utilizes a large-scale unlabeled text corpus for self-supervised training, forming a pre-trained model. This pre-training process is completed and the pre-trained model is released by Google. The fine-tuning process uses the parameters of the pre-trained model as the initial parameters of the model, inputs a manually labeled dataset, and fine-tunes the parameters to further fit and converge the model. Sentence labels in this task are of three types: 0, 1, and 2, representing sentences containing entities of primary symptoms, entities of denial symptoms, and entities of accompanying symptoms, respectively. For example, the sentence "stenosis of the lumen of the right middle cerebral artery origin" is labeled 0, the sentence "no space-occupying lesions in both kidneys" is labeled 1, and the sentence "accompanied by edema in both lower extremities" is labeled 2.

[0053] In step two, the named entity recognition system labels 17 word categories, including body parts, symptoms, drugs, diseases, time, negation words, and severity. Among them, there are 9 word tag categories related to the symptom entity: symptom (S), body parts (BP), time point (TP), time period (P), specific context (SS), negation word (A), high degree (HL), low degree (LL), and no specific category (U).

[0054] Step 3 involves designing a series of rule templates to correct the attribute value annotation results in medical record texts. The rules are mainly divided into three categories—the first category: merging rules, the content of which and examples are shown in Table 4. This type of rule primarily aims to address the problem of excessive word segmentation and annotation of complete semantic fragments in the original annotation results. For semantic fragments identified by the rules, only the last tag is retained, while other tags and spaces within the fragment are deleted, achieving the goal of merging the words within the fragment into a single unit.

[0055] Table 4. Contents and Examples of Merging Rules

[0056]

[0057] The second category is classification rules, the content of which and examples are shown in Table 5. This category of rules is mainly designed to address the issue of identical labels for "start time" and "duration" in the original annotation results. The rules identify that all time segments are "duration" segments, and replace the P label with a new label DP representing the duration, thus distinguishing between "start time" and "duration".

[0058] Table 5: Classification Rules and Examples

[0059]

[0060] The third category is negation rules, the content of which and examples are shown in Table 6. This type of rule is mainly designed to address the problem of "body parts" and "symptoms" being separated by negation words in medical record texts. For semantic segments identified by this rule, the negation words are moved forward, and Regex5 is used to merge the "body parts" and "symptoms" entities.

[0061] Table 6. Contents and Examples of the Negation Rule

[0062]

[0063]

[0064] Step four, based on the sentence classification results obtained in step one and the entity annotation results obtained in step three, replaces the symptom entity label S in sentences labeled 1 and 2 with DS and AS respectively, thereby classifying symptom entities into three categories: primary symptom (S), accompanying symptom (DS), and denial symptom (AS). For example, in the sentence "no space-occupying lesions found in both kidneys," the symptom entity "bilateral kidney space-occupying lesions" is replaced with "DS" to indicate denial of the symptom; similarly, in the sentence "accompanied by bilateral lower extremity edema," the symptom entity "bilateral lower extremity edema" is replaced with "AS" to indicate accompanying symptom.

[0065] Step 5 involves segmenting each medical record text into blocks based on the "start time". If a patient has multiple symptoms occurring at different times, this step will generate multiple text blocks. Each text block will be extracted as a record in the next step, corresponding to a symptom attribute at a specific time. For example, for a medical record text stating, "The patient was found to have elevated fasting blood glucose during a physical examination 22 years ago, accompanied by symptoms of excessive thirst, polydipsia, and polyuria... Nine years ago, the patient began to experience intermittent numbness and coldness in the limbs, and pain in the left foot... Three years ago, the patient repeatedly experienced symptoms of hypoglycemia such as palpitations and sweating...", the text is segmented using "22 years ago", "9 years ago", and "3 years ago" as dividing points. Then, a record corresponding to the corresponding time is extracted from each text block.

[0066] Step 6: Extract various attributes of symptom entities using the annotation results obtained in the above steps, and save the extraction results to the database. The database table has 8 fields: id, patientId, symptom, startTime, duration, degree, accSymptom, and denSymptom. Among them, id is the primary key of the database table, patientId is used to identify whether each record belongs to the same patient, the symptom field stores semantic fragments labeled with S in the medical record text, the startTime field stores semantic fragments labeled with P, TP, or SS in the medical record text, and the following 4 fields store semantic fragments labeled with DP, HL, LL, AS, and DS in the medical record text, respectively.

[0067] The method for extracting structured information from Chinese electronic medical records according to embodiments of the present invention has been described in detail above with reference to the accompanying drawings. As mentioned above, the present invention studies the writing characteristics and rules of Chinese electronic medical records and uses deep learning and rule templates to extract structured information from Chinese electronic medical records, thus solving the problem that existing information extraction technologies cannot achieve feature extraction of medical record text.

[0068] However, it should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. Furthermore, for the sake of brevity, detailed descriptions of known methods and techniques are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0069] This invention can be implemented in other specific forms without departing from its spirit and essential characteristics. For example, the algorithm described in a particular embodiment can be modified without departing from the basic spirit of the invention. Therefore, the present embodiments are to be regarded as exemplary rather than limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes falling within the meaning and scope of the claims and their equivalents are thus included within the scope of the invention.

Claims

1. A method for extracting structured information from Chinese electronic medical records, characterized in that: The specific steps to implement this method are as follows: Step 1: Categorize the sentences; The BERT model was used to classify sentences, resulting in three categories: sentences containing main symptom entities, accompanying symptom entities, and denial symptom entities. There are three sentence labels: 0, 1, and 2, which represent sentences containing main symptom entities, sentences containing denial symptom entities, and sentences containing accompanying symptom entities, respectively. Step 2: Word segmentation and named entity tagging; The existing Chinese electronic medical record word segmentation system and named entity recognition system were used to segment and annotate the medical record text. There are 9 categories of word tags related to symptom entities: symptom S, body part BP, time point TP, time period P, specific context SS, negation word A, high degree HL, low degree LL, and no specific category U. Step 3: Write rules to correct the annotation results; Summarize the location distribution and expression patterns of attribute values ​​of symptom entities in medical record text, and design rule templates to correct the annotation results of step two, thereby improving the accuracy of attribute extraction. Step 4: Named entity modification recognition; Based on the sentence classification results obtained in step one and the entity annotation results obtained in step three, the labels of symptom entities in different types of sentences are replaced to achieve the recognition of symptom entity modifications. Step 5: Divide each medical record text into blocks using "start time" as the dividing point; Step Six: Extract attributes and store them in the database; Step 3 designed a series of rule templates to correct the annotation results of attribute values ​​in medical record texts; the rules are mainly divided into three categories - the first category: merging rules, which are mainly to solve the problem of excessive word segmentation and annotation of complete semantic segments in the original annotation results; for the semantic segments identified by the rules, only the last tag is retained, while other tags and spaces in the segment are deleted, so as to merge the words in the segment into a whole; The second category is classification rules. These rules are designed to address the issue of identical labels for "start time" and "duration" in the original annotation results. The rules identify that all time segments contain "duration" labels, and replace the P labels with new labels DP to represent the duration, thus distinguishing between "start time" and "duration". The third category is negation rules. These rules are mainly designed to address the problem of "body parts" and "symptoms" being separated by negation words in medical record texts. For semantic segments identified by this rule, the negation words are moved forward, and Regex5 is used to merge the "body parts" and "symptoms" entities. Step four: Based on the sentence classification results obtained in step one and the entity annotation results obtained in step three, replace the symptom entity label S in sentences labeled 1 and 2 with accompanying symptoms DS and denial symptoms AS, respectively, thereby classifying the symptom entities into three categories: primary symptoms, accompanying symptoms, and denial symptoms.

2. The method for extracting structured information from Chinese electronic medical records according to claim 1, characterized in that: In step one, BERT is a pre-trained language representation model. Its main structure is a bidirectional Transformer encoder. The model input is obtained by summing the word embeddings and positional encodings of the original sequence. The input sequence first passes through a multi-head self-attention mechanism layer with 12 heads, enabling each word in the sequence to learn its multiple semantics in the current context. Then, the output of the attention mechanism layer is processed by residual connections and layer normalization before being input into the feedforward neural network. Finally, the output of the feedforward network is processed by residual connections and layer normalization again to obtain the output sequence, completing the operation of one Transformer encoder. The 12 Transformer encoders are stacked layer by layer to form the BERT model. The input of each encoder is the output of the previous encoder, and the output of the last encoder is the text feature representation output by the BERT model.

3. The method for extracting structured information from Chinese electronic medical records according to claim 1, characterized in that: Step 5 divides each medical record text into blocks based on the "start time". If a patient has multiple symptoms that appear at different times, this step will result in multiple text blocks. Each text block will be extracted into a record in the next step, corresponding to a symptom attribute at a specific time.

4. The method for extracting structured information from Chinese electronic medical records according to claim 1, characterized in that: Step 6: Extract various attributes of symptom entities using the annotation results obtained in the above steps, and save the extraction results to the database. The database table has 8 fields: id, patientId, symptom, startTime, duration, degree, accSymptom, and denSymptom. Among them, id is the primary key of the database table, patientId is used to identify whether each record belongs to the same patient, the symptom field stores semantic fragments labeled with S in the medical record text, the startTime field stores semantic fragments labeled with P, TP, or SS in the medical record text, and the following 4 fields store semantic fragments labeled with DP, HL, LL, AS, and DS in the medical record text, respectively.

Images