A medical event extraction method based on a generative model
By employing a joint extraction strategy based on generative models, the challenge of extracting medical events in complex scenarios was solved, achieving efficient and accurate extraction of medical event information and improving the quality of the medical database.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2022-09-30
- Publication Date
- 2026-05-08
AI Technical Summary
Existing deep learning algorithms struggle to efficiently handle complex medical event extraction scenarios in the medical field, especially long text event extraction and multi-event extraction, and existing methods cannot meet the requirements of accuracy and robustness.
A joint extraction strategy based on generative models is adopted, which combines a Transformer encoder, a linear classification layer, a pointer network, and a generative model. The loss function is calculated through the training and prediction process to generate structured medical event information.
It improves the efficiency and accuracy of event extraction, reduces computing power consumption, solves the entity overlap problem, supports multi-event extraction, and improves the construction quality of medical databases.
Smart Images
Figure CN115545029B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of information extraction, medical big data, and deep learning, and relates to a method for extracting medical events based on a generative model. Background Technology
[0002] With the continued rapid growth and expansion of the internet, the rate of information growth in this era is skyrocketing. Every day, tens of thousands of unstructured text data are generated and updated in our lives. However, this text data is only significant in its sheer quantity; opportunities for secondary development and utilization are extremely limited.
[0003] In recent years, artificial intelligence and cloud computing technologies have continued to develop, reaching an unprecedented new height. Relying on the powerful computing power provided by cloud computing and the algorithmic support provided by the field of artificial intelligence, these unstructured texts can be reused to rediscover and extract their value.
[0004] The construction of smart cities in my country has gone through an exploratory period from 2008 to 2014, an adjustment period from 2014 to 2015, and a breakthrough period from 2015 to 2017. Currently, smart cities are in their fourth stage. In the transportation sector, the "City Brain" monitors the operation of various road sections. If local congestion occurs, it automatically guides drivers to choose alternative routes to improve traffic efficiency. In the education sector, smart classrooms are in place, using a hardware and software integration approach to achieve centralized intelligent recording, remote interaction, and routine live and recorded broadcasts. This provides big data support for teaching decisions, enabling data-driven teaching decisions, real-time evaluation feedback, multi-dimensional communication and interaction, and intelligent resource delivery, creating an intelligent and efficient learning ecosystem. In recent years, thanks to the rapid development of natural language processing technology, powerful NLP models such as Transformer and BERT have emerged, resulting in a qualitative leap in the processing capabilities of the City Brain in the medical field. This helps doctors convert written, unstructured electronic medical records into structured text, improving doctors' work efficiency and promoting the construction of medical databases. Event extraction, as one of the important methods of structured information extraction, naturally has significant research value.
[0005] In the medical field, electronic medical records written by doctors contain a large amount of entity information such as names, ages, genders, disease types, and symptoms. This entity information can also form event information. However, manually annotating this information is time-consuming and labor-intensive, and requires professional and experienced medical personnel to annotate it correctly. Neural networks combined with deep learning can pre-learn this type of text. After learning, they can then input large amounts of text and perform text recognition and information extraction. However, due to limitations in current deep learning algorithm design and computing power, only a few algorithms have been successfully implemented, and their applicable scenarios are very limited. When encountering complex situations, such as long text event extraction, multiple event extraction, and entity overlap, existing pipeline-based event extraction algorithms cannot achieve the required accuracy and robustness.
[0006] In this context, by introducing a joint extraction mechanism and generative modeling techniques, the extraction speed and ability to handle complex situations of existing event extraction methods can be greatly improved, thus significantly enhancing information extraction efficiency. Summary of the Invention
[0007] To address the problem of medical event extraction in complex scenarios, this invention proposes a medical event extraction method based on a generative model. This method extracts more readable structured event information from existing electronic medical record texts, thus contributing to the construction of a high-quality medical database.
[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0009] A method for extracting medical events based on a generative model, the method comprising the following steps:
[0010] 1) Obtain electronic medical record text data from the database, and pre-define the entity types and event schemas to be extracted. The entities include names, ages, times, genders, onset sites and symptoms, etc., and the events include infectious diseases, parasitic diseases, tumors and endocrine diseases, etc.
[0011] 2) Based on the above annotations, manually annotate the entity and event information in the descriptive text on the annotation platform, and generate annotated text according to the format as training and testing data;
[0012] 3) The event extraction method uses a joint extraction strategy for training. Inputting medical record text and label data, a Transformer encoder is first used as the text encoder. The encoded text results are fed into a linear classification layer and a pointer network to obtain event types and entity BIO labels, calculating event type loss1 and entity label loss2. The event type and entity BIO labels are then fed into the event encoder and entity encoder, respectively. The text encoding results, event encoding results, and entity encoding results are summed and fed into the generative model to generate event argument roles, calculating argument role generation loss3. Loss1, loss2, and loss3 are summed to obtain the final loss, which is then used for backpropagation.
[0013] 4) After training is complete, save the model;
[0014] 5) During prediction, a text is input into the model, and the sequence generated by the generator model is processed to obtain the final structured event information.
[0015] Furthermore, the processing procedure in step 3) is as follows:
[0016] 3.1) Tokenize the input text, adding special tokens [start] and [end] to the beginning and end, resulting in text W = [w1, w2, ... w... n ], where w t For the token at position t+1 in the text, inputting W into the text encoder Transformer1 yields the vector representation H = [h1, h2, ..., h...]. n The process is represented as follows:
[0017] H = Transformer1(W) (1)
[0018] 3.2) Perform max pooling on H to obtain the vector representation h of the article; reduce the dimensionality of h to the number of event categories using a linear layer, and use the softmax function to obtain the probability value of each event. The process is as follows:
[0019] h = maxpooling(h) (2)
[0020] p = softmax(fc1(h)) (3)
[0021] 3.3) Based on the event type label, the gold label g of the event type can be obtained, and then the cross-entropy loss function is used to calculate the loss value for the event classification task:
[0022]
[0023] Where M event Represents the number of event categories
[0024] 3.4) Based on the entity labels annotated by BIO, the gold labels for entity recognition can be constructed as G = [g1, g2, ..., g...]. n Then, the text vector representation H obtained in 3.1) is input into the pointer network to obtain the entity label classification probability P = [p1, p2, ..., p] for each token. n Then, the cross-entropy loss function is used to calculate the loss value for the entity recognition task. The process is as follows:
[0025]
[0026] Where n represents the sequence length, M bio Represents the number of entity tags
[0027] 3.5) Perform argmax calculation on h` calculated in 3.2) to obtain the event type ID. Then, feed the event type ID into the event classification embedding layer to obtain the vector representation t of the event type. Perform argmax calculation on the entity label classification probability P calculated in 4.4) to obtain the entity label for each token. Feed the entity label into the entity embedding layer to obtain the entity vector representation E = [e1, e2, ..., e] n The process is represented as follows:
[0028] t=Embedding1(argmax(h`)) (6)
[0029] E = Embedding2(argmax(P)) (7)
[0030] 3.6) Copy the t calculated in 4.5) n times to obtain T = [t1, t2, ..., tn]. n ], so that its dimensions are the same as those of the text vector representation H;
[0031] 3.7) Add the text encoding result H, the event encoding result T, and the entity encoding result E to obtain X = [x1, x2, ..., x]. n The input is fed into the generative model Transformer2 to obtain X` = [x`1, x`2, ..., x` n Then, a linear layer is used to reduce the dimensionality to 1 dimension, and softmax is used to obtain the probability value P_gen = [p_gen1, p_gen2, ..., p_gen] for each token. n Select the vector x corresponding to the token with the highest probability among the first n tokens and add it to the end of X. This process is represented as follows:
[0032] X` = Transformer2(X) (8)
[0033] P_gen = softmax(fc2(X`)) (9)
[0034] X:=X append argmax(P_gen) (10)
[0035] 3.8) Repeat step 3.7) until the selected token is a special token [end]; resulting in the following sequence:
[0036] [Original Character 1][start][Original Character 2][start][Original Character 3][start]…[end]
[0037] Where [start] represents the separator between two argument roles, and [end] indicates the end of generation.
[0038] 3.9) Extract argument roles from the sequence obtained in 3.8) to obtain the final argument role information.
[0039] 3.10) is for calculating the loss function of 3.7) and 3.8). The labels extracted from the argument roles contain the generated complete sequence. When generating to the i-th step, the elements after the i-th element of the label sequence are masked, and the cross-entropy loss is calculated using the partially masked labels. The loss of argument role generation is expressed as the average loss at each step:
[0040]
[0041] Where N represents the number of generation times, and n represents the length of the original sequence.
[0042] 3.11) Add Loss1, Loss2, and Loss3 together to get the final Loss, and then perform backpropagation.
[0043] The beneficial effects of this invention are as follows: 1. Using a joint extraction model improves the efficiency of event extraction, reduces computational consumption, and helps improve prediction accuracy; 2. Utilizing a generative model as a decoder avoids entity overlap and reduces model complexity; 3. It overcomes the limitation of previous event extraction methods, which could only predict one event from a piece of text. Attached Figure Description
[0044] Figure 1 This is a flowchart of the logic of a medical event extraction method based on a generative model. Detailed Implementation
[0045] The invention will now be further described with reference to the accompanying drawings.
[0046] Reference Figure 1A method for extracting medical events based on a generative model includes the following steps:
[0047] 1) Obtain electronic medical record text data from the database, and pre-define the entity types and event schemas to be extracted. The entities include names, ages, times, genders, onset sites and symptoms, etc., and the events include infectious diseases, parasitic diseases, tumors and endocrine diseases, etc.
[0048] 2) Based on the above annotations, manually annotate the entity and event information in the descriptive text on the annotation platform, and generate annotated text according to the format as training and testing data;
[0049] 3) The event extraction method uses a joint extraction strategy for training. Inputting medical record text and label data, a Transformer encoder is first used as the text encoder. The encoded text results are fed into a linear classification layer and a pointer network to obtain event types and entity BIO labels, calculating event type loss1 and entity label loss2. The event type and entity BIO labels are then fed into the event encoder and entity encoder, respectively. The text encoding results, event encoding results, and entity encoding results are summed and fed into the generative model to generate event argument roles, calculating argument role generation loss3. Loss1, loss2, and loss3 are summed to obtain the final loss, which is then used for backpropagation.
[0050] The processing procedure in step 3) is as follows:
[0051] 3.1) Tokenize the input text, adding special tokens [start] and [end] to the beginning and end, resulting in text W = [w1, w2, ... w... n ],in w Let t be the token at position t+1 in the text. After inputting W into the text encoder Transformer1, the vector representation of the text is obtained as H = [h1, h2, ..., h]. n The process is represented as follows:
[0052] H = Transformer1(W) (1)
[0053] 3.2) Perform max pooling on H to obtain the vector representation h of the article; reduce the dimensionality of h to the number of event categories using a linear layer, and use the softmax function to obtain the probability value of each event. The process is as follows:
[0054] h = maxpooling(H) (2)
[0055] p = softmax(fc1(h)) (3)
[0056] 3.3) Based on the event type label, the gold label g of the event type can be obtained, and then the cross-entropy loss function is used to calculate the loss value for the event classification task:
[0057]
[0058] Where M event Represents the number of event categories
[0059] 3.4) Based on the entity labels annotated by BIO, the gold labels for entity recognition can be constructed as G = [g1, g2, ..., g...]. n Then, the text vector representation H obtained in 3.1) is input into the pointer network to obtain the entity label classification probability P = [p1, p2, ..., p] for each token. n Then, the cross-entropy loss function is used to calculate the loss value for the entity recognition task. The process is as follows:
[0060]
[0061] Where n represents the sequence length, M bio Represents the number of entity tags
[0062] 3.5) Perform argmax calculation on h` calculated in 3.2) to obtain the event type ID. Then, feed the event type ID into the event classification embedding layer to obtain the vector representation t of the event type. Perform argmax calculation on the entity label classification probability P calculated in 4.4) to obtain the entity label for each token. Feed the entity label into the entity embedding layer to obtain the entity vector representation E = [e1, e2, ..., e] n The process is represented as follows:
[0063] t=Embedding1(argmax(h`)) (6)
[0064] E = Embedding2(argmax(P)) (7)
[0065] 3.6) Copy the t calculated in 4.5) n times to obtain T = [t1, t2, ..., tn]. n ], so that its dimensions are the same as those of the text vector representation H;
[0066] 3.7) Add the text encoding result H, the event encoding result T, and the entity encoding result E to obtain X = [x1, x2, ..., x]. n The input is fed into the generative model Transformer2 to obtain X` = [x`1, x`2, ..., x` nThen, a linear layer is used to reduce the dimensionality to 1 dimension, and softmax is used to obtain the probability value P_gen = [p_gen1, p_gen2, ..., p_gen] for each token. n Select the vector x corresponding to the token with the highest probability among the first n tokens and add it to the end of X. This process is represented as follows:
[0067] X` = Trandformer2(X) (8)
[0068] P_gen = softmax(fc2(X`)) (9)
[0069] X:=X append argmax(P_gen) (10)
[0070] 3.8) Repeat step 3.7) until the selected token is a special token [end]; resulting in the following sequence:
[0071] [Original Character 1][start][Original Character 2][start][Original Character 3][start]…[end]
[0072] Where [start] represents the separator between two argument roles, and [end] indicates the end of generation.
[0073] 3.9) Extract argument roles from the sequence obtained in 3.8) to obtain the final argument role information.
[0074] 3.10) is for calculating the loss function of 3.7) and 3.8). The labels extracted from the argument roles contain the generated complete sequence. When generating to the i-th step, the elements after the i-th element of the label sequence are masked, and the cross-entropy loss is calculated using the partially masked labels. The loss of argument role generation is expressed as the average loss at each step:
[0075]
[0076] Where N represents the number of generation times, and n represents the length of the original sequence.
[0077] 3.11) Add Loss1, Loss2, and Loss3 together to get the final Loss, and then perform backpropagation;
[0078] 4) After training is complete, save the model;
[0079] 5) During prediction, a text is input into the model, and the sequence generated by the generator model is processed to obtain the final structured event information.
[0080] In this embodiment, the algorithm can be applied to the medical insurance service platform. After the user inputs the electronic medical record, the system extracts the structured medical record and determines whether the condition can be reimbursed by medical insurance.
[0081] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for extracting medical events based on generative models, characterized in that, The method includes the following steps: 1) Obtain electronic medical record text data from the database, and pre-define the entity types and event schemas to be extracted. The entities include name, age, time, gender, onset location and symptoms, and the events include infectious diseases, parasitic diseases, tumors and endocrine diseases. 2) Based on the above annotations, manually annotate the entity and event information in the descriptive text on the annotation platform, and generate annotated text according to the format as training and testing data; 3) The event extraction method uses a joint extraction strategy for training. Inputting medical record text and label data, a Transformer encoder is first used as the text encoder. The encoded text results are fed into a linear classification layer and a pointer network to obtain event types and entity BIO labels, calculating event type loss1 and entity label loss2. The event type and entity BIO labels are then fed into the event encoder and entity encoder, respectively. The text encoding results, event encoding results, and entity encoding results are summed and fed into the generative model to generate event argument roles, calculating argument role generation loss3. Loss1, loss2, and loss3 are summed to obtain the final loss, which is then used for backpropagation. 4) After training is complete, save the model; 5) During prediction, a text is input into the model, and the sequence generated by the generator model is processed to obtain the final structured event information.
2. The medical event extraction method based on generative models according to claim 1, characterized in that, The processing procedure in step 3) is as follows: 3.1) Tokenize the input text, adding special tokens [start] and [end] to the beginning and end, resulting in text W = [w1, w2, ... w... n ], where w t For the token at position t+1 in the text, inputting W into the text encoder Transformer1 yields the vector representation H = [h1, h2, ..., h...]. n The process is represented as follows: H = Transformer1(W) (1) 3.2) Perform max pooling on H to obtain the vector representation h of the article; reduce the dimensionality of h to the number of event categories using a linear layer, and use the softmax function to obtain the probability value of each event. The process is as follows: h = maxpooling(H) (2) p = softmax(fc1(h)) (3) 3.3) Based on the event type label, the gold label g of the event type can be obtained, and then the cross-entropy loss function is used to calculate the loss value for the event classification task: Where M event Indicates the number of event categories; 3.4) Based on the entity labels annotated by BIO, the gold labels for entity recognition can be constructed as G = [g1, g2, ..., g...]. n Then, the text vector representation H obtained in 3.1) is input into the pointer network to obtain the entity label classification probability P = [p1, p2, ..., p] for each token. n Then, the cross-entropy loss function is used to calculate the loss value for the entity recognition task. The process is as follows: Where n represents the sequence length, M bio Indicates the number of entity tags; 3.5) Perform argmax calculation on h` obtained in 3.2) to obtain the event type id. Then, send the event type id into the event classification embedding layer to obtain the vector representation t of the event type. Performing argmax calculation on the entity label classification probability P obtained in 4.4) yields the entity label for each token. These entity labels are then fed into the entity embedding layer to obtain the entity vector representation E = [e1, e2, ..., e2]. n The process is represented as follows: t=Embedding1(argmax(h`)) (6) E = Embedding2(argmax(P)) (7) 3.6) Copy the t calculated in 4.5) n times to obtain T = [t1, t2, ..., tn]. n ], so that its dimensions are the same as those of the text vector representation H; 3.7) Add the text encoding result H, the event encoding result T, and the entity encoding result E to obtain X = [x1, x2, ..., x]. n The input is fed into the generative model Transformer2 to obtain X` = [x`1, x`2, ..., x` n Then, a linear layer is used to reduce the dimensionality to 1 dimension, and softmax is used to obtain the probability value P_gen = [p_gen1, p_gen2, ..., p_gen] for each token. n Select the vector x corresponding to the token with the highest probability among the first n tokens and add it to the end of X. This process is represented as follows: X` = Transformer2(X) (8) P_gen = softmax(fc2(X`)) (9) X:=Xappend argmax(P_gen) (10) 3.8) Repeat step 3.7) until the selected token is a special token [end]; resulting in the following sequence: [Original Character 1][start][Original Character 2][start][Original Character 3][start]…[end] Where [start] represents the separator between two argument roles, and [end] represents the end of generation; 3.9) Extract argument roles from the sequence obtained in 3.8) to obtain the final argument role information. 3.10) To calculate the loss function for 3.7) and 3.8), the labels extracted from the argument roles contain the generated complete sequence. When generating to the i-th step, the elements after the i-th element in the label sequence are masked, and the cross-entropy loss is calculated using the partially masked labels. The loss of argument role generation is expressed as the average loss at each step: Where N represents the number of generation times, and n represents the length of the original sequence. 3.11) Add Loss1, Loss2, and Loss3 together to get the final Loss, and then perform backpropagation.