A medical event extraction method based on generative pre-trained language model
By applying a generative pre-trained language model in the medical field for event type discrimination and element extraction, the problems of high annotation cost and low extraction accuracy in the prior art are solved, and more efficient and accurate medical event extraction is achieved.
Patent Information
- Application Number
- CN202311274549.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-28
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2043-09-28
AI Technical Summary
The existing incident extraction technology in the medical field has the problems of high labeling costs and low extraction accuracy.
Using the medical event extraction method based on the generative pre-trained language model, the event type is determined and the event elements are extracted, and the event type is discriminated and elements are identified by the generative pre-trained language model with domain adaptation and task fine-tuning. Extract.
It improves the accuracy of event extraction, reduces the cost of manual annotation and data cleaning, and enhances the generalization ability and adaptability of the model.
Smart Images

Figure CN117493504B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing in the medical field, and in particular relates to a medical event extraction method based on a generative pre-trained language model. Background Art
[0002] Event extraction is the process of extracting events with semantic information from natural language text, and is an important issue in the field of natural language processing (NLP). Event extraction aims to automatically detect and extract events and entities related to predefined event types from a text. Event extraction can identify the predicates and their corresponding arguments described in natural language text, namely the participants, attributes, time, etc. of the event, and convert them into a structured data format.
[0003] The event extraction technologies commonly used in the current medical field are mostly rule-based or feature-based. Since the data in the medical field are complex and diverse, and most of them are unstructured data, event extraction based on traditional methods often has defects such as low accuracy and high annotation costs. The generative pre-trained language model has automatic learning capabilities and can automatically learn language patterns and rules through a large amount of data without the need for complex feature engineering and manual parameter adjustment. For new data and fields, the generative pre-trained language model has stronger generalization capabilities and can better adapt to a variety of application scenarios. In addition, the generative pre-trained language model uses a richer corpus and a more complex model structure during the training process, which can effectively improve the model effect. Therefore, the present invention proposes a medical field event extraction method based on a generative large model, which aims to improve the limitations of current technology. Summary of the invention
[0004] In view of the problems of high labeling cost and low extraction accuracy in the prior art, the purpose of the embodiment of the present application is to provide a medical event extraction method based on a generative pre-trained language model.
[0005] According to a first aspect of an embodiment of the present application, a method for extracting medical events based on a generative pre-trained language model is provided, comprising:
[0006] Obtain target medical texts;
[0007] Given all the medical event types to be extracted;
[0008] Define the corresponding elements to be extracted for each type of medical event;
[0009] Using an event type discrimination model to determine the event type involved in the target medical text, wherein the event type discrimination model is a generative pre-trained language model that has been domain adapted and fine-tuned for event type discrimination tasks;
[0010] According to the event type, an event extraction model is used to extract elements corresponding to the event of the target medical text, wherein the event extraction model is a generative pre-trained language model that has been fine-tuned for domain adaptation and event element extraction tasks.
[0011] Furthermore, the medical event type refers to the type of event that a patient experiences from entering the hospital to leaving the hospital, including patient admission events, patient medication events, patient laboratory test events, patient transfer to the ICU events, and patient discharge events.
[0012] Furthermore, GPT is used as the event type discrimination model.
[0013] Furthermore, the event type discrimination model is used to determine the event type involved in the target medical text, including:
[0014] Adapting generative pre-trained language models to the medical field;
[0015] Pre-train the language model of a single functional task on a predetermined task dataset to fine-tune the event type discrimination task;
[0016] Constructing a prompt word template Prompt1 of an event type discrimination model for distinguishing the type of medical events;
[0017] According to the prompt word template Prompt1, prompt words including all medical event types and target medical texts are generated and input into the event type discrimination model to obtain the event types involved in the target medical text.
[0018] Furthermore, the generative pre-trained language model is adapted to the medical field, specifically: the medical field text is converted into a word index through word segmentation, and the generative pre-trained language model is pre-trained using the word index, wherein the medical field text includes desensitized patient case records, journal articles in medical databases, conference papers, medical discipline textbooks, and news information.
[0019] Furthermore, the predetermined task data set format is consistent with the prompt word template Prompt1.
[0020] Furthermore, according to the event type, an event extraction model is used to extract elements corresponding to the event of the target medical text, including:
[0021] Adapting generative pre-trained language models to the medical field;
[0022] Pre-train the language model of a single functional task on a predetermined task dataset to fine-tune the event element extraction task;
[0023] Construct a prompt word template Prompt2 for the event extraction model used for medical event extraction;
[0024] According to the prompt word template Prompt2, the elements to be extracted corresponding to the target medical text and the events involved are generated and input into the event extraction model to obtain the event elements involved in the target medical text.
[0025] According to a second aspect of an embodiment of the present application, a medical event extraction device based on a generative pre-trained language model is provided, comprising:
[0026] An acquisition module, used to acquire target medical text;
[0027] A type given module is used to give all medical event types to be extracted;
[0028] The element definition module is used to define the corresponding elements to be extracted for each type of medical event;
[0029] An event type discrimination module, used to use an event type discrimination model to determine the event type involved in the target medical text, wherein the event type discrimination model is a generative pre-trained language model that has been domain adapted and fine-tuned for event type discrimination tasks;
[0030] An event extraction module is used to extract elements corresponding to the events of the target medical text according to the event type using an event extraction model, wherein the event extraction model is a generative pre-trained language model that has been fine-tuned for domain adaptation and event element extraction tasks.
[0031] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:
[0032] one or more processors;
[0033] A memory for storing one or more programs;
[0034] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.
[0035] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.
[0036] The technical solution provided by the embodiments of the present application may have the following beneficial effects:
[0037] It can be seen from the above embodiments that the present application utilizes the powerful expressive power of a large pre-trained language model, so that the model can automatically learn the language rules and characteristics of the medical field, and can adapt to the extraction of various event types. At the same time, the present invention can efficiently realize event extraction, reduce the cost of manual annotation and data cleaning, and improve experimental efficiency. In practical applications, the present invention has a wide range of application prospects and can be applied to tasks such as automated classification, knowledge graph construction, entity recognition, relationship extraction, event extraction, etc. in the medical field, providing more efficient, accurate and reliable methods and tools for research and practice in the medical field.
[0038] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0040] Figure 1 It is a flowchart of a medical event extraction method based on a generative pre-trained language model;
[0041] Figure 2 It is the Transformer encoder-decoder framework;
[0042] Figure 3 Schematic diagram of the training, fine-tuning and application of the event type discrimination model;
[0043] Figure 4 Schematic diagram of training, fine-tuning and application of event element extraction model;
[0044] Figure 5 A block diagram of a medical event extraction device based on a generative pre-trained language model;
[0045] Figure 6 A schematic diagram of an electronic device. DETAILED DESCRIPTION
[0046] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.
[0047] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0048] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0049] like Figure 1 As shown, a method for extracting medical events based on a generative pre-trained language model includes the following steps:
[0050] S101: Obtain target medical text;
[0051] Specifically, the target medical text is a text composed of natural language characters for event extraction in the medical field. The main ways to obtain medical text data are: documents related to the patient's medical records and examination reports in the online medical system, some medical databases such as PubMed, EMBASE, CNKI, etc. contain medical literature such as journal articles and conference papers in PDF format, and texts in HTML format on online medical websites about medical news, health information, doctor blogs, etc. After data preprocessing, format cleaning and filtering, the required medical text data can be obtained. These medical texts can also be used in the medical field adaptation process of the language model to increase the model modeling ability of medical field texts.
[0052] S102: All medical event types to be extracted are given;
[0053] In one embodiment, the medical event type refers to the various event types experienced by a patient from entering the hospital to leaving the hospital. A given set of medical event types may be, for example: [patient admission event, patient medication event, patient laboratory event, patient transfer to ICU event, patient discharge event].
[0054] S103: defining corresponding elements to be extracted for each type of medical event;
[0055] Specifically, different event types correspond to different event elements to be extracted. For example, for patient medication events, it is necessary to define the elements contained in the event as follows: {patient identification, hospitalization type, start time of medication, end time of medication, drug type, drug, dosage, medication rate}; for patient test events, it is necessary to define the elements contained in the event as follows: {patient identification, hospitalization type, test items, test substances, test categories, test dates, project indicators, indicator units, whether the indicators are abnormal}.
[0056] S104: using an event type discrimination model to determine the event type involved in the target medical text, wherein the event type discrimination model is a generative pre-trained language model that has been domain adapted and fine-tuned for event type discrimination tasks;
[0057] Specifically, the event type discrimination model is a model based on a generative pre-trained large language model. The pre-trained model usually adopts a Transformer architecture and is trained in an autoencoding form. GPT is used as a representative generative pre-trained language model for illustration.
[0058] The GPT model uses the decoder part of the Transformer encoder-decoder framework and is a neural network model based on the attention mechanism. The main principle of Transformer is to encode the input sequence into a multi-level representation and use a decoder to convert it into an output sequence. Specifically, by stacking multiple layers of Transformer encoders, the information in the input sequence is abstracted and summarized step by step. At each position in the input sequence, the encoder encodes the entire sequence through a self-attention mechanism, and uses the common information obtained to enhance the understanding of each position. Finally, the encoder encodes the context information into a high-dimensional vector representation for the decoder to use.
[0059] In the decoding phase, the model uses a round-robin mechanism to generate output sequences step by step. The decoder first accepts a special symbol as input, then predicts the next symbol each time, and uses the prediction result and the previously generated symbol to advance the next round of input. At each time step, the decoder considers all generated tokens before the current position to help decide the next token based on the unique structure of the Transformer decoder. The training process of the entire model adopts a method similar to that of a language model, which improves the accuracy and fluency of language expression by maximizing the conditional probability of the output sequence.
[0060] like Figure 2As shown in the figure, the GPT model cuts the decoder part of the transformer, cuts off the multi-head attention module used to introduce the encoder output, and cooperates with specific downstream models, such as a simple structure like "linear transformation + classification", to form the GPT model structure. Through the combination of pre-training and fine-tuning, the GPT model has acquired strong performance capabilities in language processing tasks, such as text classification, information extraction, machine translation, question-answering systems, etc.
[0061] In step S102, given a set of all medical event types, Figure 3 As shown, we are ready to build and train an event type classification model. First, we adapt the generative pre-trained language model to the medical field, aiming to make the language model better adapt to the field data; then, we fine-tune the specific task to improve the model's performance on the specific task. Due to the high cost of manually annotating data, we need to combine data enhancement methods when building a specific task training data set; finally, we combine the preset prompt word template Prompt1 with the target text text content and input it into the generative pre-trained language model. The model will generate the corresponding answer, and the answer is the event type involved in the target text.
[0062] In a specific implementation, step S104 may include the following sub-steps:
[0063] S1041: Adaptation to the medical field based on a generative pre-trained language model. Specifically, medical-related texts, such as desensitized patient case records, journal articles and conference papers in medical databases, textbooks of various medical disciplines, news information, etc., are used. After word segmentation and conversion of the text into word indexes, they are input into a generative pre-trained language model for pre-training, allowing the model to learn the language probability distribution of professional vocabulary in the medical field and the structure of paragraphs and texts.
[0064] S1042: Task fine-tuning. After domain adaptation, pre-train the language model for a single functional task on a specific task dataset to further improve the adaptability and modeling ability of the language model for specific tasks. The task dataset can be manually annotated with a batch of samples as seed data, and then the training set can be increased using data enhancement methods. The available data enhancement methods are: (1) synonym and near-synonym replacement. (2) back translation, that is, using translation software to translate the original Chinese text into English, and then translate it back into Chinese to obtain a replacement text with similar semantics. (3) Use common language models such as baichuan, qwen, llama, etc. to make it output similar sentences.
[0065] The main task in this step is to fine-tune the event type discrimination task. Using the large model adapted to the medical field as the basic model, a data set for the event type discrimination task is constructed to improve the classification accuracy of the model on this task. The training data set for the event type discrimination task is consistent with the prompt word template Prompt1 in step S1043, that is, the specific definition and examples of the task are given, and the model is allowed to choose from a given list of event types. Finally, the prompt word and the correct answer constitute a training sample. After n rounds of fine-tuning training on a specific task data set, the model will have better performance in discriminating event types than a general language model.
[0066] S1043: Constructing a prompt word template Prompt1 of a generative pre-trained language model for distinguishing the type of medical events;
[0067] As a specific embodiment of step 104, Prompt1 = "Given event types: {event_types}, determine which event types the text {text} involves, and return them in a list type. The following is an example:
[0068] The text entered is 'The patient entered our hospital through the emergency department at 9 am on May 18, 2020. The patient suddenly developed symptoms such as dizziness, nausea, vomiting, and then became unconscious. The family immediately sent the patient to the emergency department. After physical examination: The patient was unconscious, confused, unresponsive to external stimuli, normal limb muscle strength, pupils of equal size and roundness, unresponsive to light, soft neck, and no abnormalities in heart and lung auscultation. Preliminary diagnosis: hypertensive crisis, cerebral hemorrhage to be ruled out.'
[0069] The output answer is [patient admission event].
[0070] When the input text is {text}, the output answer is "
[0071] In Prompt1, according to the medical event type defined in step S102, event_types = [patient admission event, patient medication event, patient test event, etc.], text is the newly input text to be identified as the event type.
[0072] S1044: Generate prompt words including all medical event types and target medical texts according to the prompt word template Prompt1 and input them into the generative pre-trained language model to obtain event types involved in the target medical text.
[0073] S105: extracting elements corresponding to the events of the target medical text using an event extraction model according to the event type, wherein the event extraction model is a generative pre-trained language model that has been fine-tuned for domain adaptation and event element extraction tasks;
[0074] Specifically, the event extraction model to be used in this step is also a generative pre-trained large model. In one embodiment, GPT is used. Further, step S105 includes the following sub-steps:
[0075] S1051: Medical field adaptation based on generative pre-trained language model. The language model pre-trained with medical text in step S1041 can be directly used.
[0076] S1052: Fine-tuning the event element extraction task: The event element extraction task is more complex than the event type discrimination task, and there are many types of elements that need to be extracted, such as time, address, diagnosis, etc. The general language model is not very accurate in the event element extraction task, so it is necessary to fine-tune the event element extraction task in a targeted manner. The training data set for the event element extraction task requires manual annotation of the elements to be extracted for each event type. The annotated samples of each event type are aggregated to form the final training set. The format refers to the prompt word template Prompt2, and the corresponding answer is the extraction result in json format.
[0077] S1053: constructing a prompt word template Prompt2 of an event extraction model for extracting medical events;
[0078] Furthermore, if Figure 4 As shown, step S1053 can be divided into the following stages:
[0079] Given a fixed role for the generative pre-trained large language model, such as letting it take on the role of "medical data mining engineer", let the language model focus on extracting entities / relations / events from text.
[0080] Tell the language model the event type it wants to extract and give the elements to be extracted for that event type.
[0081] Use the principle of in-context learning to input some examples, including positive examples and negative examples.
[0082] As a specific embodiment, Prompt2 = "
[0083] Suppose you are a data mining engineer in the medical field, focusing on extracting entities / relations / events from text.
[0084] The event to be extracted from the text is "patient admission event", and the elements to be extracted include: [patient ID, hospitalization type, address before admission, admission time, admission diagnosis]. If the element does not exist, None is returned. The return result is displayed in json format.
[0085] Here is an example:
[0086] The text entered is 'The patient entered our hospital through the emergency department at 9 am on May 18, 2020. The patient suddenly developed symptoms such as dizziness, nausea, vomiting, and then became unconscious. The family immediately sent the patient to the emergency department. After physical examination: The patient was unconscious, confused, slow to respond to external stimuli, normal limb muscle strength, pupils of equal size and round, slow to respond to light, soft neck, and no abnormalities in heart and lung auscultation. Preliminary diagnosis: hypertensive crisis, cerebral hemorrhage to be ruled out.'
[0087] The output answer is:
[0088] {'patient_id':None,
[0089] 'Type of Hospitalization': 'Emergency',
[0090] 'Pre-admission address': None,
[0091] 'Admission time': 'May 18, 2020 9:00 am',
[0092] 'Diagnosis upon admission': 'Hypertensive crisis, cerebral hemorrhage to be ruled out'
[0093] }
[0094] When the input text is {text}, the output answer is "
[0095] The above is a prompt word template for a generative pre-trained language model for event element extraction. For other event types in the event type set, the same structure can be used to write prompt words, and examples and elements to be extracted corresponding to the event type are given in the prompt words, so that the model can understand the intent of the task more clearly.
[0096] S1054: Generate, based on the prompt word template Prompt2, elements to be extracted corresponding to the target medical text and the events involved, and input them into the event extraction model to obtain the event elements involved in the target medical text;
[0097] Specifically, the format of the model output is the same as that set in Promp2.
[0098] Corresponding to the above-mentioned medical event extraction method based on generative pre-trained language model, Figure 5 As shown, the present application also proposes a medical event extraction device based on a generative pre-trained language model, which may include:
[0099] An acquisition module 21 is used to acquire a target medical text;
[0100] A type-given module 22 is used to give all medical event types to be extracted;
[0101] An element definition module 23, used to define corresponding elements to be extracted for each type of medical event;
[0102] An event type discrimination module 24, used to determine the event type involved in the target medical text using an event type discrimination model, wherein the event type discrimination model is a generative pre-trained language model;
[0103] The event extraction module 25 is used to extract elements corresponding to the events of the target medical text according to the event type using an event extraction model, wherein the event extraction model is a generative pre-trained language model.
[0104] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0105] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.
[0106] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned medical event extraction method based on the generative pre-trained language model. Figure 6 As shown, a hardware structure diagram of a medical event extraction method based on a generative pre-trained language model provided by an embodiment of the present invention is provided for any device with data processing capabilities, except Figure 6 In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.
[0107] Correspondingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by the processor, the medical event extraction method based on the generative pre-trained language model as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.
[0108] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.
[0109] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.
Claims
1. A medical event extraction method based on a generative pre-trained language model, characterized in that: include: Obtain target medical texts; Given all the medical event types to be extracted; Define the corresponding elements to be extracted for each type of medical event; Using an event type discrimination model, the event type involved in the target medical text is determined, wherein the event type discrimination model is a generative pre-trained language model that has been domain adapted and fine-tuned for event type discrimination tasks; using the event type discrimination model, the event type involved in the target medical text is determined, including: adapting the generative pre-trained language model to the medical domain; pre-training a language model for a single functional task on a predetermined task data set to fine-tune the event type discrimination task, that is, using data enhancement methods to increase the training set, and the data enhancement methods include: (1) synonym and near-synonym replacement; (2) back translation, that is, using translation software to translate the original Chinese text into English, and then translate it back into Chinese to obtain a replacement text with similar semantics; (3) using a general language model to output similar sentences; constructing a prompt word template Prompt1 of the event type discrimination model for medical event type discrimination; based on the prompt word template Prompt1, generating prompt words containing all medical event types and target medical texts and inputting them into the event type discrimination model to obtain the event type involved in the target medical text; According to the event type, extracting elements corresponding to the event of the target medical text using an event extraction model, wherein the event extraction model is a generative pre-trained language model that has been fine-tuned for domain adaptation and event element extraction tasks; Wherein, according to the event type, extracting elements corresponding to the event of the target medical text using an event extraction model includes: Adapting generative pre-trained language models to the medical field; Pre-train the language model of a single functional task on a predetermined task dataset to fine-tune the event element extraction task; Construct a prompt word template Prompt2 for an event extraction model for medical event extraction, including: giving a fixed role to the generative pre-trained large language model, telling the language model the event type it wants to extract, giving the elements to be extracted for the event type, and using the principle of in-context learning to input some examples, including positive examples and negative examples; According to the prompt word template Prompt2, the elements to be extracted corresponding to the target medical text and the events involved are generated and input into the event extraction model to obtain the event elements involved in the target medical text.
2. The method according to claim 1, characterized in that: The medical event type refers to the type of event that a patient experiences from entering the hospital to leaving the hospital, including patient admission events, patient medication events, patient laboratory test events, patient transfer to the ICU events, and patient discharge events.
3. The method according to claim 1, characterized in that GPT is used as the event type discrimination model.
4. The method according to claim 1, characterized in that The generative pre-trained language model is adapted to the medical field, specifically: the medical field text is converted into a word index through word segmentation, and the generative pre-trained language model is pre-trained using the word index, wherein the medical field text includes desensitized patient case records, journal articles in medical databases, conference papers, textbooks of various medical disciplines, and news information.
5. The method according to claim 1, characterized in that The predetermined task data set format is consistent with the prompt word template Prompt1.
6. A medical event extraction device based on a generative pre-trained language model, characterized in that: include: An acquisition module, used to acquire target medical text; A type given module is used to give all medical event types to be extracted; The element definition module is used to define the corresponding elements to be extracted for each type of medical event; An event type discrimination module is used to use an event type discrimination model to determine the event type involved in the target medical text, wherein the event type discrimination model is a generative pre-trained language model that has been adapted to the domain and fine-tuned for the event type discrimination task; using the event type discrimination model to determine the event type involved in the target medical text, including: adapting the generative pre-trained language model to the medical domain; pre-training a language model for a single functional task on a predetermined task data set to fine-tune the event type discrimination task, that is, using a data enhancement method to increase the training set, the data enhancement methods include: (1) synonym and near-synonym replacement; (2) back translation, that is, using translation software to translate the original Chinese text into English, and then translate it back to Chinese to obtain a replacement text with similar semantics; (3) using a general language model to output similar sentences; constructing a prompt word template Prompt1 of the event type discrimination model for medical event type discrimination; based on the prompt word template Prompt1, generating prompt words containing all medical event types and target medical texts and inputting them into the event type discrimination model to obtain the event type involved in the target medical text; An event extraction module, configured to extract elements corresponding to the events of the target medical text using an event extraction model according to the event type, wherein the event extraction model is a generative pre-trained language model that has been fine-tuned for domain adaptation and event element extraction tasks; Wherein, according to the event type, extracting elements corresponding to the event of the target medical text using an event extraction model includes: Adapting generative pre-trained language models to the medical field; Pre-train the language model of a single functional task on a predetermined task dataset to fine-tune the event element extraction task; Construct a prompt word template Prompt2 for an event extraction model for medical event extraction, including: giving a fixed role to the generative pre-trained large language model, telling the language model the event type it wants to extract, giving the elements to be extracted for the event type, and using the principle of in-context learning to input some examples, including positive examples and negative examples; According to the prompt word template Prompt2, the elements to be extracted corresponding to the target medical text and the events involved are generated and input into the event extraction model to obtain the event elements involved in the target medical text.
7. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.
8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Event extraction method and system and medium
CN116341536A