Information extraction model training method and device and information extraction method and device

Abstract

The invention provides an information extraction model training method and device, and an information extraction method and device, and the training method comprises the steps: taking a first sample medical record text as a training sample, taking the approaching of the output distribution of a student model to the output distribution of a teacher model as a training target, training the student model, and obtaining an information extraction model; the teacher model takes minimization of the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text as a training target. According to the information extraction model obtained through training, the time dimension of each text sequence in the medical record text can be accurately recognized, the medical record auditing efficiency is improved, and the lightweight information extraction model remarkably reduces the computing resource requirement and the operation cost.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an information extraction model training, information extraction method and device. Background Art

[0002] Medical record text information extraction plays a vital role in the medical insurance review process. It helps to fully and accurately grasp the patient's symptoms, diagnosis results, surgery and treatment process and other key information, thereby providing strong data support and decision-making basis for medical insurance review. However, the current medical record text information extraction technology faces a series of challenges. The unstructured characteristics of medical record texts and the uneven writing quality greatly increase the difficulty of information extraction and affect the accuracy and efficiency of the extraction results.

[0003] At present, information extraction from medical records is mostly based on keywords and rules, that is, using preset keywords and rules to quickly scan and preliminarily analyze medical records using automated tools to extract key information related to medical insurance review. However, due to the complexity and diversity of medical record texts, rule-based extraction methods often cannot cover all situations, resulting in certain limitations on the completeness and accuracy of the extraction results. Summary of the invention

[0004] The present invention provides an information extraction model training, an information extraction method and a device, so as to solve the defects existing in the prior art.

[0005] The present invention provides an information extraction model training method, comprising the following steps: Obtain a pre-trained teacher model, a student model to be trained, and a first sample medical record text; the output of the teacher model is used as a soft label, and the soft label is used to guide the optimization direction of the student model during the training process; The first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal, the student model is trained to obtain an information extraction model, and the output distribution is used to characterize the time dimension of all text sequences in the first sample medical record text; The teacher model is used to extract sentence representations from the second sample medical record text, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation. The teacher model has the training goal of minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0006] According to a method for training an information extraction model provided by the present invention, the first sample medical record text is used as a training sample, the output distribution of the student model is close to the output distribution of the teacher model as a training goal, the student model is trained to obtain an information extraction model, including: Taking the first sample medical record text as a training sample and taking the output distribution of the student model approaching the output distribution of the teacher model as a training goal, training the student model to obtain a pre-trained student model; Based on the conditional probability model, the word segmentation representation extracted from the first sample medical record text by the pre-trained student model is applied to identify the dependency relationship between each word segment in the first sample medical record text, and based on the dependency relationship between each word segment, the time dimension of each text sequence extracted from the first sample medical record text by the pre-trained student model is optimized to obtain an optimized time dimension; Based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text, the pre-trained student model is trained to obtain the information extraction model.

[0007] According to an information extraction model training method provided by the present invention, the time dimension of each text sequence in the first sample medical record text is extracted based on the following steps: The first sample medical record text is divided into multiple parts to obtain multiple sub-medical record texts, and there is an overlap between the beginning and the end of each adjacent sub-medical record text of the current sub-medical record text; According to the order of each sub-medical record text in the first sample medical record text, extracting the time dimension of the corresponding sub-text sequence from each sub-medical record text based on the pre-trained student model; Based on the time dimension of each sub-text sequence, the time dimension of each text sequence in the first sample medical record text is determined.

[0008] According to a method for training an information extraction model provided by the present invention, the pre-trained student model is trained based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text to obtain the information extraction model, including: Determining a first loss value based on a difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text; Based on the first loss value, the parameters of the pre-trained student model are updated to obtain the information extraction model.

[0009] According to an information extraction model training method provided by the present invention, the loss value of the teacher model includes a second loss value, a third loss value and a fourth loss value, the second loss value is used to characterize the difference between the same sentence representations of the teacher model, the third loss value is used to characterize the difference between different sentence representations, and the fourth loss value is used to characterize the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0010] According to an information extraction model training method provided by the present invention, the teacher model is used to extract the representation of each participle in the second sample medical record text based on the context information of each participle in the second sample medical record text, extract the sentence representation based on each participle representation, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation.

[0011] The present invention also provides an information extraction method, comprising the following steps: Determine the target medical record text; Based on the information extraction model, determining the time dimension of each target text sequence in the target medical record text; Based on the time dimension of each target text sequence, extract the target information text from each target text sequence; The information extraction model is trained based on the information extraction model training method as described in any one of the above items.

[0012] The present invention also provides an information extraction model training device, comprising the following modules: An acquisition unit, used to acquire a pre-trained teacher model, a student model to be trained, and a first sample medical record text; the output of the teacher model is used as a soft label, and the soft label is used to guide the optimization direction of the student model during the training process; A training unit, configured to use the first sample medical record text as a training sample and take the output distribution of the student model approaching the output distribution of the teacher model as a training goal, train the student model to obtain an information extraction model, wherein the output distribution is used to characterize the time dimension of all text sequences in the first sample medical record text; The teacher model is used to extract sentence representations from the second sample medical record text, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation. The teacher model has the training goal of minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0013] The present invention also provides an information extraction device, comprising the following modules: A text determination unit, used for determining a target medical record text; A dimension determination unit, used to determine the time dimension of each target text sequence in the target medical record text based on the information extraction model; An extraction unit, used for extracting target information text from each target text sequence based on the time dimension of each target text sequence; The information extraction model is trained based on the information extraction model training method as described in any one of the above items.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the information extraction model training method or information extraction method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the information extraction model training methods or information extraction methods described above.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned information extraction model training methods or information extraction methods.

[0017] The information extraction model training, information extraction method and device provided by the present invention, the teacher model takes minimizing the difference between the same sentence representations, maximizing the difference between different sentence representations, and minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text as the training goal, so that the trained teacher model has strong semantic understanding ability, distinction ability and time understanding ability, and then the trained teacher model can accurately understand the semantic information of complex case texts and accurately identify the time dimension of each text sequence in the medical record text. On this basis, when the present invention uses the first sample medical record text as a training sample, the output distribution of the teacher model is used as a high-quality soft label to train the student model, so as to realize the transfer of the semantic understanding ability, distinction ability and time understanding ability of the teacher model to the lightweight student model, which not only enables the trained information extraction model to accurately identify the time dimension of each text sequence in the medical record text and improve the efficiency of medical record review, but also the lightweight information extraction model significantly reduces the computing resource requirements and operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flow chart of the information extraction model training method provided by the present invention.

[0020] Figure 2 It is a flow chart of the teacher model training method provided by the present invention.

[0021] Figure 3 It is a flow chart of another information extraction model training method provided by the present invention.

[0022] Figure 4 It is a flow chart of the information extraction method provided by the present invention.

[0023] Figure 5 It is a structural schematic diagram of the information extraction model training device provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the information extraction device provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Contemporary medical insurance audits have gradually shifted from manual audits to automatic audits by computer systems. In the process of developing medical insurance audit systems, data analysis and processing of medical record texts are necessary steps in the audit. However, due to the uneven quality of medical record writing in various regions or hospitals and the large amount of unstructured large texts, it poses a great challenge to the understanding of medical record texts during the audit. In the full medical record medical insurance audit, the system needs to conduct a comprehensive analysis of the patient's symptoms, diagnosis, surgery, examination, testing and other events and compare them with the standards established in the system to determine whether there may be violations in the medical record. However, whether these events can serve as violation conditions or evidence to support the audit often requires clarifying whether the time node or nature of their occurrence is related to this hospitalization. Therefore, it is of great significance to accurately identify the time dimension of all text sequences in the medical record text, which is mainly reflected in the following aspects: (1) Distinguishing between medical history and current condition: In the medical record text, the patient's medical history before hospitalization often contains information such as previous illnesses, family history, and surgical history. This information helps to fully understand the patient's health status, but is not directly related to the medical insurance expenses of this hospitalization. In contrast, the medical record during hospitalization mainly records the patient's condition, diagnosis, and treatment during this hospitalization, which directly affects medical insurance payments. By distinguishing this information, it is possible to more clearly define what should be included in the scope of medical insurance reimbursement and avoid misclassifying events described in previous medical history as current hospitalization expenses.

[0028] (2) Accurately verify the treatment process: The medical records during hospitalization record the diagnosis, examination, treatment plan and medication in detail. By accurately distinguishing these records, the patient's treatment path can be clearly reflected to ensure that the medical services provided by the hospital match the patient's condition. If the treatment during hospitalization in the medical record is unrelated to the patient's condition or there is excessive medical treatment, the medical insurance review can quickly identify and make adjustments to prevent unnecessary expense reimbursement.

[0029] (3) Identify the time of examination and surgery: Examination and surgery are the main components of hospitalization expenses. By distinguishing the examination items before and during hospitalization, medical insurance audits can determine which examinations were completed before hospitalization and which were performed after hospitalization, thereby avoiding reimbursement of duplicate examination expenses. At the same time, surgical records can reflect the time and measures of the patient's actual treatment, which helps to evaluate the rationality and effectiveness of the surgery and prevent overcharging or unnecessary surgical expenses from being included in reimbursement.

[0030] (4) Preventing irregularities: By distinguishing the medical records before hospitalization, during hospitalization, and after discharge, it can help identify irregularities. For example, some hospitals may unreasonably extend the length of hospitalization, or continue to prescribe treatment and examinations during hospitalization after the patient is discharged, with the intention of increasing the reimbursement amount. By strictly distinguishing the information in the medical record text, such irregular operations can be effectively identified and improper expenses can be prevented from being included in medical insurance reimbursement.

[0031] (5) Post-discharge follow-up and treatment monitoring: Medical records that record events that have not yet occurred usually contain rehabilitation plans, follow-up arrangements, and possible subsequent treatments. Although this information is important for the patient's recovery, it does not always meet the medical insurance reimbursement standards during hospitalization. Therefore, accurately distinguishing the treatment plan after discharge can ensure that medical insurance payments are limited to hospitalization expenses within a reasonable range and prevent the patient's recovery expenses from being mistakenly included in the medical insurance reimbursement during hospitalization.

[0032] At present, medical insurance audits mainly rely on the professional knowledge and experience of auditors, who judge the possibility of violations by comparing patients' hospitalization records, medical expense lists, and treatment information. Auditors need to carefully read a large number of electronic medical records, identify the time dimension of medical events, and make comprehensive judgments. However, manual review of a large number of medical records is time-consuming and labor-intensive, and inefficient. Long-term audits can easily cause auditor fatigue and affect the accuracy of judgments. In addition, different auditors may have different judgments on the same situation, resulting in poor consistency in audit results.

[0033] In addition, there are also information extraction methods based on keywords and rules for medical record texts, that is, using preset keywords and rules to quickly scan and preliminarily analyze medical record texts using automated tools to extract key information related to medical insurance review. However, due to the complexity and diversity of medical record texts, rule-based extraction methods often find it difficult to cover all situations, resulting in certain limitations on the completeness and accuracy of the extraction results. For example, time information in medical record texts may appear in many forms, such as exact dates (XX / XX / XXXX), relative time descriptions (such as "3 days ago"), or vague expressions (such as "recently"). Preset keywords and rules are difficult to cover all situations and are prone to missing complex or newly emerging expressions.

[0034] In addition, the time dimension of some medical events may need to be combined with the overall medical record content to be accurately judged. Keywords and rules cannot accurately understand the context and implicit meaning of the text, which may lead to misjudgment. The medical record format and writing style of different medical institutions may be different, and synonyms and ambiguous words often appear. The medical insurance review system based on keywords and rules does not have good generalization ability and cannot adapt to various medical record texts.

[0035] It can be seen from this that the current medical insurance review system is unable to determine the time dimension of the text sequence in the medical record, which will result in the inability to utilize the time series information in the entire medical record. Many medical event information cannot be accurately extracted as evidence to support the judgment of violations, resulting in misjudgments and missed judgments, reducing the overall precision and recall of the system.

[0036] In this regard, the present invention provides an information extraction model training method. The trained information extraction model can accurately distinguish the time dimensions of all text sequences in the medical record text, and filter a large number of medical events that are not related to the current hospitalization according to the time dimensions of each text sequence. While improving the possibility of medical text analysis, it provides validity protection for the audit results and provides accurate auxiliary support for medical insurance audits.

[0037] Figure 1 It is a flow chart of the information extraction model training method provided by the present invention, such as Figure 1 As shown, the method includes step 110 and step 120.

[0038] Step 110, obtain a pre-trained teacher model, a student model to be trained, and a first sample medical record text, and use the output of the teacher model as a soft label. The soft label is used to guide the optimization direction of the student model during the training process, so that the student model can gradually approach and imitate the performance of the teacher model.

[0039] Here, the teacher model and the student model can be models in any scenario. It should be noted that the structures of the teacher model and the student model are similar, but the difference is that the teacher model has a larger number of parameters.

[0040] As an optional embodiment, the teacher model can be a large language model (LLM), which has strong semantic understanding capabilities and can accurately grasp the complex and diverse temporal information and medical events in medical record texts. The student model can be a Longformer model, which uses a transformer architecture to process ultra-long text sequences. The maximum window length (the maximum token length processed at one time) reaches 4096, and the computational complexity is reduced by introducing a local window attention mechanism and a global attention mechanism.

[0041] The teacher model and the student model are used to parse the temporal information in the medical record text, such as dividing the text sequence in the medical record text into four time dimensions: past, present, future, and meaningless.

[0042] Among them, the past means that the events described in the medical record text occurred before the patient's current hospitalization, usually involving medical history, previous surgical history, family medical history, etc. For example, "the patient had heart surgery three years ago" or "the patient has a family history of hypertension".

[0043] Now means that the events in the medical record text are directly related to the patient's current hospitalization, usually the current diagnosis, treatment, surgery, and medication records. The text sequence corresponding to "now" is the core content of medical insurance review and determines which medical operations should be reimbursed.

[0044] Future means that the events described in the medical record text have not yet occurred, such as rehabilitation plans, follow-up arrangements, or expected subsequent treatments. The text sequence corresponding to "future" is often seen in discharge plans and long-term treatment recommendations given by doctors. It has no direct impact on the current hospitalization process and should not be included in the judgment of the medical insurance review system.

[0045] Meaningless is mainly used to mark information that is not directly related to the temporal analysis of medical records, usually including descriptive content with no medical value or sentences irrelevant to the main theme of the medical record. For example, some formatting or irrelevant statements in the medical record. The text sequence corresponding to "meaningless" is of no substantial help to the medical insurance review and needs to be filtered out to ensure that the system focuses on meaningful time information and medical events, and improve the accuracy and efficiency of analysis.

[0046] It can be seen from this that when conducting medical insurance review, based on the time dimension corresponding to the above-mentioned text sequences, irrelevant information of medical insurance review in the medical record text can be filtered out, and relevant information of medical insurance review can be extracted from the medical record text, thereby greatly improving the accuracy of medical insurance review.

[0047] Step 120: Using the first sample medical record text as a training sample and taking the output distribution of the student model approaching the output distribution of the teacher model as a training goal, the student model is trained to obtain an information extraction model, and the output distribution is used to characterize the time dimension of all text sequences in the first sample medical record text; the output distribution can be the probability distribution of all text sequences in the first sample medical record text belonging to different time dimensions, and the different time dimensions can include absolute time, relative time, time ambiguity, and the type of event and contextual relationship.

[0048] The teacher model is trained based on the second sample medical record text and the time dimension labels of each text sequence in the second sample medical record text. The teacher model is used to extract sentence representations from the second sample medical record text and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation. The teacher model takes minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension labels of each text sequence in the second sample medical record text as its training goal.

[0049] Specifically, the output distribution of the student model can be understood as the student model's predicted distribution of the time dimension of all text sequences in the first sample medical record text, and the output distribution of the teacher model can be understood as the teacher model's predicted distribution of the time dimension of all text sequences in the first sample medical record text.

[0050] When the student model is trained with the first sample medical record text as the training sample, the output distribution of the teacher model is used as a high-quality soft label, and the parameters of the student model are adjusted to achieve the transfer of knowledge in the teacher model to the student model to obtain an information extraction model. The distillation loss value can be determined based on the difference between the output distribution of the student model and the output distribution of the teacher model, and the parameters of the student model are updated based on the distillation loss value to obtain the information extraction model. The distillation loss value can be measured using the Kullback-Leibler divergence (KL divergence), which is used to measure the distance between the output distribution of the student model and the output distribution of the teacher model. The KL divergence can be determined based on the following formula: in, represents the KL divergence, represents the output distribution of the teacher model, represents the output distribution of the student model.

[0051] The smaller the distance between the output distribution of the student model and the output distribution of the teacher model, the closer the output distribution of the student model is to the output distribution of the teacher model, and the more effectively the teacher model's powerful semantic understanding ability and accurate recognition of the time dimension of each text sequence in the medical record text can be transferred to the lightweight student model, so that the trained information extraction model can accurately identify and extract relevant content (such as extracting text related to this hospitalization) based on the time dimension of each text sequence in the medical record text. Among them, the time dimension of each text sequence refers to the time order corresponding to each text sequence in the medical record text, and the time dimension here can include the past, present, future, and meaningless.

[0052] In addition, the teacher model is trained based on the second sample medical record text and the information label of the second sample medical record text. The second sample medical record text here can be the same as the first sample medical record text or different from the first sample medical record text.

[0053] The teacher model is used to extract sentence representations from the second sample medical record text, and the sentence representations are used to characterize the rich semantic information of the second sample medical record text. The teacher model determines the time dimension of each text sequence in the second sample medical record text based on the sentence representations containing rich semantic information, that is, the teacher model distinguishes the temporal order of different text sequences in the second sample medical record text. The text sequence here can be a character, a word, a phrase, a sentence, or a larger text block. For example, if multiple consecutive sentences correspond to the same time dimension, the text sequence is a text block consisting of multiple sentences.

[0054] Furthermore, the teacher model can be trained with the objectives of minimizing the difference between representations of the same sentence, maximizing the difference between representations of different sentences, and minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0055] In addition, the training objectives of the teacher model include three: ① minimizing the difference between the representations of the same sentence; ② maximizing the difference between the representations of different sentences; ③ minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0056] Among them, training goal ① aims to improve the teacher model's semantic understanding and feature extraction capabilities. By minimizing the differences between the representations of the same sentence, the teacher model can learn how to generate stable and consistent sentence representations that can accurately reflect the semantic content of the sentence. In other words, this training goal can improve the teacher model's understanding of repeated information and synonymous expressions in the text, and improve the robustness of the teacher model's semantic understanding.

[0057] Training objective ② aims to improve the discrimination ability or discriminative ability of the teacher model. By maximizing the difference between different sentence representations, the teacher model can better distinguish sentences with different semantic contents, which helps the teacher model to accurately identify and distinguish different information points in a complex text environment.

[0058] Training goal ③ aims to improve the teacher model's time comprehension or time series analysis ability. In medical record text processing, the time dimension is a key factor in understanding the progression of the disease, the treatment process, and changes in the patient's status. By minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text, the teacher model can learn how to accurately identify the time information in the medical record text and understand the impact of this time information on the overall meaning of the text.

[0059] In summary, the above training objectives respectively train the teacher model's semantic understanding ability, distinction ability, and time understanding ability, so that the trained teacher model can accurately understand the semantic information of the medical record text when processing complex medical record texts, and accurately identify the time dimension of each text sequence in the medical record text.

[0060] The information extraction model training method provided by the embodiment of the present invention is that the teacher model takes minimizing the difference between the representations of the same sentence, maximizing the difference between the representations of different sentences, and minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text as the training goal, so that the trained teacher model has strong semantic understanding ability, distinction ability and time understanding ability, and then the trained teacher model can accurately understand the semantic information of complex case texts and accurately identify the time dimension of each text sequence in the medical record text. On this basis, when the embodiment of the present invention uses the first sample medical record text as a training sample, the output distribution of the teacher model is used as a high-quality soft label to train the student model, so as to realize the transfer of the semantic understanding ability, distinction ability and time understanding ability of the teacher model to the lightweight student model, which not only enables the trained information extraction model to accurately identify the time dimension of each text sequence in the medical record text and improve the efficiency of medical record review, but also the lightweight information extraction model significantly reduces the computing resource requirements and operating costs.

[0061] As an optional embodiment, Figure 2 It is a flow chart of the teacher model training method provided by the present invention, such as Figure 2 As shown, the teacher model can be trained based on the following steps: Considering that the average character length of medical record text is relatively long (usually more than 10,000 characters), the traditional encoder model cannot generate the semantic representation of the entire medical record text in one reasoning. In this regard, the embodiment of the present invention uses LLM as the encoder of the teacher model, and its maximum context window length is usually 128k tokens. In addition, LLM is a transformer-based decoder structure, which is mainly used for autoregressive generation, that is, it depends on the previous word (or token) to gradually generate the next word, that is, each step of generation is based on the context of the previously generated word and the input sequence, which may have efficiency issues for text sequence labeling tasks.

[0062] Considering that the encoder can process all tokens in the text sequence in parallel, the embodiment of the present invention converts the LLM into an encoder structure, generates an independent representation for each token, and directly performs classification prediction. Specifically: The traditional decoder model uses a causal attention mechanism, that is, each word can only pay attention to the words before it, which limits the representation ability of the model. The embodiment of the present invention replaces the mask matrix of causal attention (i.e., the mask with 1 below the diagonal and 0 above the diagonal) with a mask matrix of all 1s, so that each word in the teacher model can access all other words in the text sequence at the same time. After enabling bidirectional attention, each token in each layer of the teacher model can use the context information to build its representation.

[0063] In order to adapt the model to the bidirectional attention mechanism, an embodiment of the present invention combines masked language modeling (randomly masking some words in the input sequence) and next word prediction. The teacher model predicts the masked words through past and future contexts, so that the teacher model can learn better word-level representation based on bidirectional attention.

[0064] After completing bidirectional attention and masked next word prediction, the teacher model can generate reasonable word segmentation representations, but in order to enable the teacher model to generate better sentence representations or sequence representations, the teacher model can be trained using massive medical corpora and the unsupervised contrastive learning method SimCSE (Simple Contrastive Learning of Sentence Embeddings). The teacher model generates two different representations for the same sentence and maximizes the similarity between them while minimizing the similarity with other sentence representations, thereby helping the model learn high-quality sequence-level representations from unlabeled data.

[0065] After completing the training of masked next word prediction and SimCSE, the teacher model can generate effective word segmentation representation and sentence representation. In order to convert the representation of multiple words in a text sequence into a vector of fixed dimension (i.e., text embedding), a pooling method can be introduced, such as EOS Token Pooling (sentence end symbol pooling), that is, the hidden state of the last token or a specific token is used as the representation of the entire sentence. In order to generate more powerful sentence or sequence-level embeddings, Mean Pooling can be used, which can comprehensively consider all tokens in the entire sequence, avoiding the limitation that a single token is not enough to capture the complex interactive relationship of all words in the sequence.

[0066] After completing the adjustment and training of the above model structure, the teacher model can be fine-tuned through supervised training using the second sample medical record text and the time dimension labels of each text sequence in the second sample medical record text. For example, a fully connected layer and a softmax function are connected after the LLM encoder to map all text sequence representations in the second sample medical record text to the time dimension, and the cross entropy loss function is used to determine the loss value. The teacher model parameters are fine-tuned based on the loss value to obtain a trained teacher model.

[0067] Among them, examples of the second sample medical record text and the time dimension labels of each text sequence in the second sample medical record text are as follows: 1. General condition: Patient Hu**, female, 33 years old. 2. Medical history: 1) 37+1 weeks pregnant, scarred uterus, awaiting delivery. 2) Expected date of delivery: ** / ** / ****, no history of dizziness, degeneration, blurred vision, generalized edema, and skin itching in the middle and late stages of pregnancy. Regular prenatal check-ups during pregnancy, normal glycocholic acid, and no special abnormalities. Currently 37+1 weeks pregnant, admitted to the hospital due to "37+1 weeks pregnant, scarred uterus, awaiting delivery". Recently, the patient has no abnormalities in diet and sleep. 3) Denies hepatitis, tuberculosis, hypertension, heart disease, diabetes, cerebrovascular disease, mental illness, and food and drug allergies. 3. Physical examination: *********. 4. Auxiliary examination: None.

[0068] Among them, the time dimension label of the text sequence corresponding to the above “1) 37+1 weeks pregnant, scarred uterus, awaiting delivery”, “3. Physical examination: *********” and “Currently 37+1 weeks pregnant, admitted to the hospital due to “37+1 weeks pregnant, scarred uterus, awaiting delivery”. The patient has no abnormal diet and sleep in recent days” is “now”, and the time dimension label of the text sequence corresponding to the above “No history of dizziness, evolution, blurred vision, generalized edema and skin itching in the middle and late pregnancy. Regular prenatal check-ups during pregnancy, normal glycocholic acid, no special abnormalities” and “Denial of hepatitis, tuberculosis, hypertension, heart disease history, diabetes, cerebrovascular disease, mental illness history, food, drug allergy history” is “past”, and the time dimension label of the text sequence corresponding to the above “Expected date of delivery: **** year ** month ** day” is “future”, and the time dimension label of the text sequence corresponding to the above “1. General condition: patient Hu**, female, 33 years old” and “4. Auxiliary examination: None” is “meaningless”.

[0069] Among them, when determining the time dimension label, corresponding prompt words can be set for each medical record text, and multiple large models can be used to standardize each medical record text using prompt words, and based on the voting mechanism, the labeling results output by each large model are merged to obtain the final time dimension label for each medical record text to improve the accuracy and robustness of the labeling. In addition, medical experts can be invited to manually review and correct the labeling results generated by the large model, thereby greatly reducing the time of labeling from scratch, while ensuring the high quality and accuracy of the final labeling results.

[0070] Based on the above embodiment, the first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal. The student model is trained to obtain an information extraction model, including: The first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal, and the student model is trained to obtain a pre-trained student model; Based on the conditional probability model, the word segmentation representation extracted from the first sample medical record text by the pre-trained student model is applied to identify the dependency relationship between each word segment in the first sample medical record text, and based on the dependency relationship between each word segment, the time dimension of each text sequence extracted from the first sample medical record text by the pre-trained student model is optimized to obtain an optimized time dimension; Based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text, the pre-trained student model is trained to obtain an information extraction model.

[0071] Specifically, the first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal. The pre-trained student model obtained by training is to independently classify each token in the first sample medical record text to obtain the time dimension of each text sequence. In other words, the pre-trained student model determines the time dimension of each text sequence from a local optimal perspective, and the time dimension of each text sequence obtained may have an unreasonable time dimension, or incoherent jump phenomenon, affecting the coherence of the text sequence of the same time dimension. For example, a text sequence of a different dimension appears between two text sequences of the same time dimension, thereby affecting the coherence of the two text sequences of the same time dimension.

[0072] To this end, an embodiment of the present invention introduces a conditional probability model to apply the word segmentation representation extracted from the first sample medical record text by the pre-trained student model, identifies the dependency relationship between each word segment in the first sample medical record text, and optimizes the time dimension of each text sequence extracted from the first sample medical record text by the pre-trained student model based on the dependency relationship between each word segment to obtain an optimized time dimension.

[0073] As an optional embodiment, a conditional probability model, such as a conditional random field (CRF), may be added after the last hidden layer of the pre-trained student model. The CRF may model two adjacent time dimensions in the medical record text based on the dependency between each word segment, and learn patterns in which certain time dimensions are unlikely to appear continuously, thereby reducing the occurrence of unreasonable time dimensions.

[0074] Compared with independent classification of each token, CRF can consider the semantic information of the entire text globally, ensuring that the time dimension of the output of the pre-trained student model is globally optimal rather than locally optimal for each token.

[0075] It can be seen that CRF can smooth the time dimension of the decoded output of the pre-trained student model by modeling the relationship between each token, avoiding incoherent jumps.

[0076] After obtaining the optimized time dimension based on the conditional probability model, the pre-trained student model is trained based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text to obtain an information extraction model. Optionally, the loss value can be determined based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text, and the parameters of the pre-trained student model can be updated based on the loss value to obtain an information extraction model.

[0077] Based on any of the above embodiments, the time dimension of each text sequence in the first sample medical record text is extracted based on the following steps: The first sample medical record text is segmented into multiple parts to obtain multiple sub-medical record texts, and there is a head-to-tail overlap between every two adjacent sub-medical record texts of the current sub-medical record text. In the order of each sub-medical record text in the first sample medical record text, the time dimension of the corresponding sub-text sequence is sequentially extracted from each sub-medical record text based on the pre-trained student model. Based on the time dimensions of each sub-text sequence, the time dimensions of each text sequence in the first sample medical record text are determined.

[0078] Specifically, although the student model uses the Longformer model to expand the maximum window length, it may still not meet the processing requirements of some medical record texts. In this regard, the embodiments of the present invention adopt the method of context window overlap, so that the student model can adaptively process medical record texts of different lengths.

[0079] The embodiments of the present invention divide the first sample medical record text into multiple windows to implement segmenting the first sample medical record text into multiple parts to obtain multiple sub-medical record texts. Among them, the length of each window is L (here L can be the maximum window length of the student model, for example, for the Longformer model, the maximum window length is 4096). If the total length of the first sample medical record text is T, then the corresponding number of windows N = T / L, and the number of windows is the number of sub-medical record texts.

[0080] To ensure the information continuity between each window, an overlapping area can be set between adjacent windows, and the length of the overlapping area is O (O < L). Then the start position of each window is , and the end position is , where i starts from 0 until the window covers the entire medical record text.

[0081] In the overlapping area, the end of the text in the previous window overlaps with the text in the current window, and the length of the overlapping area is O, that is, the sub-medical record text corresponding to the previous window and the sub-medical record text corresponding to the current window overlap at the beginning and the end (context overlap). Based on the context information of the overlapping area corresponding to each sub-medical record text, the semantic features of each sub-medical record text can be accurately extracted to avoid information gaps between different sub-medical record texts, reduce boundary effects and improve the richness of semantic features.

[0082] On this basis, according to the order of each sub-medical record text in the first sample medical record text, the time dimension of the corresponding sub-text sequence is extracted from each sub-medical record text based on the pre-trained student model, and the time dimension of each text sequence in the first sample medical record text is determined based on the time dimension of each sub-text sequence. For example, the time dimensions of the same sub-text sequences can be merged to obtain the time dimensions of each text sequence in the first sample medical record text.

[0083] Based on any of the above embodiments, the pre-trained student model is trained based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text to obtain an information extraction model, including: Determine a first loss value based on a difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text; Based on the first loss value, the parameters of the pre-trained student model are updated to obtain the information extraction model.

[0084] Specifically, the smaller the difference between the optimized time dimension and the time dimension label of each text sequence in the first sample medical record text, the smaller the first loss value, and thus the more the pre-trained student model can accurately identify the time dimension of each text sequence in the medical record text while ensuring the coherence of the time dimension.

[0085] Optionally, the first loss value It can be determined based on the following formula: in, It is the probability of the conditional probability model outputting the correct time dimension given the current input and the previous time dimension. are the conditional probability model parameters.

[0086] Based on any of the above embodiments, the loss value of the teacher model includes a second loss value, a third loss value and a fourth loss value, the second loss value is used to characterize the difference between the same sentence representations of the teacher model, the third loss value is used to characterize the difference between different sentence representations, and the fourth loss value is used to characterize the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0087] Specifically, the second loss value is used to measure whether the sentence representations generated by the teacher model for similar or identical sentences are close enough. By minimizing the second loss value, the teacher model can learn how to map semantically similar or identical sentences into similar representation spaces, thereby improving the generalization ability and robustness of the teacher model.

[0088] The third loss value is used to measure whether the sentence representations generated by the teacher model for different sentences are sufficiently distinguishable. By maximizing this third loss value, the teacher model can learn how to map semantically different sentences into different representation spaces, thereby enhancing the teacher model's discriminative ability.

[0089] The fourth loss value is used to measure whether the teacher model can accurately capture the time dimension information in the text. By minimizing the fourth loss value, the teacher model can learn how to accurately map the time dimension information in the medical record text to its corresponding label, thereby improving the accuracy of the teacher model in identifying the time dimension information in the medical record text.

[0090] In summary, the three loss values ​​of the teacher model work together in the training process of the teacher model, ensuring that the teacher model can learn effective sentence representations, the ability to distinguish different sentences, and accurately capture the time dimension information in the medical record text.

[0091] Based on any of the above embodiments, the teacher model is used to extract the representation of each participle in the second sample medical record text based on the context information of each participle in the second sample medical record text, extract the sentence representation based on each participle representation, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation.

[0092] Specifically, the teacher model can extract the representation of each word based on the context information of each word in the second sample medical record text, and after obtaining the representation of each word, combine the word representations (such as by simple averaging or weighted averaging) to obtain the sentence representation. The sentence representation can capture the key information in the sentence, including the semantic relationship between the word segments, the syntactic structure, and the overall semantics.

[0093] Based on the extracted sentence representations, the teacher model can further analyze the time dimension of each text sequence in the second sample medical record text to obtain the time dimension of each text sequence in the second sample medical record text.

[0094] In summary, when processing the second sample medical record text, the teacher model can accurately capture the key information in the text through a hierarchical representation extraction method, and determine the time dimension of the text sequence based on this information.

[0095] Based on any of the above embodiments, Figure 3is a flow chart of another information extraction model training method provided by the present invention, such as Figure 3 As shown, the method includes: Obtain sample medical record texts, and annotate the time dimension labels of each text sequence in the sample medical record texts. The sample medical record texts here may be the first sample medical record texts, the second sample medical record texts, or the first sample medical record texts and the second sample medical record texts.

[0096] Next, the teacher model is trained based on the sample medical record text. The teacher model is constructed by converting the LLM into an encoder structure, using the sample medical record text as the training sample, with the training objectives of minimizing the difference between the representations of the same sentence, maximizing the difference between the representations of different sentences, and minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text, and unsupervised training to obtain the initial teacher model.

[0097] Next, based on the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text, the initial teacher model is fine-tuned to obtain a trained teacher model.

[0098] After obtaining the trained teacher model, the sample medical record text is input into the teacher model to obtain the output distribution of the teacher model, which is regarded as the soft label of distillation training. The sample medical record text is used as the training sample, and the output distribution of the student model is close to the output distribution of the teacher model as the training goal. The student model is trained to obtain the pre-trained student model.

[0099] Next, a CRF layer is connected to the pre-trained student model. Based on CRF, the word segmentation representation extracted from the sample medical record text by the pre-trained student model is applied to identify the dependency between each word in the sample medical record text. Based on the dependency between each word, the time dimension of each text sequence extracted from the sample medical record text by the pre-trained student model is optimized to obtain the optimized time dimension.

[0100] Based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text, the pre-trained student model is trained to obtain an information extraction model.

[0101] Among them, the sample medical record text is divided into multiple windows by overlapping context windows, so that the first sample medical record text is divided into multiple parts to obtain multiple sub-medical record texts. In order to ensure the continuity of information between each window, an overlapping area can be set between adjacent windows. According to the order of each sub-medical record text in the first sample medical record text, the time dimension of the corresponding sub-text sequence is extracted from each sub-medical record text based on the pre-trained student model. Based on the time dimension of each sub-text sequence, the time dimension of each text sequence in the first sample medical record text is determined.

[0102] Based on any of the above embodiments, Figure 4 It is a flow chart of the information extraction method provided by the present invention, such as Figure 4 As shown, the method includes: Step 410: determine the target medical record text; Step 420: Determine the time dimension of each target text sequence in the target medical record text based on the information extraction model; Step 430: extracting target information text from each target text sequence based on the time dimension of each target text sequence; The information extraction model is trained based on the information extraction model training method described in any of the above embodiments.

[0103] Specifically, the target medical record text is the text that needs to be extracted. Based on the information extraction model described in any of the above embodiments, the time dimension of each target text sequence in the target medical record text can be accurately determined, and then the target text sequence corresponding to the target time dimension (such as the "now" time dimension) can be spliced ​​into a target information text, which can be a text related to medical insurance review.

[0104] The information extraction model training device provided by the present invention is described below. The information extraction model training device described below and the information extraction model training method described above can be referenced to each other.

[0105] Based on any of the above embodiments, Figure 5 is a schematic diagram of the structure of the information extraction model training device provided by the present invention, such as Figure 5 As shown, the device comprises: An acquisition unit 510 is used to acquire a pre-trained teacher model, a student model to be trained, and a first sample medical record text; the output of the teacher model is used as a soft label, and the soft label is used to guide the optimization direction of the student model during the training process; A training unit 520 is used to train the student model using the first sample medical record text as a training sample and taking the output distribution of the student model approaching the output distribution of the teacher model as a training goal to obtain an information extraction model, wherein the output distribution is used to characterize the time dimension of all text sequences in the first sample medical record text; The teacher model is trained based on the second sample medical record text and the time dimension labels of each text sequence in the second sample medical record text. The teacher model is used to extract sentence representations from the second sample medical record text and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation. The teacher model takes minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension labels of each text sequence in the second sample medical record text as its training goal.

[0106] Based on any of the above embodiments, the first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal, and the student model is trained to obtain an information extraction model, including: The first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal, and the student model is trained to obtain a pre-trained student model; Based on the conditional probability model, the word segmentation representation extracted from the first sample medical record text by the pre-trained student model is applied to identify the dependency relationship between each word segment in the first sample medical record text, and based on the dependency relationship between each word segment, the time dimension of each text sequence extracted from the first sample medical record text by the pre-trained student model is optimized to obtain an optimized time dimension; Based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text, the pre-trained student model is trained to obtain an information extraction model.

[0107] Based on any of the above embodiments, the time dimension of each text sequence in the first sample medical record text is extracted based on the following steps: The first sample medical record text is divided into multiple parts to obtain multiple sub-medical record texts, and there is an overlap between the beginning and the end of each adjacent sub-medical record text of the current sub-medical record text; According to the order of each sub-medical record text in the first sample medical record text, the time dimension of the corresponding sub-text sequence is extracted from each sub-medical record text based on the pre-trained student model; Based on the time dimension of each sub-text sequence, the time dimension of each text sequence in the first sample medical record text is determined.

[0108] Based on any of the above embodiments, the pre-trained student model is trained based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text to obtain an information extraction model, including: Determine a first loss value based on a difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text; Based on the first loss value, the parameters of the pre-trained student model are updated to obtain the information extraction model.

[0109] Based on any of the above embodiments, the loss value of the teacher model includes a second loss value, a third loss value and a fourth loss value, the second loss value is used to characterize the difference between the same sentence representations of the teacher model, the third loss value is used to characterize the difference between different sentence representations, and the fourth loss value is used to characterize the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

[0110] Based on any of the above embodiments, the teacher model is used to extract the representation of each participle in the second sample medical record text based on the context information of each participle in the second sample medical record text, extract the sentence representation based on each participle representation, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation.

[0111] Based on any of the above embodiments, Figure 6 is a schematic diagram of the structure of the information extraction device provided by the present invention, such as Figure 6 As shown, the device comprises: A text determination unit 610, used to determine a target medical record text; A dimension determination unit 620, for determining the time dimension of each target text sequence in the target medical record text based on the information extraction model; An extraction unit 630 is used to extract target information text from each target text sequence based on the time dimension of each target text sequence; The information extraction model is trained based on the information extraction model training method described in any of the above embodiments.

[0112] Figure 7 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the information extraction model training method or the information extraction method.

[0113] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0114] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the information extraction model training method or the information extraction method provided by the above methods.

[0115] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the information extraction model training method or the information extraction method provided by the above-mentioned methods.

[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training an information extraction model, characterized in that: include: Obtain a pre-trained teacher model, a student model to be trained, and a first sample medical record text; The output of the teacher model is used as a soft label, and the soft label is used to guide the optimization direction of the student model during the training process; The first sample medical record text is used as a training sample, and the output distribution of the student model is close to the output distribution of the teacher model as a training goal, the student model is trained to obtain an information extraction model, and the output distribution is used to characterize the time dimension of all text sequences in the first sample medical record text; The teacher model is used to extract sentence representations from the second sample medical record text, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation. The teacher model has the training goal of minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

2. The information extraction model training method according to claim 1, characterized in that: The method of taking the first sample medical record text as a training sample and taking the output distribution of the student model approaching the output distribution of the teacher model as a training goal, training the student model to obtain an information extraction model includes: Taking the first sample medical record text as a training sample and taking the output distribution of the student model approaching the output distribution of the teacher model as a training goal, training the student model to obtain a pre-trained student model; Based on the conditional probability model, the word segmentation representation extracted from the first sample medical record text by the pre-trained student model is applied to identify the dependency relationship between each word segment in the first sample medical record text, and based on the dependency relationship between each word segment, the time dimension of each text sequence extracted from the first sample medical record text by the pre-trained student model is optimized to obtain an optimized time dimension; Based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text, the pre-trained student model is trained to obtain the information extraction model.

3. The information extraction model training method according to claim 2, characterized in that: The time dimension of each text sequence in the first sample medical record text is extracted based on the following steps: The first sample medical record text is divided into multiple parts to obtain multiple sub-medical record texts, and there is an overlap between the beginning and the end of each adjacent sub-medical record text of the current sub-medical record text; According to the order of each sub-medical record text in the first sample medical record text, extracting the time dimension of the corresponding sub-text sequence from each sub-medical record text based on the pre-trained student model; Based on the time dimension of each sub-text sequence, the time dimension of each text sequence in the first sample medical record text is determined.

4. The information extraction model training method according to claim 2, characterized in that: The method of training the pre-trained student model based on the difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text to obtain the information extraction model includes: Determining a first loss value based on a difference between the optimized time dimension and the time dimension labels of each text sequence in the first sample medical record text; Based on the first loss value, the parameters of the pre-trained student model are updated to obtain the information extraction model.

5. The information extraction model training method according to any one of claims 1 to 4, characterized in that: The loss value of the teacher model includes a second loss value, a third loss value and a fourth loss value. The second loss value is used to characterize the difference between the same sentence representations of the teacher model, the third loss value is used to characterize the difference between different sentence representations, and the fourth loss value is used to characterize the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

6. The information extraction model training method according to any one of claims 1 to 4, characterized in that: The teacher model is used to extract the representation of each participle in the second sample medical record text based on the context information of each participle in the second sample medical record text, extract the sentence representation based on each participle representation, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation.

7. An information extraction method, characterized in that: include: Determine the target medical record text; Based on the information extraction model, determining the time dimension of each target text sequence in the target medical record text; Based on the time dimension of each target text sequence, extract the target information text from each target text sequence; The information extraction model is trained based on the information extraction model training method according to any one of claims 1 to 6.

8. An information extraction model training device, characterized in that: include: An acquisition unit, used for acquiring a pre-trained teacher model, a student model to be trained, and a first sample medical record text; The output of the teacher model is used as a soft label, and the soft label is used to guide the optimization direction of the student model during the training process; A training unit, configured to use the first sample medical record text as a training sample and take the output distribution of the student model approaching the output distribution of the teacher model as a training goal, train the student model to obtain an information extraction model, wherein the output distribution is used to characterize the time dimension of all text sequences in the first sample medical record text; The teacher model is used to extract sentence representations from the second sample medical record text, and determine the time dimension of each text sequence in the second sample medical record text based on the sentence representation. The teacher model has the training goal of minimizing the difference between the time dimension of each text sequence in the second sample medical record text and the time dimension label of each text sequence in the second sample medical record text.

9. An information extraction device, characterized in that: include: A text determination unit, used for determining a target medical record text; A dimension determination unit, used to determine the time dimension of each target text sequence in the target medical record text based on the information extraction model; An extraction unit, used for extracting target information text from each target text sequence based on the time dimension of each target text sequence; The information extraction model is trained based on the information extraction model training method according to any one of claims 1 to 6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the information extraction model training method according to any one of claims 1 to 6, or the information extraction method according to claim 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the information extraction model training method according to any one of claims 1 to 6, or the information extraction method according to claim 7.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the information extraction model training method according to any one of claims 1 to 6, or the information extraction method according to claim 7.

Images