Information extraction method and device, equipment, storage medium and program product

Abstract

The invention discloses an information extraction method and device, equipment, a storage medium and a program product, and relates to the technical field of information processing, and the method comprises the steps: selecting a target task instruction from a first-class task instruction set; each first-class task instruction indicates that structured information composed of at least one layer of information is extracted; different first-class task instructions indicate that the extracted structured information contains different information hierarchy numbers; inputting the target task instruction and the image report into a pre-trained information extraction model to obtain structured information which is extracted from the image report and is composed of one or more layers of information; each training sample in a training sample set for training the information extraction model at least comprises an image report sample and a first class of task instructions; the label of the training sample at least comprises the structured information which is extracted from the image report sample and is composed of at least one layer of information, and automatic extraction of the structured information of different layers is achieved.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to an information extraction method, device, equipment, storage medium and program product. Background Art

[0002] As an important component of clinical medical data, imaging reports are of great significance for clinical analysis, decision-making and diagnosis. In existing clinical databases, imaging reports are stored in unstructured form as raw text, and due to the different writing habits of doctors, the information in them is often fragmented and semi-normalized, which brings challenges to data mining and data analysis. In addition, in smart medical applications, the complex unstructured raw imaging report text also makes it difficult for artificial intelligence models to learn and understand, which can easily lead to unstable model training and difficulty in obtaining valuable information in the report.

[0003] In order to facilitate clinical decision-making and obtain high-quality model corpus data, it is of great significance to extract structured information from imaging reports through automated means. Summary of the invention

[0004] In view of the above problems, the present application provides an information extraction method, device, equipment, storage medium and program product to realize automated information extraction. The specific scheme is as follows:

[0005] The first aspect of the present application provides an information extraction method, the method comprising:

[0006] Selecting a target task instruction in the first task instruction set; each first task instruction in the first task instruction set instructs: extracting structured information composed of at least one level of information; different first task instructions instruct the extracted structured information to contain different levels of information;

[0007] Inputting the target task instruction and the image report into a pre-trained information extraction model to obtain structured information consisting of one or more levels of information extracted by the information extraction model from the image report based on the target task instruction;

[0008] The information extraction model is trained by a training sample set, each training sample in the training sample set includes at least an image report sample and a first-category task instruction in the first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information extracted from the image report sample and corresponding to the first-category task instruction in the training sample, which is composed of at least one level of information.

[0009] In a possible implementation, selecting a target task instruction from a task instruction set includes:

[0010] displaying each first-category task instruction in the first-category task instruction set; obtaining a selection instruction for any first-category task instruction; and determining any first-category task instruction as the target task instruction;

[0011] Alternatively, an input information extraction requirement is received, and a first-category task instruction matching the information extraction requirement is searched in the first-category task instruction set as the target task instruction.

[0012] In a possible implementation, each training sample further includes at least one type of task instruction from among the second type of task instruction, the third type of task instruction and the fourth type of task instruction; different types of task instructions indicate different tasks;

[0013] The second type of task instruction indicates: determining whether the image report sample contains an entity in a target entity list; the target entity list is sampled from the structured information containing the most levels of annotations on the image report sample; correspondingly, the label of the training sample also includes second annotation information, and the second annotation information indicates whether the image report sample contains an entity in the target entity list;

[0014] The third type of task instructions indicate: explaining the medical definition of one or more entities appearing in the image report sample; the one or more entities are sampled from the structured information with the most levels annotated on the image report sample; accordingly, the label of the training sample also includes third annotation information, and the third annotation information is the medical definition of the one or more entities;

[0015] The fourth type of task instructions indicate: give the medically associated entity of at least one entity in the imaging report sample; the at least one entity is sampled from the structured information containing the most levels that is annotated for the imaging report sample; accordingly, the label of the training sample also includes fourth annotation information, and the fourth annotation information is the pre-annotated medically associated entity of the at least one entity.

[0016] In a possible implementation, the information extraction model is trained in the following manner:

[0017] Inputting the image report sample, the first type of task instruction and the at least one type of task instruction into the information extraction model, obtaining a first task result corresponding to the first type of task instruction and at least one task result corresponding to the at least one type of task instruction output by the information extraction model;

[0018] The parameters of the information extraction model are updated based on the first task result and the loss of the at least one task result relative to the label of the training sample.

[0019] In one possible implementation, at each iteration of training:

[0020] The target entity list is randomly sampled from the structured information with the most levels annotated on the image report sample;

[0021] The one or more entities are randomly sampled from the structured information with the most levels annotated on the image report sample;

[0022] The at least one entity is randomly sampled from the structured information with the most levels annotated on the image report sample.

[0023] In one possible implementation, at each iteration of training:

[0024] The first type of task instructions are randomly selected from the first type of task instruction set.

[0025] A second aspect of the present application provides an information extraction device, the device comprising:

[0026] The selection module is used to select a target task instruction in the first task instruction set; each first task instruction in the first task instruction set indicates: extracting structured information composed of at least one level of information; different first task instructions indicate that the extracted structured information contains different information levels;

[0027] An extraction module, in which a user inputs the target task instruction and the image report into a pre-trained information extraction model, and obtains structured information consisting of one or more levels of information extracted by the information extraction model from the image report based on the target task instruction;

[0028] The information extraction model is trained by a training sample set, each training sample in the training sample set includes at least an image report sample and a first-category task instruction in the first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information extracted from the image report sample and corresponding to the first-category task instruction in the training sample, which is composed of at least one level of information.

[0029] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the information extraction method of the first aspect or any implementation of the first aspect.

[0030] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0031] The memory is used to store computer programs;

[0032] The processor is used to execute the computer program so that the electronic device can implement the information extraction method of the first aspect or any implementation manner of the first aspect.

[0033] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the information extraction method of the first aspect or any implementation of the first aspect.

[0034] By means of the above-mentioned technical scheme, the information extraction method, apparatus, device, storage medium and program product provided by the present application select a target task instruction in a first-category task instruction set; each first-category task instruction in the first-category task instruction set indicates: extracting structured information consisting of at least one level of information; different first-category task instruction instructions indicate that the extracted structured information contains different numbers of information layers; the target task instruction and the image report are input into a pre-trained information extraction model to obtain structured information consisting of one or more levels of information extracted from the image report by the information extraction model based on the target task instruction; the information extraction model is trained by a training sample set, and each training sample in the training sample set includes at least an image report sample and a first-category task instruction in the first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information consisting of at least one level of information extracted from the image report sample and corresponding to the first-category task instruction in the training sample. That is to say, the present application presets a first-category task instruction set, and each first-category task instruction in the first-category task instruction set indicates: extracting structured information composed of at least one level of information; different first-category task instruction instructions indicate that the extracted structured information contains different numbers of information layers; an information extraction model is obtained based on the first-category task instruction set and image report sample training, so that the trained information extraction model can extract structured information composed of different levels of information from the image report based on different first-category task instructions, thereby realizing the automatic extraction of structured information at different levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0036] Figure 1 A flowchart of an implementation of the information extraction method provided in this application;

[0037] Figure 2 An architecture diagram for training a large language model based on multi-task instruction drive provided in this application;

[0038] Figure 3 The system architecture diagram for information extraction based on a trained large language model provided in this application;

[0039] Figure 4 A schematic diagram of the structure of the information extraction device provided by this application;

[0040] Figure 5 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0041] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0042] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0043] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0044] In order to automatically extract structured information from imaging reports, one possible implementation method is to decompose the structured information extraction task process into multiple sub-processes such as named entity recognition, relationship extraction, and terminology normalization, and finally merge the results of each sub-process to obtain the final structured information. However, the various information entities in medical imaging reports have the characteristics of multiple information levels. Actual clinical applications often have different requirements for the level of information extraction, and the above information extraction method cannot meet the information extraction requirements at different levels. In addition, the above information extraction method requires training different natural language models to adapt to each sub-process. The structured information extracted by this implementation method has problems such as low result accuracy and low reasoning efficiency, and cannot be well adapted to clinical needs.

[0045] In order to realize the automatic extraction of structured information at multiple levels, the following scheme is proposed.

[0046] like Figure 1 As shown, a flowchart of an implementation of the information extraction method provided in an embodiment of the present application may include:

[0047] Step S101: Select a target task instruction in the first-category task instruction set; each first-category task instruction in the first-category task instruction set indicates: extract structured information consisting of at least one level of information; different first-category task instructions indicate that the extracted structured information contains different numbers of information levels.

[0048] In this application, the task for extracting structured information is recorded as a first-class task (also called a main task or a main task), and the instruction for indicating the task of extracting structured information is recorded as a first-class task instruction (also called a main task instruction or a main task instruction). In order to realize the extraction of information at multiple levels, this application sets up multiple first-class tasks, and different first-class tasks are used to extract structured information composed of information at different levels. Different first-class tasks are indicated by different first-class task instructions.

[0049] Taking medical imaging reports as an example, the structured information extracted from medical imaging reports, which consists of information at different levels, can include the following:

[0050] It only contains structured information at the "normal / abnormal signs" level.

[0051] Contains structured information at two levels: "normal / abnormal signs" and "anatomical location".

[0052] Contains three levels of structured information: "normal / abnormal signs", "anatomical location" and "supplementary information".

[0053] Based on the above three types of structured information, the first-category task instruction set may include three first-category task instructions, which are respectively recorded as:

[0054] The first task instruction instructs to extract structured information including the level of "normal / abnormal signs". As an example, the first task instruction may be "You are now a professional radiologist. Please extract key information from the report in a manner that complies with the following schema definition. "Schema: normal signs / abnormal signs" or other similar forms, and the output form is a label list of normal / abnormal signs."

[0055] The second task instruction instructs to extract structured information including two levels of "normal / abnormal signs" and "anatomical location". As an example, the second task instruction may be "You are now a professional radiologist. Please extract key information from the report in a manner that complies with the following schema definition. "Schema: <normal / abnormal, anatomical location>" or other similar forms. The output form is a structured dictionary with normal / abnormal signs as the key and anatomical location as the value".

[0056] The third task instruction instructs to extract structured information including three levels of "normal / abnormal signs", "anatomical location" and "supplementary information". As an example, the third task instruction can be "You are now a professional radiologist. Please extract key information from the report in a manner that conforms to the following schema definition." Schema: <abnormality, anatomical location, supplementary information 1, supplementary information 2, ...>" or other similar forms, and the output form is a two-level nested dictionary structure, such as "{abnormality 1: {anatomical location 1: [supplementary information 1, supplementary information 2, ...]}, abnormality 2: {anatomical location 2: [...]}, ...}". Among them, the supplementary information may include but is not limited to: the uncertainty, severity, time progression, relative significance, quantitative description and other information of the abnormality occurring at a certain location. The supplementary information to be extracted may be a combination of one or more.

[0057] Step S102: Input the target task instruction and the image report into a pre-trained information extraction model to obtain structured information consisting of one or more levels of information extracted from the image report by the information extraction model based on the target task instruction.

[0058] The information extraction model is trained by a training sample set, each training sample in the training sample set includes at least an image report sample and a first-category task instruction in a first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information extracted from the image report sample and corresponding to the first-category task instruction in the training sample, which is composed of at least one level of information.

[0059] The information extraction model may be a large language model (LLM), and specifically may be a generative large language model.

[0060] An imaging report is a text file that describes what was seen in the imaging.

[0061] The present application inputs the target task instruction as a prompt together with the image report into the information extraction model, and obtains structured information consisting of one or more levels of information extracted by the information extraction model from the image report based on the target task instruction.

[0062] For example, if the target task instruction is the aforementioned first task instruction, the information extraction model will only extract normal sign labels or abnormal sign labels from the imaging report, but will not extract anatomical location labels or auxiliary information description labels.

[0063] If the target task instruction is the aforementioned second task instruction, the information extraction model will extract normal sign labels or abnormal sign labels and their corresponding anatomical location labels from the imaging report, but will not extract the auxiliary information description labels.

[0064] If the target task instruction is the aforementioned third task instruction, the information extraction model will extract normal sign labels or abnormal sign labels from the imaging report, as well as the anatomical location labels and auxiliary information description labels corresponding to the abnormal sign labels.

[0065] Optionally, when the training samples only include image report samples and the first type of task instructions, the information extraction model can be trained in the following way:

[0066] The training samples (i.e., the image report samples and the first type of task instructions) are input into the information extraction model to obtain structured information extracted by the information extraction model from the image report samples in the training samples based on the first type of task instructions in the training samples.

[0067] The parameters of the information extraction model are updated based on the loss of the structured information extracted by the information extraction model relative to the labels of the training samples. Optionally, a standard autoregressive generation method based on the next token prediction can be used to calculate the loss between the structured information extracted by the information extraction model and the labels of the training samples, and the parameters of the information extraction model are updated based on the loss. The specific method of updating the parameters of the information extraction model based on the loss can be found in existing solutions, which will not be repeated here.

[0068] The information extraction method provided by the embodiment of the present application presets a first-class task instruction set, and each first-class task instruction in the first-class task instruction set indicates: extracting structured information composed of at least one level of information; different first-class task instructions indicate that the extracted structured information contains different numbers of information layers; an information extraction model is obtained based on the first-class task instruction set and image report sample training, so that the trained information extraction model can extract structured information composed of different levels of information from the image report based on different first-class task instructions, thereby realizing the automatic extraction of structured information at different levels. Moreover, the structured information extracted from the image report based on the present application has high accuracy and high extraction efficiency, and can be well adapted to medical clinical needs.

[0069] In an optional embodiment, an implementation method of selecting the target task instruction in the first type of task instruction set may be:

[0070] Displays each first-category task instruction in the first-category task instruction set.

[0071] Optionally, an interactive interface may be outputted, which may provide a first-category task instruction drop-down option box, and the user may select a first-category task instruction matching the information extraction requirement in the drop-down option box according to the information extraction requirement.

[0072] Obtaining a selection instruction for any first-category task instruction. After the user selects any first-category task instruction, the generation of the selection instruction for the first-category task instruction is triggered.

[0073] Determine any of the first-category task instructions as the target task instruction, that is, determine the first-category task instruction selected by the user as the target task instruction.

[0074] In an optional embodiment, another implementation of selecting the target task instruction from the first type of task instruction set may be:

[0075] Receive input information extraction requirements.

[0076] The information extraction requirements here are the information extraction requirements input by users in natural language. Users can input information extraction requirements through keyboard or voice. Based on this, users can express in their own language which levels of structured information to be extracted are included, which is more user-friendly.

[0077] Search the first-category task instruction that matches the information extraction requirement in the first-category task instruction set as the target task instruction.

[0078] Optionally, the received information extraction requirement can be input into a pre-trained classification model to obtain the probability that the information extraction requirement output by the classification model belongs to each first-category task instruction in the first-category task instruction set, and then the first-category task instruction corresponding to the highest probability is searched in the first-category task instruction set as the target task instruction. The classification model can be trained as follows: an information extraction requirement sample can be input into the classification model to obtain the probability that the information extraction requirement sample output by the classification model belongs to each first-category task instruction in the first-category task instruction set, and the parameters of the classification model are updated with the first-category task instruction corresponding to the highest probability as the label of the information extraction requirement sample as the target. The label of the information extraction requirement sample is the pre-labeled first-category task instruction.

[0079] Optionally, the received information extraction requirements can be input into a pre-trained instruction generation model to obtain the first-category task instructions generated by the instruction generation model, and the first-category task instructions with the highest similarity to the first-category task instructions generated by the instruction generation model in the first-category task instruction set are determined as target task instructions. The instruction generation model can be trained in the following manner: the information extraction requirement samples can be input into the instruction generation model to obtain the first-category task instructions output by the instruction generation model, and the parameters of the instruction generation model are updated with the goal that the first-category task instructions output by the instruction generation model approach the label of the extraction requirement sample (i.e., the first-category task instructions generated by the instruction generation model based on the information extraction requirement sample after the parameter update are closer to the label of the information extraction requirement sample). The label of the information extraction requirement sample is the pre-labeled first-category task instruction.

[0080] Optionally, the semantic similarity between the information extraction requirement and each first-category task instruction in the first-category task instruction set may be calculated respectively, and the first-category task instruction with the highest semantic similarity to the information extraction requirement may be determined as the target task instruction.

[0081] In an optional embodiment, each training sample may further include an auxiliary task instruction, which may include at least one of the three types of task instructions: the second type of task instruction, the third type of task instruction, and the fourth type of task instruction; different types of task instructions indicate different tasks.

[0082] The second type of task instruction indicates: determine whether there are entities in the target entity list in the image report sample; the target entity list is sampled from the structured information with the most levels annotated for the image report sample. In the case where the second type of task instruction is included in the training sample, the label of the training sample also includes second annotation information, and the second annotation information represents whether there are entities in the target entity list in the image report sample. Since the target entity list is sampled from the structured information with the most levels annotated for the image report sample, the entities in the target entity list must exist in the image report sample, so the second annotation information represents the existence of entities in the target entity list in the image report sample.

[0083] That is, the second type of task instruction is used to inquire whether the entity in the target entity list appears in the image report sample of the information extraction model. As an example, the second type of task instruction can be:

[0084] "You are now a professional radiologist. Please determine whether the entities in the list [entity 1, entity 2, ...] appear in the imaging report." or other similar forms; the output form is a "yes / no" binary judgment with the same length as the input entity label list.

[0085] The "list [entity 1, entity 2, ...]" in the above second type of task instruction is the target entity list. The entities in the target entity list may include but are not limited to the following: normal signs, abnormal signs, anatomical locations, etc.

[0086] The third type of task instruction indicates: explain the medical definition of one or more entities appearing in the image report sample. The one or more entities are sampled from the structured information with the most levels of annotations for the image report sample. In the case where the third type of task instruction is included in the training sample, the label of the training sample also includes third annotation information, and the third annotation information is the medical definition of the one or more entities.

[0087] That is, the third type of task instruction is used to require the information extraction model to generate the medical definition corresponding to the above one or more entities. As an example, the third type of task instruction can be:

[0088] "You are now a professional radiologist. Please explain the medical concepts [entity 1, entity 2, ...] that appear in the following imaging report." or other similar forms; the output form is "Entity 1: Concept 1; Entity 2: Concept 2; ... ".

[0089] The one or more entities in the third category of task instructions mentioned above may include but are not limited to the following: normal signs, abnormal signs, anatomical positions, etc.

[0090] The third annotation information may be obtained from an open source standard medical knowledge base, wherein the open source standard medical knowledge base may be a Unified Medical Language System (UMLS) or the like.

[0091] The fourth type of task instruction indicates: provide a medically associated entity of at least one entity in the image report sample. The at least one entity is sampled from the structured information with the most levels annotated for the image report sample. In the case where the fourth type of task instruction is included in the training sample, the label of the training sample also includes fourth annotation information, and the fourth annotation information is the medically associated entity of the at least one entity annotated in advance.

[0092] That is, the fourth type of task instruction is used to require the information extraction model to generate other medically related conceptual entities corresponding to the above at least one entity. As an example, the fourth type of task instruction can be:

[0093] "You are now a professional radiologist. Please return other medical concepts that are associated with the medical concepts [entity 1, entity 2, ...] that appear in the following imaging reports." or other similar forms; the output form is "entity 1: [associated entity 1, associated entity 2, ...]; entity 2: [...]; ... ".

[0094] At least one entity in the fourth category of task instructions may include but is not limited to the following: normal signs, abnormal signs, anatomical positions, etc.

[0095] The fourth annotation information may be obtained from an open source standard medical knowledge base, wherein the open source standard medical knowledge base may be a Unified Medical Language System (UMLS) or the like.

[0096] As mentioned above, the structured information extracted from the imaging report may include only one level of information, two levels of information, three levels of information, or more levels of information.

[0097] In an embodiment of the present application, for each imaging report sample, only one structured information may be manually annotated, that is, the structured information containing the most levels, and structured information with fewer levels may be extracted from the structured information containing the most levels.

[0098] If the first type of task instruction in the training sample indicates that the number of levels of structured information extracted from the image report is not the structured information containing the largest number of levels, then corresponding information can be extracted from the structured information containing the largest number of levels as annotation information.

[0099] For example, for medical imaging reports, the structured information with the largest number of levels is the structured information containing normal / abnormal signs + anatomical positions + ancillary information annotations. For the normal / abnormal signs + anatomical positions extraction task, the normal / abnormal signs + anatomical positions data can be extracted from the complete normal / abnormal signs + anatomical positions + ancillary information annotations as annotations. For ancillary tasks that require the use of a knowledge base, the corresponding information (e.g., medical definitions, medical related entities, etc.) can be retrieved from the knowledge base as annotation data based on the sampled entity labels.

[0100] In an optional embodiment, when the training sample includes at least one type of task instruction among the subsidiary task instructions (including: the second type of task instructions, the third type of task instructions and the fourth type of task instructions), the information extraction model can be trained in the following manner:

[0101] Input training samples (including: image report samples, first-category task instructions and at least one type of task instructions among subsidiary task instructions) into the information extraction model, and obtain the first task result corresponding to the first-category task instructions and at least one task result corresponding to at least one type of task instructions among the subsidiary task instructions output by the information extraction model.

[0102] Parameters of the information extraction model are updated based on the first task result and the loss of at least one task result relative to the label of the training sample.

[0103] Optionally, when each training sample includes an image report sample, a first-category task instruction, and a second-category task instruction, the information extraction model can be trained in the following manner:

[0104] An image report sample, a first type of task instruction, and a second type of task instruction are input into the information extraction model to obtain a first task result and a second task result output by the information extraction model; the first task result is structured information extracted from the image report sample by the information extraction model based on the first type of task instruction, and the second task result is generated by the information extraction model based on the second type of task instruction and the image report, and characterizes whether there is an entity in the target entity list in the image report sample.

[0105] Based on a first loss of a first task result relative to the first labeled information and a second loss of a second task result relative to the second labeled information, parameters of the information extraction model are updated.

[0106] Specifically, the first loss and the second loss can be weightedly summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of reducing the total loss (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0107] Optionally, when each training sample includes an image report sample, a first-category task instruction, and a third-category task instruction, the information extraction model can be trained in the following manner:

[0108] An imaging report sample, a first type of task instruction, and a third type of task instruction are input into the information extraction model to obtain a first task result and a third task result output by the information extraction model; the first task result is structured information extracted from the imaging report sample by the information extraction model based on the first type of task instruction, and the third task result is a medical definition of at least one entity appearing in the imaging report sample generated by the information extraction model based on the third type of task instruction and the imaging report sample.

[0109] Based on a first loss of the first task result relative to the first labeled information and a third loss of the third task result relative to the third labeled information, parameters of the information extraction model are updated.

[0110] Specifically, the first loss and the third loss can be weighted and summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of making the total loss smaller and smaller (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0111] Optionally, when each training sample includes an image report sample, a first-category task instruction, and a fourth-category task instruction, the information extraction model can be trained in the following manner:

[0112] The imaging report sample, the first type of task instructions and the fourth type of task instructions are input into the information extraction model to obtain the first task result and the fourth task result output by the information extraction model; the first task result is the structured information extracted from the imaging report sample by the information extraction model based on the first type of task instructions, and the fourth task result is the medical associated entity of at least one entity in the imaging report sample generated by the information extraction model based on the fourth type of task instructions and the imaging report sample.

[0113] Based on a first loss of the first task result relative to the first labeled information and a fourth loss of the fourth task result relative to the fourth labeled information, parameters of the information extraction model are updated.

[0114] Specifically, the first loss and the fourth loss can be weightedly summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of making the total loss smaller and smaller (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0115] Optionally, when each training sample includes an image report sample, a first-category task instruction, a second-category task instruction, and a third-category task instruction, the information extraction model can be trained in the following manner:

[0116] Inputting an imaging report sample, a first type of task instruction, a second type of task instruction, and a third type of task instruction into an information extraction model, obtaining a first task result, a second task result, and a third task result output by the information extraction model; the first task result is structured information extracted from the imaging report sample by the information extraction model based on the first type of task instruction, the second task result is generated by the information extraction model based on the second type of task instruction and the imaging report sample, and characterizes whether there is an entity in the target entity list in the imaging report sample, and the third task result is a medical definition of at least one entity appearing in the imaging report sample generated by the information extraction model based on the third type of task instruction and the imaging report sample;

[0117] Parameters of the information extraction model are updated based on a first loss of the first task result relative to the first labeled information, a second loss of the second task result relative to the second labeled information, and a third loss of the third task result relative to the third labeled information.

[0118] Specifically, the first loss, the second loss, and the third loss can be weighted and summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of making the total loss smaller and smaller (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0119] Optionally, when each training sample includes the image report sample, the first type of task instruction, the second type of task instruction, and the fourth type of task instruction, the information extraction model can be trained in the following manner:

[0120] Input an image report sample, a first type of task instruction, a second type of task instruction, and a fourth type of task instruction into an information extraction model to obtain a first task result, a second task result, and a fourth task result output by the information extraction model; the first task result is structured information extracted from the image report sample by the information extraction model based on the first type of task instruction, the second task result is generated by the information extraction model based on the second type of task instruction and the image report sample, and represents whether there is an entity in the target entity list in the image report sample, and the fourth task result is a medically associated entity of at least one entity in the image report sample generated by the information extraction model based on the fourth type of task instruction and the image report sample;

[0121] Parameters of the information extraction model are updated based on a first loss of the first task result relative to the first labeled information, a second loss of the second task result relative to the second labeled information, and a fourth loss of the fourth task result relative to the fourth labeled information.

[0122] Specifically, the first loss, the second loss and the fourth loss can be weighted and summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of making the total loss smaller and smaller (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0123] Optionally, when each training sample includes an image report sample, a first-category task instruction, a third-category task instruction, and a fourth-category task instruction, the information extraction model can be trained in the following manner:

[0124] An imaging report sample, a first type of task instruction, a third type of task instruction, and a fourth type of task instruction are input into the information extraction model to obtain a first task result, a third task result, and a fourth task result output by the information extraction model; the first task result is structured information extracted from the imaging report sample by the information extraction model based on the first type of task instruction, the third task result is a medical definition of at least one entity appearing in the imaging report sample generated by the information extraction model based on the third type of task instruction and the imaging report sample, and the fourth task result is a medically associated entity of at least one entity in the imaging report sample generated by the information extraction model based on the fourth type of task instruction and the imaging report sample.

[0125] Parameters of the information extraction model are updated based on a first loss of the first task result relative to the first labeled information, a second loss of the third task result relative to the third labeled information, and a fourth loss of the fourth task result relative to the fourth labeled information.

[0126] Specifically, the first loss, the third loss, and the fourth loss can be weightedly summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of making the total loss smaller and smaller (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0127] Optionally, when each training sample includes an image report sample, a first-category task instruction, a second-category task instruction, a third-category task instruction, and a fourth-category task instruction, the information extraction model can be trained in the following manner:

[0128] An imaging report sample, a first type of task instruction, a second type of task instruction, a third type of task instruction and a fourth type of task instruction are input into the information extraction model to obtain a first task result, a second task result, a third task result and a fourth task result output by the information extraction model; the first task result is structured information extracted from the imaging report sample by the information extraction model based on the first type of task instruction, the second task result is generated by the information extraction model based on the second type of task instruction and the imaging report sample, and characterizes whether there is an entity in the target entity list in the imaging report sample, the third task result is a medical definition of at least one entity appearing in the imaging report sample generated by the information extraction model based on the third type of task instruction and the imaging report sample, and the fourth task result is a medically associated entity of at least one entity in the imaging report sample generated by the information extraction model.

[0129] The parameters of the information extraction model are updated based on a first loss of the first task result relative to the first labeled information, a second loss of the second task result relative to the second labeled information, a second loss of the third task result relative to the third labeled information, and a fourth loss of the fourth task result relative to the fourth labeled information.

[0130] Specifically, the first loss, the second loss, the third loss and the fourth loss can be weighted and summed to obtain the total loss. The parameters of the information extraction model can be updated with the goal of reducing the total loss (that is, each time the parameters of the information extraction model are updated, the calculated total loss is smaller than the total loss calculated before the update).

[0131] By introducing tasks related to medical knowledge (i.e., tasks indicated by the second category of task instructions, and / or tasks indicated by the third category of task instructions, and / or tasks indicated by the fourth category of task instructions) during the training process of the information extraction model, the information extraction model can fully consider medical knowledge in the learning of information extraction tasks, which can effectively avoid catastrophic forgetting of medical knowledge and reduce medical factual errors or hallucinations, thereby more efficiently, accurately and standardizedly extracting different levels of structured information from imaging reports.

[0132] In an optional embodiment, in each iteration of training the information extraction model:

[0133] The target entity list specified by the second type of task instruction is randomly sampled from the structured information with the most levels of annotations on the image report samples, which is equivalent to the second type of task instruction being randomly generated.

[0134] The one or more entities specified by the third type of task instructions are randomly sampled from the structured information with the most levels of annotations on the image report samples, which is equivalent to the third type of task instructions being randomly generated.

[0135] At least one entity specified by the fourth type of task instruction is randomly sampled from the structured information with the most levels annotated on the image report sample, which is equivalent to the fourth type of task instruction being randomly generated.

[0136] In an optional embodiment, in each iteration of training the information extraction model, the first type of task instructions are randomly selected from the first type of task instruction set.

[0137] By randomly generating subsidiary task instructions (i.e., second-category task instructions, third-category task instructions, and fourth-category task instructions) or randomly selecting first-category task instructions, data utilization is increased, making the generalization ability of the trained information extraction model stronger.

[0138] like Figure 2 As shown, it is an architecture diagram of training a large language model based on multi-task instruction drive provided in an embodiment of the present application.

[0139] In this architecture diagram, the "information extraction main task instruction set" is the first type of task instruction set of this application, and the "information extraction auxiliary task instruction set" contains multiple auxiliary task instructions (such as the second type of task instructions, the third type of task instructions and the fourth type of task instructions of this application). "Complete and precise annotation information" is the structured information with the most levels of annotations for the image report.

[0140] The task instruction sampler is used to randomly sample a first-category task instruction from the "information extraction main task instruction set" and to extract at least one subsidiary task instruction from the "information extraction subsidiary task instruction set".

[0141] The multi-task data processor is used to extract corresponding information from the complete precise annotation information as the main task label (i.e., the first annotation information) according to the sampled first-category task instructions, and randomly sample entities from the complete precise annotation information according to the sampled subsidiary task instructions. If the auxiliary task instructions include the second-category task instructions, it is also necessary to determine whether the sampled entity exists in the imaging report as a subsidiary task label (i.e., the second annotation information). If the auxiliary task instructions include the third-category task instructions, it is also necessary to extract the medical definition of the sampled entity from the medical knowledge base as a subsidiary task label (i.e., the third annotation information). If the auxiliary task instructions include the fourth-category task instructions, it is also necessary to extract the medically associated entity of the sampled entity from the medical knowledge base as a subsidiary task label (the fourth annotation information).

[0142] The multi-task data processor inputs each task instruction into the large language model, and the image report text is also input into the large language model, and the large language model outputs the task results corresponding to each task instruction.

[0143] The autoregressive generation loss is calculated based on the results of each task and the corresponding labels, and the parameters of the large language model are updated based on the loss.

[0144] After the large language model is trained, it can be used to extract structured information from imaging reports based on the information extraction main task instruction set.

[0145] like Figure 3 As shown, it is a system architecture diagram for information extraction based on a trained large language model provided in an embodiment of the present application.

[0146] Among them, the task instruction selection interface can provide a drop-down box for the user to select the information extraction main task instruction (i.e., the first type of task instruction) according to the information extraction requirements (i.e., the information hierarchy contained in the structured information); or, the task instruction selection interface can provide a text input interface so that the user can input the information extraction task requirements in a natural language in the text input interface, and then the task instruction selection interface can select the information extraction main task instruction that matches the user's information extraction task requirements from the information extraction main task instruction set according to the information extraction requirements input by the user in a natural language.

[0147] The task instruction selection interface inputs the selected task instruction (i.e., the information extraction subject task instruction) into the large language model. At the same time, the image report is also input into the large language model. The large language model extracts and outputs structured information in the image report according to the input task instruction.

[0148] Corresponding to the method embodiment, the present application also provides an information extraction device, such as Figure 4 As shown, a schematic diagram of a structure of an information extraction device provided in an embodiment of the present application may include:

[0149] Selection module 401 and extraction module 402;

[0150] The selection module 401 is used to select a target task instruction in the first task instruction set; each first task instruction in the first task instruction set indicates: extracting structured information composed of at least one level of information; different first task instructions indicate that the structured information extracted contains different levels of information;

[0151] In the extraction module 402, the user inputs the target task instruction and the image report into a pre-trained information extraction model, and obtains structured information consisting of one or more levels of information extracted by the information extraction model from the image report based on the target task instruction;

[0152] The information extraction model is trained by a training sample set, each training sample in the training sample set includes at least an image report sample and a first-category task instruction in the first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information extracted from the image report sample and corresponding to the first-category task instruction in the training sample, which is composed of at least one level of information.

[0153] The information extraction device provided by the embodiment of the present application is pre-set with a first-class task instruction set, each first-class task instruction in the first-class task instruction set indicates: extracting structured information composed of at least one level of information; different first-class task instructions indicate that the extracted structured information contains different numbers of information layers; an information extraction model is obtained based on the first-class task instruction set and image report sample training, so that the trained information extraction model can extract structured information composed of different levels of information from the image report based on different first-class task instructions, thereby realizing the automatic extraction of structured information at different levels. Moreover, the structured information extracted from the image report based on the present application has high accuracy and high extraction efficiency, and can be well adapted to medical clinical needs.

[0154] In an optional embodiment, when selecting a target task instruction from a task instruction set, the selection module 401 is used to:

[0155] displaying each first-category task instruction in the first-category task instruction set; obtaining a selection instruction for any first-category task instruction; and determining any first-category task instruction as the target task instruction;

[0156] Alternatively, an input information extraction requirement is received, and a first-category task instruction matching the information extraction requirement is searched in the first-category task instruction set as the target task instruction.

[0157] In an optional embodiment, each training sample further includes at least one type of task instruction from among the second type of task instruction, the third type of task instruction and the fourth type of task instruction; different types of task instructions indicate different tasks;

[0158] The second type of task instruction indicates: determining whether the image report sample contains an entity in a target entity list; the target entity list is sampled from the structured information containing the most levels of annotations on the image report sample; correspondingly, the label of the training sample also includes second annotation information, and the second annotation information indicates whether the image report sample contains an entity in the target entity list;

[0159] The third type of task instructions indicate: explaining the medical definition of one or more entities appearing in the image report sample; the one or more entities are sampled from the structured information with the most levels annotated on the image report sample; accordingly, the label of the training sample also includes third annotation information, and the third annotation information is the medical definition of the one or more entities;

[0160] The fourth type of task instructions indicate: give the medically associated entity of at least one entity in the imaging report sample; the at least one entity is sampled from the structured information containing the most levels that is annotated for the imaging report sample; accordingly, the label of the training sample also includes fourth annotation information, and the fourth annotation information is the pre-annotated medically associated entity of the at least one entity.

[0161] In an optional embodiment, the information extraction model is trained in the following manner:

[0162] Inputting the image report sample, the first type of task instruction and the at least one type of task instruction into the information extraction model, obtaining a first task result corresponding to the first type of task instruction and at least one task result corresponding to the at least one type of task instruction output by the information extraction model;

[0163] The parameters of the information extraction model are updated based on the first task result and the loss of the at least one task result relative to the label of the training sample.

[0164] In an optional embodiment, in each iteration of training:

[0165] The target entity list is randomly sampled from the structured information with the most levels annotated on the image report sample;

[0166] The one or more entities are randomly sampled from the structured information with the most levels annotated on the image report sample;

[0167] The at least one entity is randomly sampled from the structured information with the most levels annotated on the image report sample.

[0168] In an optional embodiment, in each iteration of training:

[0169] The first type of task instructions are randomly selected from the first type of task instruction set.

[0170] The present application also provides an electronic device in an embodiment. Figure 5As shown, it shows a structural schematic diagram of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application can be a terminal device, such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc. Of course, the terminal device can independently complete the above information extraction method, or it can interact with the server device to complete the above information extraction method. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0171] like Figure 5 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0172] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a memory card, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0173] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any information extraction method provided in the embodiment of the present application.

[0174] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any information extraction method provided in the embodiment of the present application.

[0175] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0176] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0177] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. Professionals and technicians may use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.

[0178] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0179] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0180] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An information extraction method, characterized in that: The method comprises: Selecting a target task instruction in the first task instruction set; each first task instruction in the first task instruction set instructs: extracting structured information composed of at least one level of information; different first task instructions instruct the extracted structured information to contain different levels of information; Inputting the target task instruction and the image report into a pre-trained information extraction model to obtain structured information consisting of one or more levels of information extracted by the information extraction model from the image report based on the target task instruction; The information extraction model is trained by a training sample set, each training sample in the training sample set includes at least an image report sample and a first-category task instruction in the first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information extracted from the image report sample and corresponding to the first-category task instruction in the training sample, which is composed of at least one level of information.

2. The method according to claim 1, characterized in that The selecting of the target task instruction in the first type of task instruction set comprises: displaying each first-category task instruction in the first-category task instruction set; obtaining a selection instruction for any first-category task instruction; and determining any first-category task instruction as the target task instruction; Alternatively, an input information extraction requirement is received, and a first-category task instruction matching the information extraction requirement is searched in the first-category task instruction set as the target task instruction.

3. The method according to claim 1, characterized in that Each training sample further includes at least one type of task instruction from among the second type of task instruction, the third type of task instruction and the fourth type of task instruction; different types of task instructions indicate different tasks; The second type of task instruction indicates: determining whether the image report sample contains an entity in a target entity list; the target entity list is sampled from the structured information containing the most levels of annotations on the image report sample; correspondingly, the label of the training sample also includes second annotation information, and the second annotation information indicates whether the image report sample contains an entity in the target entity list; The third type of task instructions indicate: explaining the medical definition of one or more entities appearing in the image report sample; the one or more entities are sampled from the structured information with the most levels annotated on the image report sample; accordingly, the label of the training sample also includes third annotation information, and the third annotation information is the medical definition of the one or more entities; The fourth type of task instructions indicate: give the medically associated entity of at least one entity in the imaging report sample; the at least one entity is sampled from the structured information containing the most levels that is annotated for the imaging report sample; accordingly, the label of the training sample also includes fourth annotation information, and the fourth annotation information is the pre-annotated medically associated entity of the at least one entity.

4. The method according to claim 3, characterized in that The information extraction model is trained in the following way: Inputting the image report sample, the first type of task instruction and the at least one type of task instruction into the information extraction model, obtaining a first task result corresponding to the first type of task instruction and at least one task result corresponding to the at least one type of task instruction output by the information extraction model; The parameters of the information extraction model are updated based on the first task result and the loss of the at least one task result relative to the label of the training sample.

5. The method according to claim 3, characterized in that: In each training iteration: The target entity list is randomly sampled from the structured information with the most levels annotated on the image report sample; The one or more entities are randomly sampled from the structured information with the most levels annotated on the image report sample; The at least one entity is randomly sampled from the structured information with the most levels annotated on the image report sample.

6. The method according to claim 3, characterized in that: In each training iteration: The first type of task instructions are randomly selected from the first type of task instruction set.

7. An information extraction device, characterized in that: The device comprises: The selection module is used to select a target task instruction in the first task instruction set; each first task instruction in the first task instruction set indicates: extracting structured information composed of at least one level of information; different first task instructions indicate that the extracted structured information contains different information levels; An extraction module, in which a user inputs the target task instruction and the image report into a pre-trained information extraction model, and obtains structured information consisting of one or more levels of information extracted by the information extraction model from the image report based on the target task instruction; The information extraction model is trained by a training sample set, each training sample in the training sample set includes at least an image report sample and a first-category task instruction in the first-category task instruction set; the label of the training sample includes at least first annotation information, and the first annotation information is structured information extracted from the image report sample and corresponding to the first-category task instruction in the training sample, which is composed of at least one level of information.

8. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the information extraction method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the information extraction method according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the information extraction method as described in any one of claims 1 to 6.

Images