Text structuring method, medical big language model training method and related device
By fine-tuning the combination of the large medical language model and the data extraction model, the structured processing problem of electronic health records is solved, and efficient and accurate utilization of medical data is achieved. It is suitable for a variety of structured tasks, reducing costs and manual labeling requirements.
Patent Information
- Application Number
- CN202411047714.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-27
AI Technical Summary
How to convert electronic health records into computer-understandable formats to improve the efficiency of medical information utilization, it is difficult for the prior art to effectively process medical data in natural language forms.
A fine-tuned medical large language model is used to perform structured tasks on the target text, and sample structured data is extracted in combination with the data extraction model. It adapts to different types of text processing needs through structured tasks of multiple value domain types, reducing manual annotation and resource consumption.
It improves the accuracy and efficiency of text structure of medical large language models, can handle multiple types of structured tasks, save costs, and does not require manual annotation, and is suitable for structured processing of electronic case texts.
Smart Images
Figure CN119207683A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of natural language processing, and in particular to a text structuring method, a training method for a large medical language model, and related devices. Background Art
[0002] With the rapid development of medical informatization, a large number of electronic health records appear in the form of natural language, which poses a challenge to computer processing and data analysis. Therefore, how to convert electronic health records into a computer-understandable format so as to make more efficient use of this valuable medical information has become one of the urgent problems to be solved. Summary of the invention
[0003] This application at least provides a text structuring method, a training method for a large medical language model, and related devices.
[0004] The present application provides a text structuring method, which includes: obtaining a target text; performing a structuring task on the target text using a fine-tuned medical big language model to obtain target structured data of the target text; wherein the fine-tuned medical big language model is obtained by fine-tuning the medical big language model using sample text and sample structured data of the sample text; and the sample structured data is obtained by extracting the sample text using a data extraction model.
[0005] Therefore, by fine-tuning the medical big language model, the medical big language model's ability to structure texts can be improved. Since the medical big language model itself has a good ability to understand texts, fine-tuning it can retain its ability to understand texts. When structuring the target text, it can better understand the target text and obtain target structured data with higher accuracy. Furthermore, the sample structured data can be extracted using a data extraction model without manual labeling, which can improve the efficiency of structuring tasks.
[0006] Among them, the target structured data includes the fields in the target text and the values corresponding to the fields. The fine-tuned medical large language model can perform various types of structured tasks. Each type of structured task corresponds to the output of a value domain type of structured data. The value domain type of the digitized data is determined by the value domain corresponding to the field in the structured data.
[0007] Therefore, the fine-tuned medical large language model is able to perform multiple types of structured tasks, which can greatly save costs compared to training a model for each type of structured task.
[0008] The value domain type includes at least one of an extraction type, a nested extraction type, a single-select type, and a multiple-select type.
[0009] Therefore, by setting multiple value domain types, it is possible to adapt to each type of structured task, so that the fine-tuned medical large language model can perform multiple types of structured tasks.
[0010] Before using the fine-tuned medical big language model to perform structured processing on the target text to obtain the target structured data of the target text, it also includes: obtaining a fine-tuning data set, wherein the fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from the corresponding sample text; using the medical big language model to perform structured processing on the sample text corresponding to each sample structured data, respectively, to obtain the predicted structured data corresponding to each sample structured data; based on each sample structured data and the corresponding predicted structured data, the medical big language model is fine-tuned.
[0011] Therefore, by using the structured data of each sample and the corresponding predicted structured data to fine-tune the medical big language model, the medical big language model's ability to structure text can be improved.
[0012] Among them, obtaining the fine-tuning data set includes: inputting the sample text and the prompt text corresponding to the sample text into the data extraction model, the prompt text corresponding to the sample text is used to instruct the data extraction model to perform the data extraction task on the sample text; using the data extraction model to perform the data extraction task on the sample text based on the prompt text to obtain the sample structured data in the fine-tuning data set.
[0013] Therefore, by extracting sample text through a data extraction model, it is possible to obtain sample structured data of the sample text with higher accuracy while saving labor costs.
[0014] Among them, the fine-tuning data set contains sample structured data of at least one value domain type, and the sample structured data includes sample fields in the sample text and values corresponding to the sample fields; before the sample text and the prompt text corresponding to the sample text are input into the data extraction model, it also includes: determining the sample fields in the sample text and the value domain type corresponding to the sample fields; based on the sample fields and the value domain type of the sample fields, constructing the prompt text corresponding to the sample text.
[0015] Therefore, different prompt texts can be constructed according to the sample fields and value domain types, and then sample structured data suitable for different structured tasks can be obtained according to different prompt texts.
[0016] Among them, based on the sample field and the value domain type of the sample field, construct the prompt text corresponding to the sample text, including: selecting a prompt text template that matches the value domain type of the sample field; wherein the prompt text template includes field items; filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text.
[0017] Therefore, the sample field is filled into the field item of the prompt text template to generate the prompt text corresponding to the sample text.
[0018] Among them, the value domain type includes an extraction type, and the prompt text template of the extraction type is used to indicate the extraction of the value corresponding to the sample field filled in the field item in the sample text; and / or, the value domain type includes a radio type, and the prompt text template of the radio type also includes a radio value domain item, and the prompt text template of the radio type is used to indicate the selection of the radio value corresponding to the sample field in the sample text from multiple radio values of the radio value domain item; filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text, including: filling the sample field into the field item of the prompt text template, and filling the multiple radio values corresponding to the sample field into the radio value domain item of the prompt text template to generate the prompt text corresponding to the sample text; and / or, the value domain type includes a multiple-select type, and the prompt text template of the multiple-select type also includes a multiple-select value domain item, and the prompt text template of the multiple-select type is used to indicate the selection of the sample from the multiple multiple-select values of the multiple-select value domain item. At least two multiple-choice values corresponding to the sample field in the text; filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text, including: filling the sample field into the field item of the prompt text template, and filling the multiple multiple-choice values corresponding to the sample field into the multiple-choice value domain item of the prompt text template to generate the prompt text corresponding to the sample text; and / or, the value domain type includes a nested extraction type, the prompt text template of the nested extraction type also includes an entity item, and the prompt text template of the nested extraction type is used to indicate to find the value of a specified attribute from the sample text, and the specified attribute is the attribute represented by the sample field in the field item of the entity answer in the entity item; filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text, including: filling the entity into the entity item, and filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text.
[0019] Therefore, according to the difference in the value domain type, a prompt text matching the value domain type is generated, so that the data extraction model can extract different sample structured data.
[0020] Among them, the extraction type includes a nested extraction type, and the nested extraction type also corresponds to an entity text extraction template. The entity text extraction template includes an entity field item. The entity text extraction template is used to indicate the extraction of several values corresponding to the entity field filled in the entity field item in the sample text; before filling the entity answer into the entity item and filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text, it also includes: filling the entity field of the sample text into the entity field item of the entity text extraction template to generate the entity prompt text corresponding to the sample text; using the data extraction model to perform an entity extraction task on the sample text based on the entity prompt text to obtain several values corresponding to the entity field; using at least one of the several values corresponding to the entity field as the entity answer in the entity item.
[0021] Therefore, through the entity text extraction template, the entity prompt text can be obtained, so that the data extraction model can use the entity prompt text and sample text to obtain the entity answer in the entity item.
[0022] The prompt text also includes a special description item, and the text content filled in the special description item is used for at least one of the following: providing the data extraction model with an understanding of the sample field, and indicating the output format of the data extraction model.
[0023] Therefore, by setting special description items, the data extraction model can better understand the sample fields, improve the accuracy of the data output by the data extraction model, and further improve the accuracy of the sample structured data.
[0024] Among them, the target text is the electronic medical record text.
[0025] Therefore, it is possible to carry out structured processing on the electronic medical case text and obtain target structured data of the electronic medical case text, which can enable users to better realize the research of the electronic medical case data.
[0026] The present application provides a training method for a medical big language model, the method comprising: obtaining a fine-tuning data set, wherein the fine-tuning data set comprises at least one sample structured data, each sample structured data is extracted from a corresponding sample text using a data extraction model; using the medical big language model to perform structured processing on the sample text corresponding to each sample structured data, respectively, to obtain predicted structured data corresponding to each sample structured data; and fine-tuning the medical big language model based on each sample structured data and the corresponding predicted structured data.
[0027] Therefore, fine-tuning the medical big language model using the sample structured data in the fine-tuning dataset and the predicted structured data of the sample structured data by the medical big language model can improve the medical big language model's ability to process text in a structured manner.
[0028] The present application provides a text structuring device, which includes a first acquisition module and a first processing module. The first acquisition module is used to acquire a target text, and the first processing module is used to perform a structuring task on the target text using a fine-tuned medical large language model to obtain target structured data of the target text; wherein the fine-tuned medical large language model is obtained by fine-tuning the medical large language model using sample text and sample structured data of the sample text; and the sample structured data is obtained by extracting the sample text using a data extraction model.
[0029] The present application provides a training device for a medical large language model, which includes a second acquisition module, a second processing module and a fine-tuning module. The second acquisition module is used to acquire a fine-tuning data set, wherein the fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from a corresponding sample text using a data extraction model; the second processing module is used to use the medical large language model to perform structured processing on the sample text corresponding to each sample structured data, respectively, to obtain predicted structured data corresponding to each sample structured data; the fine-tuning module is used to fine-tune the medical large language model based on each sample structured data and the corresponding predicted structured data.
[0030] The present application provides an electronic device, which includes a processor and a memory, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement any of the above-mentioned text structuring methods or medical large language model training methods.
[0031] The present application provides a computer-readable storage medium storing program instructions, which can be executed to implement any of the above-mentioned text structuring methods or medical large language model training methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0033] Figure 1 It is a flowchart of an embodiment of a text structuring method provided by the present application;
[0034] Figure 2 It is a flowchart of another embodiment of the text structuring method provided by the present application;
[0035] Figure 3 It is a flowchart of an implementation method of step S21 provided by this application;
[0036] Figure 4 It is a flowchart of another implementation method of the text structuring method provided by the present application;
[0037] Figure 5 It is a flowchart of an implementation method of a training method of a large medical language model provided by the present application;
[0038] Figure 6 It is a schematic diagram of a framework of an embodiment of a text structuring device provided by the present application;
[0039] Figure 7 It is a schematic diagram of the framework of an embodiment of a training device for a large medical language model provided by the present application;
[0040] Figure 8 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;
[0041] Fig. 9 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION
[0042] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0043] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.
[0044] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of the text structuring method provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 1 The process sequence shown is limited. Figure 1 As shown, the method comprises the following steps:
[0045] Step S11: Obtain target text.
[0046] Step S12: Use the fine-tuned medical large language model to perform a structuring task on the target text to obtain target structured data of the target text.
[0047] The method of this embodiment is used to obtain structured data of a text, wherein the text may be a target text, and the target text may be a medical text, an educational text, a food safety text, etc. Taking the case where the target text is a medical text as an example, the medical text may be an electronic medical case text, a medical book, a medical magazine, and other medical-related texts, etc. In a specific implementation scenario, the target text may be an electronic medical case text, and the electronic medical case text may be directly obtained from a medical system, uploaded by a user, or scanned from a paper medical case text.
[0048] Among them, the target structured data of the target text can be obtained by performing a structured task on the target text using a fine-tuned medical large language model, and the fine-tuned medical large language model can be obtained by fine-tuning the medical large language model using sample text and sample structured data of the sample text. Fine-tuning can be performed using any of the LoRA (Low-Rank Adaptation Of Large Language Model, a low-rank adaptive method for large language models), QloRA (Query-specific LORA), SFT (Supervised Fine-Tuning) and P-tuning (Prompt Tuning) fine-tuning techniques to fine-tune the medical large language model. Users can choose the fine-tuning technology based on task requirements and the amount of resources. In a specific embodiment, the QloRA fine-tuning technology is used to fine-tune the medical large language model to significantly reduce resource consumption.
[0049] The target structured data of the target text may be the key information of the target text. In one implementation scenario, the target structured data may include fields in the target text and values corresponding to the fields, wherein the fields in the target text may be any information in the target text, for example, entities, medical terms, titles, and other information in the target text. In a specific embodiment, the fields in the target text are, for example, at least one of diabetes, inner diameter of aortic root, number of drainage tubes, examination results, and the like. The value corresponding to the field may include a numerical value, or may include text or other characters. For example, when the field is the number of drainage tubes, the value corresponding to the field may be 2; for another example, when the field is the examination result, the value corresponding to the field may be sinus rhythm or low voltage.
[0050] The structured tasks in the present application may include multiple types. The fine-tuned medical large language model can perform multiple types of structured tasks. Each type of structured task corresponds to outputting structured data of a value domain type. The value domain type of the structured data is determined by the value domain corresponding to the field in the structured data. In an implementation scenario, the type of structured task may include at least one of an extraction structured task, a nested extraction structured task, a selection structured task, and a prompt structured task. Among them, the extraction type structured task is used to extract target structured data from the target text; the nested extraction type structured task is used to extract several entity answers and the values corresponding to the fields corresponding to the several entity answers in the target text from the target text; the selection type structured task is used to select the value corresponding to the field in the target text from multiple values of the value domain item. The selection type structured task can include single-choice structured tasks and multiple-choice structured tasks. The single-choice structured task is used to select a value from multiple values of the value domain item, and the multiple-choice structured task is used to select multiple values from multiple values of the value domain item; the prompt type structured task is used to indicate the output format of the medical big language model and help the medical big language model understand the field.
[0051] Corresponding to various types of structured tasks, the domain type includes at least one of an extraction type, a nested extraction type, a single-choice type, and a multiple-choice type. The domain type of the structured data can be determined by the domain corresponding to the field in the structured data. In an implementation scenario, the domain type of the structured data can be determined based on information such as whether the domain corresponding to the field has a fixed answer, multiple fixed answers, or an answer with an indefinite range.
[0052] By fine-tuning the medical big language model, it is possible to perform structured tasks on the target text. Since the medical big language model itself has a good ability to understand text, especially medical text, fine-tuning it can retain its ability to understand text. When performing structured processing on the target text, it can better understand the target text and obtain target structured data with higher accuracy. Furthermore, the sample structured data can be extracted using a data extraction model without manual labeling, which can improve the efficiency of structured tasks.
[0053] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the text structuring method provided by the present application, the method comprising the following steps:
[0054] Step S21: Obtain a fine-tuning dataset.
[0055] In one implementation scenario, the fine-tuning data set includes at least one sample structured data, each sample structured data is extracted from a corresponding sample text, the sample text may be an electronic medical record text, the sample structured data may be extracted from the corresponding sample text using a data extraction model, the data extraction model may be an existing large language model with excellent structural capabilities, such as ChatGPT; the data extraction model may also be a medical large language model before fine-tuning. The sample structured data may also be manually extracted from the corresponding sample text.
[0056] Step S22: Use the medical big language model to perform structured processing on the sample texts corresponding to each sample structured data to obtain predicted structured data corresponding to each sample structured data.
[0057] In one implementation scenario, a sample text is input into a medical big language model, and the medical big model outputs predicted structured data of the sample text. It can be understood that when the sample text is input into the medical big language model, corresponding prompt information can be given to the medical big language model to prompt the medical big language model to perform a structured task type corresponding to the prompt information. For example, the medical big language model can be prompted to perform an extraction-type structured task on the sample text.
[0058] Step S23: fine-tune the medical large language model based on each sample structured data and the corresponding predicted structured data.
[0059] In one implementation scenario, the difference between the predicted structured data of each sample text output by the medical big language model and the sample structured data extracted from each sample text can be calculated, and the medical big language model can be fine-tuned based on the difference between each sample structured data and each predicted structured data corresponding to each sample structured data.
[0060] Step S24: Obtain target text.
[0061] Step S25: Use the fine-tuned medical large language model to perform a structuring task on the target text to obtain target structured data of the target text.
[0062] For the specific implementation of step S24 and step S25, please refer to the relevant description of steps S11 and S12, which will not be repeated here.
[0063] Those skilled in the art will appreciate that, in the above method of this embodiment, the writing order of the steps does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0064] In this implementation mode, sample structured data and predicted structured data are used to fine-tune the medical big language model, so that the medical big language model has the ability to structure text.
[0065] See also Figure 3 , Figure 3 is a flowchart of an implementation method of step S21 provided in the present application. Step S21 may include:
[0066] Step S311: input the sample text and the prompt text corresponding to the sample text into the data extraction model.
[0067] In one embodiment, the prompt text corresponding to the sample text is used to instruct the data extraction model to perform a data extraction task on the sample text. The prompt text is used to instruct the data extraction model what data to extract from the sample text. Different prompt texts may correspond to different types of structured tasks. For example, the prompt text of the extraction type is used to instruct the data extraction model to extract the value corresponding to the sample field in the prompt text of the extraction type from the sample text; the prompt text of the single-choice type is used to instruct the data extraction model to select the single-choice value corresponding to the sample field in the prompt text of the single-choice type from multiple single-choice values of the prompt text of the single-choice type; the prompt text of the multiple-choice type is used to instruct the data extraction model to select at least two multiple-choice values corresponding to the sample field in the prompt text of the multiple-choice type from multiple multiple-choice values of the prompt text of the multiple-choice type; the prompt text of the nested extraction type is used to instruct the data extraction model to find out from the sample text the value of the attribute represented by the sample field in the field item of the entity answer of the nested extraction type prompt text.
[0068] Step S312: using the data extraction model to perform a data extraction task on the sample text based on the prompt text to obtain sample structured data in the fine-tuning data set.
[0069] In this embodiment, the extraction of sample structured data from the sample text is performed by the data extraction model. Specifically, the sample text and the prompt text corresponding to the sample text can be input into the data extraction model, and the data extraction model is used to perform the data extraction task on the sample text based on the prompt text to obtain the sample structured data of the sample text; the sample structured data of each sample text is combined into a fine-tuning data set. Among them, the fine-tuning data set may include sample structured data of at least one value domain type, and the sample structured data includes sample fields in the sample text and values corresponding to the sample fields. The sample fields are similar to the fields in the target text, and the values corresponding to the sample fields are similar to the values corresponding to the fields in the target text, which will not be repeated here.
[0070] See also Figure 4 , Figure 4: is a flowchart of another embodiment of the text structuring method provided by the present application, the method comprising the following steps:
[0071] Step S41: Determine the sample fields in the sample text and the value domain types corresponding to the sample fields.
[0072] In one implementation scenario, the sample field can be pre-set, and the sample field can be part of the content in the sample text. For example, the sample field is an entity in the sample text, and an entity recognition method can be used to identify the entity in the sample text; the sample field can also be determined by the user, and a model can be used to extract entities in the sample text.
[0073] The value domain type corresponding to the sample field can be determined based on whether the value corresponding to the sample field in the sample text is a single answer, multiple answers, or an indefinite range of answers. If the value corresponding to the sample field in the sample text is a single answer, the value domain type corresponding to the sample field can be used as an extraction type or a single-select type; if the value corresponding to the sample field in the sample text is multiple answers, the value domain type corresponding to the sample field can be used as an extraction type, a nested extraction type, or a multiple-select type.
[0074] Step S42: constructing a prompt text corresponding to the sample text based on the sample field and the value domain type of the sample field.
[0075] In an implementation scenario, a prompt text template matching the value domain type of the sample field can be selected according to the value domain type of the sample field; wherein the prompt text template includes field items; the sample field is filled into the field item of the prompt text template to generate a prompt text corresponding to the sample text. wherein the value domain type includes at least one of an extraction type, a nested extraction type, a single selection type, and a multiple selection type.
[0076] In a specific implementation scenario, when the value domain type of the sample field includes an extraction type, the prompt text template matching the extraction type may include "##Structured Task##Based on the information below, determine what {field name} is? Please extract the corresponding value {medical text} in the text", where the "field name" of the prompt text template is the field item, and the prompt text template of the extraction type is used to indicate the value corresponding to the sample field filled in the field item in the extracted sample text. Fill the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text, and the value corresponding to the field item can be extracted according to the prompt text. For example, when the sample field is the inner diameter of the aortic root, the sample field is filled into the field item of the prompt text template, and the resulting prompt text includes "##Structured task##Based on the information below, determine what {the inner diameter of the aortic root} is? Please extract the corresponding value {medical text} in the text". If the sample text includes "measured value, inner diameter of the aortic root, 40; inner diameter of the left atrium, 37", the prompt text is input into the data extraction model, and the medical text output by the data extraction model is 40, which is the value corresponding to the sample field.
[0077] In a specific implementation scenario, the value domain type of the sample field includes a radio-select type, the prompt text template of the radio-select type also includes a radio-select value domain item, and the prompt text template of the radio-select type is used to indicate the selection of a radio-select value corresponding to the sample field in the sample text from multiple radio-select values of the radio-select value domain item; the sample field is filled into the field item of the prompt text template to generate a prompt text corresponding to the sample text, including: filling the sample field into the field item of the prompt text template, and filling multiple radio-select values corresponding to the sample field into the radio-select value domain item of the prompt text template to generate a prompt text corresponding to the sample text.
[0078] For example, the sample field is the number of drainage tubes, and the multiple radio-select values corresponding to the sample field are 1, 2, 3 or more, and not mentioned. Then the prompt text corresponding to the sample text may include "##Structured Task##Based on the information below, determine what {number of drainage tubes} is? Please select 1 {medical text} in the value range [1, 2, 3 or more, not mentioned]." If the sample text includes "One negative pressure drainage tube is placed in the upper pole of the right kidney, and one single-lumen drainage tube is placed in the hepatorenal recess (next to the common bile duct)", the data extraction model can obtain the value corresponding to the number of drainage tubes as 2 according to the prompt text and the sample text.
[0079] In a specific implementation scenario, the value domain type of the sample field includes a multiple-choice type, and the prompt text template of the multiple-choice type also includes a multiple-choice value domain item. The prompt text template of the multiple-choice type is used to indicate the selection of at least two multiple-choice values corresponding to the sample field in the sample text from multiple multiple-choice values of the multiple-choice value domain item; the sample field is filled into the field item of the prompt text template to generate a prompt text corresponding to the sample text, including: filling the sample field into the field item of the prompt text template, and filling the multiple multiple-choice values corresponding to the sample field into the multiple-choice value domain item of the prompt text template to generate a prompt text corresponding to the sample text.
[0080] For example, the sample field is the test result, and the multiple selection values corresponding to the sample field are sinus rhythm, sinus bradycardia, sinus tachycardia, sinus arrhythmia, atrial premature beats, ventricular premature beats, atrial tachycardia, ventricular tachycardia, low voltage, and normal electrocardiogram. Then the prompt text corresponding to the sample text may include "##Structured task##Based on the information below, determine what {test result} is? Please select all the contents mentioned in the text {medical text} in the value range [sinus rhythm, sinus bradycardia, sinus tachycardia, sinus arrhythmia, atrial premature beats, ventricular premature beats, atrial tachycardia, ventricular tachycardia, low voltage, and normal electrocardiogram]." If the sample text includes "sinus rhythm limb lead low voltage trend", the data extraction model can obtain the corresponding values of the test result as sinus rhythm and low voltage based on the prompt text and the sample text.
[0081] In a specific implementation scenario, the value domain type of the sample field includes a nested extraction type, and the prompt text template of the nested extraction type also includes an entity item. The prompt text template of the nested extraction type is used to indicate to find the value of a specified attribute from the sample text, and the specified attribute is the attribute represented by the entity answer in the entity item with respect to the sample field in the field item; the sample field is filled into the field item of the prompt text template to generate a prompt text corresponding to the sample text, including: filling the entity answer into the entity item, and filling the sample field into the field item of the prompt text template to generate a prompt text corresponding to the sample text.
[0082] For example, the entity answer of the entity item is diabetes, and the sample field is symptoms. The prompt text corresponding to the sample text is "##Structured Task##Based on the information below, determine what {diabetes} {symptoms} are? Please extract the corresponding value {medical text} in the text." If the sample text includes "The symptoms of diabetes are polydipsia, polyphagia, polyuria, and weight loss", the data extraction model can obtain the values of diabetes symptoms based on the prompt text and the sample text as polydipsia, polyphagia, polyuria, and weight loss.
[0083] Among them, the entity answer in the entity item can be pre-set or extracted from the sample text. In a specific implementation scenario, the entity answer in the entity item is extracted from the sample text by the data extraction model using the entity prompt text. Specifically, the nested extraction type also corresponds to an entity text extraction template, and the entity text extraction template includes an entity field item. The entity text extraction template is used to indicate the extraction of several values corresponding to the entity field filled in the entity field item in the sample text. Specifically, before filling the entity answer into the entity item and filling the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text, the entity field of the sample text can be filled into the entity field item of the entity text extraction template to generate the entity prompt text corresponding to the sample text; the data extraction model is used to perform the entity extraction task on the sample text based on the entity prompt text to obtain several values corresponding to the entity field; at least one of the several values corresponding to the entity field is used as the entity answer in the entity item.
[0084] For example, if the entity field of the sample text is a symptom, the entity text extraction template can be similar to the prompt text template that matches the extraction type, which will not be repeated here. Fill the symptom into the entity field item of the entity text extraction template to obtain the entity prompt text; all symptoms that appear in the sample text can be extracted using the entity prompt text. Select at least one of all the extracted symptoms as the entity answer.
[0085] In a specific implementation scenario, the prompt text corresponding to the sample text also includes a special description item, and the text content filled in the special description item is used for at least one of the following: providing the data extraction model with understanding of the sample field, and indicating the output format of the data extraction model. For example, the value domain type is a single-choice type, the sample field is the relationship with the patient, and the multiple single-choice values are father, mother, brother, sister, grandfather, grandmother, grandfather, grandmother, other collateral relatives, and not mentioned. The text content filled in the special description item is to output only relatives with diseases. The sample text is "Father is 37 years old, healthy, blood type B, work: IT industry. Mother is 33 years old, healthy, blood type B, work: finance. Brother is 1 year old, healthy. Grandfather has a history of hepatitis B", then the data output by the data extraction model is grandfather.
[0086] Step S43: input the sample text and the prompt text corresponding to the sample text into the data extraction model.
[0087] Step S44: using the data extraction model to perform a data extraction task on the sample text based on the prompt text to obtain sample structured data in the fine-tuning data set.
[0088] In one embodiment, the sample text and the prompt text are input into the data extraction model, and the content output by the data extraction model is the sample structured data of the sample text; or the content output by the data extraction model is processed to obtain the sample structured data of the sample text.
[0089] For the specific implementation of step S43 and step S44, please refer to the relevant description of step S311 and step S312, which will not be repeated here.
[0090] Step S45: Use the medical big language model to perform structured processing on the sample texts corresponding to each sample structured data to obtain predicted structured data corresponding to each sample structured data.
[0091] Step S46: fine-tune the medical large language model based on each sample structured data and the corresponding predicted structured data.
[0092] Step S47: Obtain target text.
[0093] Step S48: Use the fine-tuned medical large language model to perform a structuring task on the target text to obtain target structured data of the target text.
[0094] Among them, for the specific implementation of step S45 and step S46, please refer to the relevant description of step S22 and step S23, and for the specific implementation of step S47 and step S48, please refer to the relevant description of step S11 and step S12, which will not be repeated here.
[0095] Those skilled in the art will appreciate that, in the above method of this embodiment, the writing order of the steps does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0096] This embodiment can obtain sample structured data that can adapt to different structured tasks by constructing prompt texts corresponding to sample texts of different value domain types. The sample structured data that can adapt to different structured tasks is used to fine-tune the medical language model so that the medical language model can adapt to various types of structured tasks.
[0097] See also Figure 5 , Figure 5 : is a flowchart of an implementation method of a training method for a medical large language model provided by the present application, the method comprising the following steps:
[0098] Step S51: Obtain a fine-tuning dataset.
[0099] The fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from a corresponding sample text using a data extraction model.
[0100] Step S52: Use the medical big language model to perform structured processing on the sample texts corresponding to each sample structured data to obtain predicted structured data corresponding to each sample structured data.
[0101] Step S53: fine-tune the medical large language model based on each sample structured data and the corresponding predicted structured data.
[0102] For the specific steps, please refer to the relevant contents in the aforementioned embodiments, which will not be described in detail here.
[0103] In this embodiment, the sample structured data in the fine-tuning data set may correspond to sample structured data of multiple value domain types. After fine-tuning the medical large language model using all the sample structured data, the medical large language model can perform multiple types of structured tasks corresponding to multiple value domain types. It can be understood that in other embodiments, the sample structured data in the fine-tuning data set may correspond to sample structured data of only one value domain type. After fine-tuning the medical large language model using the sample structured data, the medical large language model can perform structured tasks of the type corresponding to the value domain type.
[0104] See also Figure 6 , Figure 6 It is a schematic diagram of the framework of an embodiment of a text structuring device provided by the present application.
[0105] The text structuring device 60 may include a first acquisition module 61 and a first processing module 62, wherein the first acquisition module 61 is used to acquire the target text; the first processing module 62 is used to perform a structuring task on the target text using a fine-tuned medical big language model to obtain target structured data of the target text; wherein the fine-tuned medical big language model is obtained by fine-tuning the medical big language model using sample text and sample structured data of the sample text; and the sample structured data is obtained by extracting the sample text using a data extraction model.
[0106] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above text structured method embodiment. Its specific implementation can refer to the description of the above method embodiment. For the sake of brevity, it will not be repeated here.
[0107] See also Figure 7 , Figure 7 It is a schematic diagram of the framework of an embodiment of a training device for a large medical language model provided in the present application.
[0108] The training device 70 of the medical big language model includes a second acquisition module 71, a second processing module 72 and a fine-tuning module 73. The second acquisition module 71 is used to acquire a fine-tuning data set, wherein the fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from a corresponding sample text using a data extraction model; the second processing module 72 is used to use the medical big language model to perform structured processing on the sample text corresponding to each sample structured data, and obtain the predicted structured data corresponding to each sample structured data; the fine-tuning module 73 is used to fine-tune the medical big language model based on each sample structured data and the corresponding predicted structured data.
[0109] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the method described in the above medical large language model training method embodiment. Its specific implementation can refer to the description of the above method embodiment, and for the sake of brevity, it will not be repeated here.
[0110] See also Figure 8 , Figure 8 It is a schematic diagram of the framework of an embodiment of an electronic device provided by the present application.
[0111] In this embodiment, the electronic device 80 includes a memory 81 and a processor 82, and the processor 82 is used to execute the program instructions stored in the memory 81 to implement the steps in any of the above-mentioned text structuring methods or model training method embodiments. In a specific implementation scenario, the electronic device 80 may include, but is not limited to: computer equipment, electrical equipment, microcomputers, desktop computers, servers, and in addition, the electronic device 80 may also include mobile devices such as laptops and tablet computers, which are not limited here.
[0112] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps in the training method embodiment of any of the above-mentioned text structuring methods or models. The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 82 can be implemented by an integrated circuit chip.
[0113] See also Fig. 9 , Fig. 9 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application.
[0114] The computer-readable storage medium 90 provided in this embodiment stores program instructions 91 that can be executed by a processor. When the program instructions 91 are executed by the processor, they are used to implement any of the above-mentioned text structuring methods or model training methods.
[0115] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0116] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another subsystem, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0117] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0118] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.
[0119] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A text structuring method, characterized in that: include: Get the target text; Using the fine-tuned medical large language model to perform a structuring task on the target text to obtain target structured data of the target text; The fine-tuned medical big language model is obtained by fine-tuning the medical big language model using sample text and sample structured data of the sample text; and the sample structured data is obtained by extracting the sample text using a data extraction model.
2. The method according to claim 1, characterized in that The target structured data includes fields in the target text and values corresponding to the fields. The fine-tuned medical large language model can perform multiple types of structured tasks. Each type of structured task corresponds to the output of structured data of a value domain type. The value domain type of the structured data is determined by the value domain corresponding to the fields in the structured data.
3. The method according to claim 2, characterized in that The value domain type includes at least one of an extraction type, a nested extraction type, a single-select type, and a multiple-select type.
4. The method according to any one of claims 1 to 3, characterized in that: Before performing the structuring task on the target text using the fine-tuned medical large language model to obtain target structured data of the target text, the method further includes: Acquire a fine-tuning data set, wherein the fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from a corresponding sample text; Using the medical big language model, respectively perform structural processing on the sample texts corresponding to each of the sample structured data to obtain predicted structured data corresponding to each of the sample structured data; Based on each of the sample structured data and the corresponding predicted structured data, the medical large language model is fine-tuned.
5. The method according to claim 4, characterized in that The obtaining of the fine-tuning dataset comprises: Inputting the sample text and the prompt text corresponding to the sample text into the data extraction model, wherein the prompt text corresponding to the sample text is used to instruct the data extraction model to perform a data extraction task on the sample text; The data extraction model is used to perform the data extraction task on the sample text based on the prompt text to obtain sample structured data in the fine-tuning dataset.
6. The method according to claim 5, characterized in that The fine-tuning data set includes sample structured data of at least one value domain type, and the sample structured data includes sample fields in the sample text and values corresponding to the sample fields; Before inputting the sample text and the prompt text corresponding to the sample text into the data extraction model, the method further includes: Determine the sample field in the sample text and the value domain type corresponding to the sample field; Based on the sample field and the value domain type of the sample field, a prompt text corresponding to the sample text is constructed.
7. The method according to claim 6, characterized in that The constructing the prompt text corresponding to the sample text based on the sample field and the value domain type of the sample field includes: Selecting a prompt text template that matches the value domain type of the sample field; wherein the prompt text template includes field items; Fill the sample field into the field item of the prompt text template to generate a prompt text corresponding to the sample text.
8. The method according to claim 7, characterized in that The value domain type includes an extraction type, and the prompt text template of the extraction type is used to instruct to extract the value corresponding to the sample field filled in the field item in the sample text; and / or, The value domain type includes a radio type, the prompt text template of the radio type also includes a radio value domain item, and the prompt text template of the radio type is used to indicate to select a radio value corresponding to the sample field in the sample text from multiple radio values of the radio value domain item; The step of filling the sample field into the field item of the prompt text template to generate a prompt text corresponding to the sample text comprises: filling the sample field into the field item of the prompt text template, and filling a plurality of radio-select values corresponding to the sample field into the radio-select value field item of the prompt text template to generate a prompt text corresponding to the sample text; and / or, The value domain type includes a multiple-choice type, and the prompt text template of the multiple-choice type also includes a multiple-choice value domain item, and the prompt text template of the multiple-choice type is used to indicate selecting at least two multiple-choice values corresponding to the sample field in the sample text from multiple multiple-choice values of the multiple-choice value domain item; the filling of the sample field into the field item of the prompt text template to generate the prompt text corresponding to the sample text includes: filling the sample field into the field item of the prompt text template, and filling the multiple multiple-choice values corresponding to the sample field into the multiple-choice value domain item of the prompt text template to generate the prompt text corresponding to the sample text; and / or, The value domain type includes a nested extraction type, and the prompt text template of the nested extraction type also includes an entity item. The prompt text template of the nested extraction type is used to indicate to find the value of a specified attribute from the sample text, and the specified attribute is the attribute represented by the entity answer in the entity item with respect to the sample field in the field item; filling the sample field into the field item of the prompt text template to generate a prompt text corresponding to the sample text includes: filling the entity answer into the entity item, and filling the sample field into the field item of the prompt text template to generate a prompt text corresponding to the sample text.
9. The method according to claim 8, characterized in that The extraction type includes a nested extraction type, the nested extraction type further corresponds to an entity text extraction template, and the entity text extraction template includes an entity field item; Before filling the entity answer into the entity item and filling the sample field into the field item of the prompt text template to generate a prompt text corresponding to the sample text, the method further includes: Filling the entity field of the sample text into the entity field item of the entity text extraction template to generate entity prompt text corresponding to the sample text; Using the data extraction model to perform the entity extraction task on the sample text based on the entity prompt text, so as to obtain a number of values corresponding to the entity field; At least one of the multiple values corresponding to the entity field is used as the entity answer in the entity item.
10. The method according to claim 6, characterized in that The prompt text also includes a special instruction item, and the text content filled in the special instruction item is used for at least one of the following: providing the data extraction model with an understanding of the sample field, and indicating the output format of the data extraction model.
11. The method according to claim 1, characterized in that: The target text is an electronic medical record text.
12. A method for training a large medical language model, characterized in that: include: Acquire a fine-tuning data set, wherein the fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from a corresponding sample text using a data extraction model; Using the medical big language model, respectively perform structural processing on the sample texts corresponding to each of the sample structured data to obtain predicted structured data corresponding to each of the sample structured data; Based on each of the sample structured data and the corresponding predicted structured data, the medical large language model is fine-tuned.
13. A text structuring device, characterized in that: include: A first acquisition module, used for acquiring a target text; The first processing module is used to perform a structuring task on the target text using a fine-tuned medical big language model to obtain target structured data of the target text; wherein the fine-tuned medical big language model is obtained by fine-tuning the medical big language model using sample text and sample structured data of the sample text; and the sample structured data is obtained by extracting the sample text using a data extraction model.
14. A training device for a large medical language model, characterized in that: include: A second acquisition module is used to acquire a fine-tuning data set, wherein the fine-tuning data set includes at least one sample structured data, and each sample structured data is extracted from a corresponding sample text using a data extraction model; A second processing module is used to perform structured processing on the sample texts corresponding to each of the sample structured data using the medical large language model to obtain predicted structured data corresponding to each of the sample structured data; A fine-tuning module is used to fine-tune the medical large language model based on each of the sample structured data and the corresponding predicted structured data.
15. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores program instructions, and the processor is configured to execute the program instructions to implement the method according to any one of claims 1 to 12.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and the program instructions can be executed to implement the method according to any one of claims 1 to 12.