Method and apparatus for automatically annotating medical text by constructing prompt on basis of large model, device, and medium

Abstract

A method and apparatus for automatically annotating a medical text by constructing a prompt on the basis of a large model, a device, and a medium. The method comprises: preprocessing medical text data to generate a sentence sample; constructing an entity-type-agnostic Prompt template on the basis of the sentence sample, and capturing general entity information in a medical text; constructing an entity-type-aware Prompt template on the basis of the captured general entity information and a preset entity type, and capturing entity type information in the medical text; constructing an entity-relation-type-agnostic Prompt template on the basis of the captured entity type information and a preset general entity-relation type, and capturing general entity-relation information in the medical text; and constructing an entity-relation-type-aware Prompt template on the basis of the captured general entity-relation information and a preset entity-relation type, and capturing entity-relation type information in the medical text.

Description

Method, device and equipment for automatically annotating medical text based on large model construction prompt and medium

[0001] The present application relates to the technical field of medical text annotation, and particularly relates to a method, device and equipment for automatically annotating medical text based on large model construction prompt and medium. BACKGROUND

[0002] In the medical field, a large amount of text data needs to be accurately labeled to extract valuable information, such as medical records, medical literature and clinical trial reports, etc. Traditional text annotation methods rely on expert manual labeling, which is time-consuming and costly. Although there are some automated text annotation tools, they usually rely on specific domain rules or statistical models, which are difficult to adapt to complex and variable medical texts.

[0003] In recent years, with the rapid development of pre-trained language models (such as GPT series, BERT, T5, etc.), they have shown great potential in natural language processing, especially in text annotation tasks. Using pre-trained general language models for predictive annotation of medical texts often fails to fully utilize the specific knowledge and context of the medical field, resulting in low annotation accuracy.

[0004] Through the above, the rule-based method needs to manually write a large number of rules, which is difficult and prone to errors for complex medical texts. The machine learning-based method requires a large amount of labeled data for model training, but it is very difficult to obtain a large amount of labeled data in the medical field. The general language model fails to fully integrate medical professional knowledge, affecting the annotation quality. SUMMARY

[0005] The present application provides a method, device and equipment for automatically annotating medical text based on large model construction prompt.

[0006] In a first aspect, the present application provides a method for automatically annotating medical text based on large model construction prompt, comprising the following steps:

[0007] Preprocessing the medical text data to generate sentence samples;

[0008] Based on the sentence samples, an entity non-zone classification Prompt template is constructed to capture general entity information in the medical text;

[0009] Based on the captured general entity information, an entity type is preset, and an entity zone classification Prompt template is constructed to capture entity type information in the medical text;

[0010] Based on the captured entity type information, a preset general entity relationship type is constructed, an entity relationship no-zone classification Prompt template is constructed, and general information of entity relationships in the medical text is captured.

[0011] Based on the captured general information of entity relationships, a preset entity relationship type is constructed, an entity relationship zone classification Prompt template is constructed, and category information of entity relationships in the medical text is captured.

[0012] As a further limitation of the technical scheme of the application, the step of preprocessing the medical text data to generate sentence samples includes:

[0013] Based on massive medical text data, data tables and data elements are screened, and medical text data in the data elements is extracted. Medical text data is collected, including medical records, medical papers, drug instruction manuals, etc.

[0014] As a further limitation of the technical scheme of the application, the step of preprocessing the text data to generate sentence samples includes:

[0015] The extracted large text data is divided into sentences according to the preset characteristic symbols;

[0016] The abnormal values in the divided sentences are detected and missing values are processed;

[0017] The sentences after missing value processing are subjected to overall deduplication processing through similarity calculation;

[0018] The deduplicated sentences are generated as sentence samples; the similarity between the sentences in the sentence samples is zero.

[0019] As a further limitation of the technical scheme of the application, based on the sentence samples, the step of constructing an entity no-zone classification Prompt template to capture general entity information in the medical text includes:

[0020] Based on the sentence samples, each character combination in the sentence is obtained;

[0021] An entity no-zone classification Prompt template for each character is constructed;

[0022] A large model inference service is called, character combinations belonging to general entities are selected, and a first-stage output result is constructed.

[0023] As a further limitation of the technical scheme of the application, based on the captured general entity information, the step of constructing an entity zone classification Prompt template to capture entity type information in the medical text includes:

[0024] Based on the first-stage output result, a preset entity category is constructed;

[0025] The entity region classification Prompt template of each general entity is constructed, the large model inference service is called, the character combination belonging to the entity category is selected, and the second stage output result, that is, the captured entity type information, is constructed.

[0026] As a further limitation of the technical scheme of the application, based on the captured entity type information, a general entity relationship type is preset, an entity relationship non-region classification Prompt template is constructed, and the steps of capturing general information of entity relationships in the medical text include:

[0027] Based on the second stage output result, an entity relationship is preset.

[0028] The entity relationship non-region classification Prompt template between each category of entities is constructed, the large model inference service is called, the entity type combination belonging to the general relationship is selected, and the third stage output result is constructed.

[0029] As a further limitation of the technical scheme of the application, based on the captured entity relationship general information, an entity relationship type is preset, an entity relationship region classification Prompt template is constructed, and the steps of capturing entity relationship category information in the medical text include:

[0030] Based on the third stage output result, the entity type combination belonging to the general relationship in the sentence is obtained, and the entity relationship type is preset.

[0031] The entity relationship region classification Prompt template between each entity type is constructed, the LLM inference service is called, each entity type combination belonging to the entity relationship type is selected, and the final output result is constructed.

[0032] In a second aspect, the technical scheme of the application also provides a device for automatically labeling medical text based on a large model constructed prompt, which comprises a medical text data acquisition module, a text data preprocessing module, a general entity information acquisition module, an entity type information acquisition module, an entity relationship general information acquisition module, and an entity relationship category information acquisition module.

[0033] The medical text data acquisition module is used for filtering data tables and data elements based on massive medical text data, and extracting medical text data in the data elements.

[0034] The text data preprocessing module is used for preprocessing the medical text data to generate sentence samples.

[0035] The general entity information acquisition module is used for constructing an entity non-region classification Prompt template based on the sentence samples, and capturing general entity information in the medical text.

[0036] The entity type information acquisition module is configured to acquire entity type information in the medical text based on the captured general entity information, preset entity types, construct an entity region classification Prompt template, and capture entity type information in the medical text.

[0037] The entity relationship general information acquisition module is configured to acquire entity relationship general information in the medical text based on the captured entity type information, preset general entity relationship types, construct an entity relationship region classification Prompt template, and capture entity relationship general information in the medical text.

[0038] The entity relationship category information acquisition module is configured to acquire entity relationship category information in the medical text based on the captured entity relationship general information, preset entity relationship types, construct an entity relationship region classification Prompt template, and capture entity relationship category information in the medical text.

[0039] As a further limitation of the technical scheme of the present application, the text data preprocessing module comprises a sentence segmentation unit, an anomaly detection processing unit, a deduplication processing unit, and a sentence sample generation unit.

[0040] The sentence segmentation unit is configured to segment the extracted large text data into sentences according to the preset characteristic symbols.

[0041] The anomaly detection processing unit is configured to detect and handle missing values in the segmented sentences.

[0042] The deduplication processing unit is configured to perform overall deduplication processing on the sentences after missing value handling through similarity calculation.

[0043] The sentence sample generation unit is configured to generate sentence samples from the sentences after deduplication processing; in the sentence samples, the similarity between sentences is zero.

[0044] As a further limitation of the technical scheme of the present application, the general entity information acquisition module is specifically configured to acquire each character combination in the sentence based on the sentence sample, construct an entity region classification Prompt template for each character, call a large model inference service, select character combinations belonging to general entities, and construct a first-stage output result.

[0045] As a further limitation of the technical scheme of the present application, the entity type information acquisition module is specifically configured to preset entity categories based on the first-stage output result, construct an entity region classification Prompt template for each general entity, call a large model inference service, select character combinations belonging to entity categories, construct a second-stage output result, and capture entity type information.

[0046] As a further limitation of the technical solution of the application, the entity relationship general information acquisition module is specifically used for presetting entity relationships based on the second stage output result; constructing entity relationship non-zone classification Prompt templates between various types of entities, calling a large model inference service, selecting entity type combinations belonging to general relationships, and constructing a third stage output result.

[0047] As a further limitation of the technical solution of the application, the entity relationship class information acquisition module is specifically used for obtaining entity type combinations belonging to general relationships in a sentence based on the third stage output result, presetting entity relationship types; constructing entity relationship zone classification Prompt templates between various types of entities, calling an LLM inference service, selecting various entity type combinations belonging to entity relationship types, and constructing a final output result.

[0048] In a third aspect, the technical solution of the application also provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the method for automatically labeling medical text based on a large model constructing prompt according to the first aspect.

[0049] In a fourth aspect, the technical solution of the application also provides a non-transitory computer readable storage medium storing computer instructions, and the computer instructions enable the computer to execute the method for automatically labeling medical text based on a large model constructing prompt according to the first aspect.

[0050] As can be seen from the above technical solution, the application has the following advantages: through the automatic prompt construction and labeling process, the time and cost of manual labeling can be significantly reduced, and the efficiency of medical text labeling can be improved.

[0051] The application of large models makes the labeling process more intelligent and can handle large-scale medical text data. The combination of entity non-classified and classified Prompt templates can ensure the capture of general entity information and specific entity type information in medical text, thereby improving the accuracy of labeling. The use of entity relationship non-classified and classified Prompt templates can more accurately capture general entity relationship information and specific relationship category information in medical text. Since this method is based on large models, it has strong generalization ability and can adapt to medical text data in different fields and styles. By presetting entity types and entity relationship types, flexible and customizable solutions can be provided for different medical text labeling tasks. The automated labeling process reduces the influence of human factors on the labeling results, thereby reducing the labeling error rate. The careful design and construction of Prompt templates help to more accurately extract key information from medical text, further reducing the labeling error rate.

[0052] In addition, the design principle of the present application is reliable, the structure is simple, and it has very wide application prospects.

[0053] As can be seen, compared with the prior art, the present application has outstanding substantial characteristics and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art descriptions. Obviously, for those of ordinary skill in the art, other drawings can also be obtained without creative labor.

[0055] FIG. 1 is a schematic flowchart of the method of an embodiment of the present application.

[0056] FIG. 2 is a schematic block diagram of the device of an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the present application.

[0058] As shown in FIG. 1, the present application provides a method for automatically labeling medical text based on large model construction prompt, which includes the following steps:

[0059] Step 1: Based on massive medical text data, screening data table and data elements, and extracting medical text data in data elements;

[0060] Step 2: Preprocessing the medical text data to generate sentence samples;

[0061] Step 3: Based on the sentence samples, constructing entity non-zone classification Prompt template to capture general entity information in medical text;

[0062] Step 4: Based on the captured general entity information, preset entity type, construct entity zone classification Prompt template, capture entity type information in medical text;

[0063] Step 5: Based on the captured entity type information, preset general entity relationship type, construct entity relationship non-zone classification Prompt template, capture general entity relationship information in medical text;

[0064] Step 6: Based on the captured general entity relationship information, preset entity relationship type, construct entity relationship zone classification Prompt template, capture entity relationship category information in medical text.

[0065] In some embodiments, the step of preprocessing the text data to generate sentence samples includes:

[0066] The extracted large text data is divided into sentences according to the preset feature symbols such as period, line feed, etc. The cleaning function detects and processes abnormal values and handles missing values in the sentence. The deduplication function removes the whole sentence. The desensitization function de-identifies, anonymizes or encrypts sensitive information to protect personal privacy and business secrets, generating sentence samples. The similarity between sentences in the sentence samples is zero.

[0067] In some embodiments, the step of constructing an entity non-zone classification Prompt template based on the sentence samples to capture general entity information in medical text includes:

[0068] Based on the sentence samples, obtaining each character combination in the sentence;

[0069] Constructing an entity non-zone classification Prompt template for each character;

[0070] Calling a large model inference service, selecting character combinations belonging to general entities, and constructing the first stage output result.

[0071] Based on a sentence sample, such as: body temperature: 36.5, obtain each character combination in the sentence, construct the 'entity non-zone classification Prompt template' of each character, call the LLM inference service, select the character combination belonging to 'general entity', and construct the first stage output result:

[0072] First, the character combination of the sentence 'body temperature: 36.5' has 28 kinds, which are ['body', 'body temperature', 'body temperature:', 'body temperature:3', 'body temperature:36', 'body temperature:36.', 'body temperature:36.5', 'temperature', 'temperature:', 'temperature:3', 'temperature:36', 'temperature:36.', 'temperature:36.5', ':', ':3', ':36', ':36.', ':36.5', '3', '36', '36.', '36.5', '6', '6.', '6.5', '.', '.5', '5'], each of which is a character fragment, such as the first character fragment 'body', which constructs the entity non-zone classification Prompt template of the character fragment. The starting position of the character fragment 'body' in the text is '0' and the end position is '1'; the remaining 27 fragments are also constructed in the same way 'entity non-zone classification Prompt template', and the template data is passed to the LLM inference service. The inference service is an inference service API interface encapsulated by the llama2-13b-base model after fine-tuning, deployed as a docker service, which loads the fine-tuned model after starting. When the API interface receives a template data request, tokenize the input prompts to get token ids; calculate the maximum length total_len of the current batch to create the input token tensor, and the maximum length cannot exceed the cache size; from the position of the shortest prompt in the current batch, as the starting position of generation, start generating; input the token tensor into the transformer model to calculate logits, get logits with a shape of (batch_size, hidden_size) (the output of the last layer of the transformer); softmax+top_p sampling to get the current predicted token and update the current position to prepare for predicting the next token; decode to get the generated text.

[0073] In the template data, the model's answer is limited to [‘belongs to’,‘does not belong to’], and the inference result is further post-processed. When the model answer is 'belongs to', the character fragment in the template data is considered as 'general entity', and the output result is as follows:

[0074] [

[0075] {

[0076] "text": "Body temperature: 36.5",

[0077] "entities": [

[0078] {

[0079] "entity": "body temperature",

[0080] "start_idx": 0,

[0081] "end_idx": 2,

[0082] "type": "General Entity",

[0083] "id": "0"

[0084] },

[0085] {

[0086] "entity": "36.5",

[0087] "start_idx": 3,

[0088] "end_idx": 6,

[0089] "type": "General Entity",

[0090] "id": "1"

[0091] }

[0092] ]

[0093] }

[0094] In some embodiments, the steps of capturing entity type information in medical text include: based on captured general entity information, pre-setting entity types, constructing an entity differentiation category Prompt template, and capturing entity type information.

[0095] Based on the first stage output result, preset entity categories are determined;

[0096] An entity region category prompt template of each general entity is constructed, a large model inference service is called, a character combination belonging to the entity category is selected, and a second stage output result, i.e., captured entity type information, is constructed.

[0097] Based on the first stage output result, the preset entity category partial result is as follows:

[0098] {

[0099] "diagnosis": "disease with clear nature of lesion, pregnancy-related diagnosis, currently medically recognized disease with unknown cause: such as "Reese's syndrome", "Ogilvie syndrome" and other diseases (mainly xxx syndrome). Diseases with clear nature of lesion include clinically clear diseases and pathological diagnosis, such as diabetes and lung small cell carcinoma."

[0100] "pathogen": "a biological agent that causes a patient to develop a certain disease, such as bacteria, viruses, fungi, and parasites."

[0101] "medical device": "some medical auxiliary devices used in medical diagnosis and treatment."

[0102] "observation item": "mainly describes the characteristics of the described object (matter or event) from a certain dimension, and also includes some quantitative evaluation of the function or lesion level of medicine."

[0103] "abnormal cells": "abnormal cells."

[0104] "part and lesion positioning word": "division information indicating the structure inside the part. For example, leaf, segment, broken end, wall, tube wall, muscle layer, cutting end, cyst wall, center, tumor bed, inside, superficial muscle layer, etc."

[0105] "observation result": "normal, good conclusion description."

[0106] "inspection item": "1. Inspection methods, instrument inspection items, and test major items applied to patients in order to discover, confirm diseases or symptoms, and find more information about diseases or symptoms; 2. Test sub-items including test index items, immunohistochemical index, and gene index."

[0107] "human body secretion": "mainly used to describe the visible human body secretion or excretion components of gross organs or lesions, or the human body tissue components separated from the human body."

[0108] "Treatment": "Various conventional treatment methods medically performed, excluding surgical procedures, including chemotherapy, radiotherapy, targeted therapy, laser therapy, psychological therapy, general treatment (such as exercise, diet, rest), rehabilitation therapy, etc."

[0109] "Drug": "Using the name of a chemical drug, including biological treatment agents such as vaccines, biological tumor vaccines, and contrast agents used for imaging examinations."

[0110] "Usage method": "Only for the dosage, time, route of administration, and frequency of drug use, and is marked separately."

[0111] "Nursing classification": "Nursing type or level."

[0112] "Modification": "Modification of certain attributes of the described object, including size, shape, color, etc."

[0113] "Score item": "Currently only ECOG, KPS, PS, IPI, aaIPI, and postnatal Apgar score, or other abbreviations that are determined to be scores."

[0114] "Imaging findings": "1. The appearance of the disease on sonography, and the projection of the lesion on imaging; 2. The imaging characteristics of the lesion, a sonographic manifestation of the lesion or site on imaging, used to determine the nature of the lesion."

[0115] "Symptoms": "Abnormal sensations and states, abnormal clinical and examination findings exhibited by the body due to the occurrence of disease, in addition to signs."

[0116] "Drug allergen": "Describes the patient's allergy history and the drug related to the allergy."

[0117] "Anesthesia method": "Mainly general anesthesia, local anesthesia, local infiltration anesthesia, intravenous anesthesia, etc."

[0118] "Feature verb": "Characteristic verb in medical text description with specific meaning and connection between entities."

[0119] "Family member": "Description of family members in family history, used to avoid linking family members' diseases or surgery information in family history or birth history fields to the patient's information."

[0120] "Cell": "Used to mark normal cells and cells that are difficult to determine whether they are normal."

[0121] "surgical procedure": "includes surgical, procedure, puncture, biopsy, and other invasive diagnostic and therapeutic procedures."

[0122] "quantitative result": "includes quantitative results or positive or negative results, grade descriptions (+++), etc."

[0123] "possibility": "used to express the possibility of a certain diagnosis, symptoms, side effects, or the occurrence of a certain treatment."

[0124] "mass": "lumps, masses, lumps, nodules, neoplasms, space-occupying lesions, packages, blocks."

[0125] "chemotherapy regimen": "chemotherapy regimen name."

[0126] "typing and grading": "indicates the type or grade of the disease or diagnosis."

[0127] "filter item": "a medical term with no special meaning."

[0128] "organization": "mainly describes the name, level, department, and other medical institutions."

[0129] "time": "represents a time point, including date and time node."

[0130] "life event": "events in life, mainly the description of the triggering factors of symptoms or diseases and the history of traumatic events."

[0131] "material environment": "mainly describes the personal history of radioactive substances, toxic substances, and epidemic situations."

[0132] Based on the first stage output result, build the 'entity region classification prompt template', such as the first 'general entity' content 'temperature', whose starting position is 0 and ending position is 2. Based on this information, the 'entity region classification prompt template' is constructed as follows:

[0133] "According to the definition of entity category and the given text, complete the task specified.

[0134] Definition of 'diagnosis' entity category: "disease with clear nature of lesion, pregnancy-related diagnosis, and currently recognized disease with unknown etiology: such as "Reese's syndrome" and "Ogilvie syndrome" (mainly xxx syndrome). Diseases with clear nature of lesion include clinically clear diseases and pathological diagnoses, such as diabetes and lung small cell carcinoma."

[0135] Text: Temperature: 36.5;

[0136] Question: Does the character fragment 'temperature' in the text belong to the 'diagnosis' class entity?

[0137] Notes:

[0138] 1. The starting position of the character fragment 'temperature' in the text is '0', and the ending position is '2';

[0139] Answer 'belongs' or 'does not belong';

[0140] The answer is """, which needs to be combined with the remaining 48 entity categories. The 'Entity Region Category Prompt Template' is also constructed in this way. The template data is passed to the LLM inference service. The inference service is a reasoning service API interface that encapsulates the llama2-13b-base model after fine-tuning, which is deployed as a docker service. After the service starts, it loads the fine-tuned model. When the API interface receives a template data request, it tokenizes the input prompts to get token ids. Calculate the maximum length total_len of the current batch to create the input token tensor. The maximum length cannot exceed the cache size. From the position of the shortest prompt in the current batch, start generating. Input the token tensor into the transformer model to calculate logits, which is the output of the last layer of the transformer with a shape of (batch_size, hidden_size). Softmax + top_p sampling to get the current predicted token and update the current position to prepare for the next token prediction. Decode to get the generated text.

[0141] In some embodiments, based on the captured entity type information, a preset general entity relationship type is constructed, and the steps of capturing general entity relationship information in the medical text include:

[0142] Based on the second stage output result, preset entity relationship;

[0143] Constructing an entity relationship without region category prompt template between various categories of entities, calling a large model inference service, selecting entity type combinations that belong to general relationships, and constructing a third stage output result.

[0144] Based on the output results of the second stage, a total of 3 entity types are obtained, the first 'observation item' type entity content 'temperature' and the second 'vital sign' type entity, and the 'entity relationship no category classification prompt template' is constructed as:

[0145] "Please complete the specified judgment according to the definition of the relationship and the given text.

[0146] The definition of the 'general relationship' relationship: a relationship that meets certain connections in semantics and medicine.

[0147] Text: Temperature: 36.5

[0148] Question: Determine whether the entity 'temperature' and the entity 'temperature' in the text satisfy the 'general relationship' relationship. Do not consider the direction of the relationship.

[0149] Notes:

[0150] 1. The entity 'temperature' is of the type 'observation item', with a starting position of '0' and an ending position of '2' in the text.

[0151] 2. The entity 'temperature' is of the type 'vital sign', with a starting position of '0' and an ending position of '2' in the text.

[0152] Answer'satisfy' or 'not satisfy'.

[0153] The answer is """, and the 'entity relationship no category classification prompt template' is constructed. The template data is passed to the LLM reasoning service, which is a reasoning service API interface encapsulated by the Qwen1.5-14B model after fine-tuning different from the first stage. The service is deployed as a docker service, which loads the fine-tuned model after starting. When the API interface receives template data requests, it tokenizes the input prompts to obtain token ids; calculate the maximum length total_len of the current batch to create the input token tensor, and the maximum length cannot exceed the cache size; from the position of the shortest prompt in the current batch, as the starting position of generation, start generating; input the token tensor into the transformer model to calculate logits, which are the output of the last layer of the transformer with a shape of (batch_size, hidden_size); softmax + top_p sampling to get the current predicted token and update the current position to prepare for predicting the next token; decode to get the generated text.

[0154] In some embodiments, the steps of capturing entity relationship category information in medical text include: based on the captured general entity relationship information, pre-setting entity relationship types, constructing an entity relationship distinction category Prompt template, and including:

[0155] Based on the output of the third stage, obtain the combinations of entity types belonging to the general relation in the sentence, and preset the entity relation types;

[0156] Construct a Prompt template to distinguish the entity relationships between different entity types, call the LLM inference service, select the combinations of entity types that belong to the entity relationship types, and construct the final output result.

[0157] Preset entity relationship types include:

[0158] {

[0159] "Modification": "Usually indicates the subject's description of a certain property, state, characteristic, or change of an object. This relationship helps us understand the specific connection between the subject and object in more detail, providing more information about events, states, or characteristics in the text."

[0160] "Conclusion / Outcome": This typically represents the final assessment or summary of an observation, examination, or treatment. This conclusion or outcome is derived from medical, examination, surgical, or medical record descriptions and is used to describe the patient's condition, physical status, or treatment effectiveness. It is a summary of key observations or actions in medical information, providing a comprehensive understanding of the patient's condition. These conclusions or outcomes may involve information on physiological status, disease diagnosis, treatment effectiveness, pathological results, etc.

[0161] "Parallel": This refers to the simultaneous existence or manifestation of two or more symptoms, diseases, or medical features described in the text. These items are listed side-by-side to indicate that they occurred or coexisted at a specific point in time or within a specific context.

[0162] "Treatment" refers to the measures or medical procedures taken or applied during medical treatment to alleviate, improve, or cure a patient's symptoms, disease, or condition. These treatments include medication, surgery, physical therapy, radiation therapy, chemotherapy, etc. In a triad, the subject of the "treatment" relation is usually a medical procedure, drug, or treatment plan, and the predicate is the associated symptom, disease, or condition.

[0163] "Causes": "Refers to a causal relationship where the first element is the cause event or condition, the second element is the resulting effect or impact, and the third element is the reason for the cause event or condition. Specifically, the 'causes' relationship here indicates that one event, action, or condition has led to or resulted in another event, symptom, or outcome."

[0164] "Accumulates / transfers / invades": "Describes the location of a tumor or lesion within a patient's body and its impact on surrounding tissues. Specifically, this relationship is used to indicate whether a tumor or lesion has accumulated at the original site (accumulates), has metastasized to other sites, or has invaded surrounding tissues."

[0165] "Concurrent / accompanying": "Refers to the simultaneous occurrence of one symptom or disease alongside another symptom or disease in a patient, indicating a synchronous occurrence or mutual influence between the two. This relationship reflects the interaction between different systems in a patient's body and often requires a comprehensive medical consideration to fully understand the patient's health status."

[0166] "Location": "Describes the specific location or site of a physiological structure or lesion mentioned in the text. It is usually an indication of a body region, organ, tissue, or lesion that helps determine the specific location where the described condition occurs."

[0167] "Subclass": "Defined as a more specific, detailed classification or subclass of a symptom, disease, or treatment measure in the medical field. Here, the first element in the triple usually represents a specific manifestation or type of a symptom, disease, or treatment measure, the second element is'subclass', indicating a more specific classification of that manifestation or type, and the third element provides additional description or information about this relationship."

[0168] "Contains": "Indicates that one entity (usually the first element) contains or encompasses another entity (usually the third element). This relationship reflects a containment relationship between two entities, where the first entity includes or contains the third entity. This can represent various relationships in medical texts, such as a drug being included in a specific treatment regimen, a chemotherapy regimen including specific drugs, a pathological examination including specific disease diagnoses, etc."

[0169] "Diagnosis": Refers to a medical judgment in the medical field, where a doctor or medical professional makes a clear and formal determination or confirmation of a patient's condition through a series of examinations, tests, or surgeries. In each triple, the first element represents a specific condition or symptom, the second element is "diagnosis," and the third element represents the determined disease or pathological state. This relationship reflects the doctor's clear understanding of the patient's condition and medical diagnosis.

[0170] Based on the output results of the third stage, a total of 1 group of 'general relationships' is obtained. Based on the two types of entities of this group of relationships, the 'entity relationship area category prompt template' is constructed as:

[0171] Relationship definition:

[0172] [{

[0173] "Modification": Usually indicates the description of the subject's certain properties, state, characteristics, or changes of the object. This relationship helps us understand the specific connection between the subject and the object in more detail, providing more information about the events, states, or characteristics in the text.",..."Diagnosis": Refers to a medical judgment in the medical field, where a doctor or medical professional makes a clear and formal determination or confirmation of a patient's condition through a series of examinations, tests, or surgeries. In each triple, the first element represents a specific condition or symptom, the second element is "diagnosis," and the third element represents the determined disease or pathological state. This relationship reflects the doctor's clear understanding of the patient's condition and medical diagnosis.

[0174] }]

[0175] Medical text: Body temperature: 36.5.

[0176] Question: Determine which relationship in the relationship list the entity 'body temperature' and the entity '36.5' belong to in the above medical text. Directly output the relationship name and do not consider the direction of the relationship.

[0177] Notes:

[0178] 1. The entity 'body temperature' is of type 'vital sign' with a starting position of '0' and an ending position of '2' in the text.

[0179] 2. The entity '36.5' is of type 'value result' with a starting position of '3' and an ending position of '6' in the text.

[0180] Answer: Construct a 'Prompt template for distinguishing entity relationships' and pass this template data to the LLM inference service. This inference service is an API interface encapsulated from the Qwen1.5-14B model, which has been fine-tuned differently from the first stage. Deployed as a Docker service, this service loads the fine-tuned model upon startup. When the API interface receives the template data request, it tokenizes the input prompts to obtain token IDs; calculates the maximum length of the current batch (total_len) to create the input token tensor, ensuring the maximum length does not exceed the cache size; starts generation from the position of the shortest prompt in the current batch; the input token tensor is passed to the transformer model to calculate logits, resulting in logits of shape (batch_size, hidden_size) (the output of the last layer of the transformer); softmax + top_p sampling is used to obtain the currently predicted token, and the current position is updated to prepare for predicting the next token; the generated text is decoded. The inference results are formatted, and the above process is repeated for each group of 'general relationships', ultimately yielding the following output:

[0181] [

[0182] {

[0183] "text": "Body temperature: 36.5",

[0184] "entities": [

[0185] {

[0186] "entity": "body temperature",

[0187] "start_idx": 0,

[0188] "end_idx": 2,

[0189] "type": "Observation",

[0190] "id": "0"

[0191] },

[0192] {

[0193] "entity": "body temperature",

[0194] "start_idx": 0,

[0195] "end_idx": 2,

[0196] "type": "vital signs",

[0197] "id": "1"

[0198] },

[0199] {

[0200] "entity": "36.5",

[0201] "start_idx": 3,

[0202] "end_idx": 6,

[0203] "type": "Value-based result",

[0204] "id": "0"

[0205] }

[0206] ],

[0207] "spo_list": [

[0208] {

[0209] "Combined": true,

[0210] "predicate": "modify",

[0211] "subject": "body temperature",

[0212] "subject_type": "Vital signs",

[0213] "object": {

[0214] "@value": "36.5"

[0215] },

[0216] "object_type": {

[0217] "@value": "Value result"

[0218] },

[0219] "subject_start_idx": 0,

[0220] "subject_end_idx": 1,

[0221] "object_start_idx": 3,

[0222] "object_end_idx": 6

[0223] }

[0224] ]

[0225] }

[0226] 。

[0227] In this embodiment of the invention, for medical record text, a large-scale, high-quality unstructured text data of medical records is selected from the DWD data model library to ensure the diversity and integrity of the data, covering more than a thousand fields such as the patient's medical history, examination, testing, diagnostic basis, differential diagnosis, and surgical process, covering the entire life cycle of the patient; necessary preprocessing work such as cleaning, standardization, and feature extraction is performed on the data; the fine-tuning dataset is checked to ensure that the labels and context correspond correctly, avoid the inclusion of noise or erroneous information, and ensure that the fine-tuning dataset has enough examples involving situations that the model can answer and situations that it cannot answer, so as to avoid unbalanced data distribution; in terms of model selection, the llama2-13b-base model is selected for entity class annotation, and different datasets are built for distinguishable and undistinguished types for fine-tuning; the Qwen1.5-14B model is selected for entity relationship class annotation, and different datasets are built for distinguishable and undistinguished types for fine-tuning; dataset example: [ { "instruction": "Please determine whether the character fragment '###' in the text belongs to the 'general entity' class entity based on the given text: *****\n", "input": "", "output": ""}, { ...}].

[0228] In terms of fine-tuning strategy, LoRA updates are applied only to selected linear layers, while the remaining layers remain fixed. During training, a low-rank matrix product is added as a trainable parameter to the original weight matrix as a bypass, to simulate the change in parameters. The effect of gradually changing the original parameter matrix is ​​achieved by learning the low-rank matrix, rather than directly updating the original parameters. Specifically, assuming the pre-trained weights are ${w_0\ \epsilon \ \mathbb{R}^{d*k}}$, and the trainable parameters are ${\varDelta W\ =\ BA}$, where ${B\ \epsilon \ \mathbb{R}^{d*r}}$ and ${A\ \epsilon \ \mathbb{R}^{r*d}}$, during initialization, matrix ${A}$ is initialized using a Gaussian function, and matrix ${B}$ is initialized to zero, so that the bypass does not affect the original model before training begins, i.e., the change in parameters is 0. For the input ${x}$ of this weight, the output is ${h\ =\ W_0x+∆W\ x\ =W_0x+BAx}$. Fine-tuning training is performed on the target dataset. The gradient of the loss function with respect to the low-rank matrices W+ and W- is calculated through forward and backward propagation. The low-rank matrix is ​​updated according to the gradient, and then combined with the original model parameters according to the LoRA formula to obtain new parameter values. Thus, the model performance is improved without significantly increasing the number of model parameters.

[0229] Error detection mechanism: During fine-tuning, periodically evaluate model performance on an independent validation set, monitor key indicators such as accuracy, F1 score, etc.; according to the performance on the validation set, adjust the hyperparameters related to LoRA, such as the rank of the low-rank matrix, learning rate, regularization strength, etc., to optimize the performance of the model; Establish a strict output verification mechanism to ensure that the answers provided by the model are accurate and reliable; Introduce additional knowledge bases for fact checking.

[0230] As shown in Figure 2, the embodiment of the application also provides a device for constructing prompt automatic annotation of medical text based on large model, which comprises a medical text data acquisition module, a text data preprocessing module, a general entity information acquisition module, an entity type information acquisition module, an entity relationship general information acquisition module and an entity relationship category information acquisition module.

[0231] The medical text data acquisition module is used for filtering data tables and data elements based on massive medical text data, and extracting medical text data in the data elements;

[0232] The text data preprocessing module is used for preprocessing the medical text data to generate sentence samples;

[0233] The general entity information acquisition module is used for constructing entity non-zone category Prompt template based on the sentence sample, and capturing general entity information in the medical text;

[0234] The entity type information acquisition module is used for presetting entity types based on the captured general entity information, constructing entity zone category Prompt template, and capturing entity type information in the medical text;

[0235] The entity relationship general information acquisition module is used for presetting general entity relationship types based on the captured entity type information, constructing entity relationship non-zone category Prompt template, and capturing entity relationship general information in the medical text;

[0236] The entity relationship category information acquisition module is used for presetting entity relationship types based on the captured entity relationship general information, constructing entity relationship zone category Prompt template, and capturing entity relationship category information in the medical text.

[0237] As a further limitation of the technical scheme of the application, the text data preprocessing module comprises a sentence segmentation unit, an anomaly detection processing unit, a deduplication processing unit and a sentence sample generation unit;

[0238] The sentence segmentation unit is used for segmenting the extracted large text data into sentences according to the preset feature symbols;

[0239] Anomaly detection processing unit for detecting and handling missing values in the segmented sentences;

[0240] De-duplication processing unit for performing overall de-duplication processing on the sentences after missing value handling through similarity calculation;

[0241] Sentence sample generation unit for generating sentence samples from the de-duplicated sentences, wherein the similarity between the sentences in the sentence samples is zero.

[0242] In some embodiments, the general entity information acquisition module is specifically configured to acquire character combinations in the sentences based on the sentence samples, construct entity non-zone classification Prompt templates for the characters, call a large model inference service, select character combinations belonging to general entities, and construct a first-stage output result.

[0243] In some embodiments, the entity type information acquisition module is specifically configured to preset entity categories based on the first-stage output result, construct entity zone classification Prompt templates for the general entities, call a large model inference service, select character combinations belonging to the entity categories, and construct a second-stage output result, i.e., captured entity type information.

[0244] In some embodiments, the entity relationship general information acquisition module is specifically configured to preset entity relationships based on the second-stage output result, construct entity relationship non-zone classification Prompt templates between the categories of entities, call a large model inference service, select entity type combinations belonging to general relationships, and construct a third-stage output result.

[0245] In some embodiments, the entity relationship category information acquisition module is specifically configured to acquire entity type combinations belonging to general relationships in the sentences based on the third-stage output result, preset entity relationship types, construct entity relationship zone classification Prompt templates between the entity types, call an LLM inference service, select entity type combinations belonging to the entity relationship types, and construct a final output result.

[0246] The embodiment of the present application also provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus. The communication bus can be used for information transmission between the electronic device and the sensor. The processor can call logical instructions in the memory to execute the following method: step 1: screening a data table and data elements based on massive medical text data and extracting medical text data in the data elements; step 2: generating a sentence sample by preprocessing the medical text data; step 3: constructing an entity non-zone classification Prompt template based on the sentence sample, and capturing general entity information in the medical text; step 4: based on the captured general entity information, presetting an entity type, constructing an entity zone classification Prompt template, and capturing entity type information in the medical text; step 5: based on the captured entity type information, presetting a general entity relationship type, constructing an entity relationship non-zone classification Prompt template, and capturing general entity relationship information in the medical text; and step 6: based on the captured general entity relationship information, presetting an entity relationship type, constructing an entity relationship zone classification Prompt template, and capturing entity relationship category information in the medical text.

[0247] In addition, the logical instructions in the memory described above can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk and various program code storage media.

[0248] The embodiment of the present application provides a kind of non-transient computer readable storage medium, which stores computer instruction, the computer instruction makes computer execute the method provided by the method embodiment described above, for example, it includes: step 1: based on mass medical text data screening data table and data element, and extract medical text data in data element;Step 2: medical text data is preprocessed to generate sentence sample;Step 3: based on sentence sample, construct entity no-zone classification Prompt template, capture general entity information in medical text;Step 4: based on the general entity information captured, preset entity type, construct entity zone classification Prompt template, capture entity type information in medical text;Step 5: based on the entity type information captured, preset general entity relationship type, construct entity relationship no-zone classification Prompt template, capture general information of entity relationship in medical text;Step 6: based on the general information of entity relationship captured, preset entity relationship type, construct entity relationship zone classification Prompt template, capture entity relationship category information in medical text.

[0249] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0250] The embodiment of the present application provides a device for automatically labeling medical text based on large model prompt construction, which belongs to the same inventive concept as the method for automatically labeling medical text based on large model prompt construction described above. Details not described in detail in the embodiment of the device for automatically labeling medical text based on large model prompt construction can be referred to the embodiment of the method for automatically labeling medical text based on large model prompt construction described above.

[0251] The device for automatically labeling medical text based on large model prompt construction is the unit and algorithm steps of each example described in combination with the embodiments disclosed herein, which can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been described generally in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0252] Those skilled in the art can understand that the method of automatically labeling medical text based on a large model can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied in the form of a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0253] Although the present application is described in detail through reference to the attached drawings and preferred embodiments, it is not intended to limit the present application to the embodiments. It is understood that various modifications or replacements can be made to the embodiments of the present application by those skilled in the art without departing from the spirit and essence of the present application, and such modifications or replacements shall be within the scope of the present application. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and such changes or replacements shall be within the protection scope of the present application.

Claims

1. A method for constructing a prompt automatic labeling medical text based on a large model, characterized in that, The method comprises the following steps: Preprocessing medical text data to generate sentence samples; Based on the sentence sample, an entity non-zone classification Prompt template is constructed to capture general entity information in the medical text; Based on the captured general entity information, preset entity types, and constructed entity zone classification Prompt template, entity type information in the medical text is captured; Based on the captured entity type information, a general entity relationship type is preset, and an entity relationship non-zone classification Prompt template is constructed to capture general entity relationship information in the medical text; Based on the captured general entity relationship information, a general entity relationship type is preset, and an entity relationship zone classification Prompt template is constructed to capture entity relationship category information in the medical text.

2. The method for constructing a prompt for automatically labeling medical text based on a large model according to claim 1, characterized in that, The step of preprocessing medical text data to generate sentence samples includes: Based on massive medical text data, data tables and data elements are screened, and medical text data in the data elements is extracted.

3. The method for constructing a prompt for automatically labeling medical text based on a large model according to claim 2, characterized in that, The step of preprocessing text data to generate sentence samples includes: The extracted large text data is divided into sentences according to the preset characteristic symbols; Detecting and processing missing values in the divided sentences; The missing value processed sentences are subjected to overall deduplication processing through similarity calculation; The deduplicated sentences are generated as sentence samples; the similarity between the sentences in the sentence samples is zero.

4. The method for constructing a prompt for automatically labeling medical text based on a large model according to claim 3, characterized in that, The step of constructing an entity non-zone classification Prompt template based on the sentence sample to capture general entity information in the medical text includes: Based on the sentence sample, each character combination in the sentence is obtained; An entity non-zone classification Prompt template for each character is constructed; The first stage output result is constructed by calling a large model inference service and selecting character combinations belonging to general entities.

5. The method for constructing a prompt for automatically labeling medical text based on a large model according to claim 4, characterized in that, The step of constructing an entity zone classification Prompt template based on the captured general entity information and preset entity types to capture entity type information in the medical text includes: Based on the first stage output result, an entity category is preset; The entity zone classification Prompt template for each general entity is constructed, a large model inference service is called, character combinations belonging to the entity category are selected, and the second stage output result, i.e., the captured entity type information, is constructed.

6. The method for constructing a prompt for automatically labeling medical text based on a large model according to claim 5, characterized in that, The step of constructing an entity relationship non-zone classification Prompt template based on the captured entity type information and preset general entity relationship types to capture general entity relationship information in the medical text includes: Based on the second stage output result, an entity relationship is preset; The entity relationship non-zone classification Prompt template between entities of each category is constructed, a large model inference service is called, entity type combinations belonging to the general relationship are selected, and the third stage output result is constructed.

7. The method for constructing a prompt for automatically labeling medical text based on a large model according to claim 6, characterized in that, The step of constructing an entity relationship zone classification Prompt template based on the captured general entity relationship information and preset entity relationship types to capture entity relationship category information in the medical text includes: Based on the third stage output result, each entity type combination belonging to the general relationship in the sentence is obtained, and an entity relationship type is preset; The entity relationship zone classification prompt template between each entity type is constructed, the LLM inference service is called, each entity type combination belonging to the entity relationship type is selected, and a final output result is constructed.

8. An apparatus for constructing a prompt for automatically labeling medical text based on a large model, the apparatus comprising: The medical text data acquisition module, the text data preprocessing module, the general entity information acquisition module, the entity type information acquisition module, the entity relationship general information acquisition module, and the entity relationship category information acquisition module are included. The medical text data acquisition module is configured to filter data tables and data elements based on massive medical text data, and extract medical text data in the data elements. The text data preprocessing module is configured to preprocess the medical text data to generate sentence samples. The general entity information acquisition module is configured to construct an entity zone-free classification prompt template based on the sentence samples, and capture general entity information in the medical text. The entity type information acquisition module is configured to construct an entity zone classification prompt template based on the captured general entity information and preset entity types, and capture entity type information in the medical text. The entity relationship general information acquisition module is configured to construct an entity relationship zone-free classification prompt template based on the captured entity type information and preset general entity relationship types, and capture entity relationship general information in the medical text. The entity relationship category information acquisition module is configured to construct an entity relationship zone classification prompt template based on the captured entity relationship general information and preset entity relationship types, and capture entity relationship category information in the medical text.

9. An electronic device, comprising: The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores computer program instructions executable by the at least one processor. The computer program instructions are executed by the at least one processor to enable the at least one processor to perform the method of automatically labeling medical text based on a large model constructing prompt according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions, which cause the computer to perform the method of automatically labeling medical text based on a large model constructing prompt according to any one of claims 1 to 7.

Images