Medical record analysis method and device, electronic equipment and readable storage medium

Abstract

The invention provides a medical record analysis method and device, electronic equipment and a readable storage medium, and the method comprises the steps: enabling a planning agent to divide each process step of a medical record analysis process into a plurality of minimum sub-tasks according to prompt words, and enabling the planning agent to distribute a corresponding task execution agent for each sub-task, the intelligent agent work assembly line executes the medical record analysis process, not only is the semantics of the complex medical context adapted, but also the efficiency and the accuracy of medical record analysis are improved, and based on the structure of the intelligent agent, the large language model module is a vertical domain model with professional domain task execution capability, and the large language model module is a large-language model with professional domain task execution capability. And meanwhile, the input and output of each node agent in the agent work assembly line are transparent, readable and interpretable, so that the stability of the output quality can be ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical record analysis, and in particular relates to a medical record analysis method, device, electronic equipment and readable storage medium. Background Art

[0002] In the process of health management services and hospitals seeing patients for the first time, it is necessary to comprehensively analyze the patient's medical records, such as symptoms, tests, treatments, and treatment effects, in order to accurately determine the patient's current condition and develop an appropriate treatment plan. Faced with the actual situation that health management institutions or outpatient doctors need to review a large number of medical records, manual reading efficiency is limited, which becomes a bottleneck for processing. At present, there is an urgent need to use computer tools to improve the efficiency of medical record processing and to provide intelligent evaluation of whether the current treatment plan meets the recommendations of clinical guidelines.

[0003] In the prior art, medical records are processed mainly in the following ways: 1. Use grammatical templates to extract key information from medical records. This method has low adaptability and is difficult to adapt to the semantics of complex medical contexts. 2. Use NLP and general large model technology to summarize medical records (single medical record) or extract information. Although it solves the semantic recognition of complex medical contexts, it does not have logical judgment functions. After fine-tuning the large language model for the medical field, there are still problems with unstable and uncontrollable output quality: because the decision-making process of the large model is not transparent and has defects such as hallucinations and biases, the stability of the output quality cannot be guaranteed and cannot be controlled in the medical field that requires a high degree of interpretability. Summary of the invention

[0004] Based on this, in order to solve the above technical problems, a medical record analysis method, device, electronic device and readable storage medium are provided.

[0005] The technical solution adopted by the present invention is as follows: As a first aspect of the present invention, a medical record analysis method is provided, which analyzes a target medical record to be analyzed through a preset medical record analysis process, and is characterized by comprising: S101, generating prompt words according to the content of the target medical record, the content of the expected output and the requirements, and inputting them into the planning agent; S102, the planning agent divides each process step of the medical record analysis process into a plurality of minimum subtasks according to the prompt words; S103, the planning agent establishes and allocates a corresponding task execution agent for each of the subtasks to form an agent workflow for executing the medical record analysis process, and at the same time, sends prompt word content corresponding to the corresponding subtask to each task execution agent; Each agent includes: A general large language model module is used to output results according to prompt words and reflect on the output results according to reflection instructions; A memory module, used for storing the input and output records of the general large language model module and the corresponding reflection results in text form; The knowledge module is used to store the knowledge information required for the tasks undertaken by the agent; An instruction module, used to generate prompt words according to information input by a user or other intelligent agent, information in a memory module and knowledge information in a knowledge module, and input a reflection instruction into the general large language model module when a reflection instruction is built in, so that the general large language model module outputs a result and then reflects on the result; S104, each task execution agent generates a prompt word through its instruction module according to the prompt word content sent by the planning agent, and inputs the prompt word into its general large language model module to execute the corresponding subtask; Among them, one of the multiple task execution agents for executing multiple minimum subtasks of a process step is used to summarize the execution results of the remaining task execution agents, obtain the execution result of the process step, and send it to the planning agent; S105: The planning agent outputs the execution results of the process steps.

[0006] As a second aspect of the present invention, a medical record analysis device is provided, which analyzes a target medical record to be analyzed through a preset medical record analysis process, and is characterized by comprising: The first unit is used for S101, generating prompt words according to the content of the target medical record, the content of the expected output and the requirements, and inputting them into the planning agent; The second unit is used for S102, wherein the planning agent divides each process step of the medical record analysis process into a plurality of minimum subtasks according to the prompt word; The third unit is used for S103, wherein the planning agent establishes and allocates a corresponding task execution agent for each of the subtasks to form an agent workflow for executing the medical record analysis process, and at the same time, sends the prompt word content corresponding to the corresponding subtask to each task execution agent; Each agent includes: A general large language model module is used to output results according to prompt words and reflect on the output results according to reflection instructions; A memory module, used for storing the input and output records of the general large language model module and the corresponding reflection results in text form; The knowledge module is used to store the knowledge information required for the tasks undertaken by the agent; An instruction module, used to generate prompt words according to information input by a user or other intelligent agent, information in a memory module and knowledge information in a knowledge module, and input a reflection instruction into the general large language model module when a reflection instruction is built in, so that the general large language model module outputs a result and then reflects on the result; The fourth unit is used for S104, each task execution agent generates a prompt word through its instruction module according to the prompt word content sent by the planning agent, and inputs the prompt word into its general large language model module to execute the corresponding subtask; Among them, one of the multiple task execution agents for executing multiple minimum subtasks of a process step is used to summarize the execution results of the remaining task execution agents, obtain the execution result of the process step, and send it to the planning agent; The fifth unit is used for S105, wherein the planning agent outputs the execution results of the process steps.

[0007] As a third aspect of the present invention, an electronic device is provided, comprising a storage module, wherein the storage module comprises instructions loaded and executed by a processor, wherein when the instructions are executed, the processor executes a medical record analysis method according to the first aspect.

[0008] As a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores one or more programs. When the one or more programs are executed by a processor, a medical record analysis method of the first aspect is implemented.

[0009] The present invention divides each process step of the medical record analysis process into multiple minimum subtasks according to the prompt words by the planning agent, and assigns a corresponding task execution agent to each subtask, thereby forming an agent workflow. The medical record analysis process is executed by the agent workflow, which not only adapts to the semantics of complex medical contexts, but also improves the efficiency and accuracy of medical record analysis.

[0010] Among them, the intelligent agent can make the large language model module output the execution results of the tasks it undertakes based on external information, the input and output records of the general large language model module in the memory module and the corresponding reflection results, and the knowledge information in the knowledge module, making the large language model module a vertical domain model with the ability to execute tasks in professional fields, thereby improving the cost-effectiveness.

[0011] At the same time, unlike the prior art of processing medical records through large language models, the input and output of each node (agent) in the intelligent agent workflow of the present invention are transparent, readable and interpretable, thereby ensuring the stability of the output quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments: Figure 1 A flowchart of a medical record analysis method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a medical record analysis device provided by an embodiment of the present invention; Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention; Figure 4 is a schematic diagram of the structure of an intelligent agent according to an embodiment of the present invention; Figure 5 A schematic diagram of an intelligent agent workflow for executing a medical record analysis process according to an embodiment of the present invention; Figure 6 is an exemplary schematic diagram of a knowledge tree according to an embodiment of the present invention; Figure 7 A knowledge retrieval principle diagram of a knowledge tree according to an embodiment of the present invention; Figure 8 This is a schematic diagram of pruning and closing irrelevant branches according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will illustrate the implementation methods of the present invention in conjunction with the drawings in the specification. It should be noted that the implementation methods involved in this specification are not exhaustive and do not represent the only implementation methods of the present invention. The following corresponding embodiments are only for the purpose of clearly illustrating the invention content of the patent of the present invention, and are not intended to limit its implementation methods. For ordinary technicians in this field, different forms of changes and modifications can be made on the basis of the description of this embodiment. All obvious changes or modifications that belong to the technical concept and invention content of the present invention are also within the scope of protection of the present invention.

[0014] The embodiment of the present application is used to analyze the target medical records to be analyzed through a preset medical record analysis process, and the medical record analysis process includes five process steps in sequence: medical record data extraction, key indicator calculation, actual treatment plan extraction, guideline knowledge base treatment plan matching, and treatment plan and knowledge base matching degree judgment.

[0015] The present application embodiment relates to an intelligent agent, such as Figure 4 As shown, it includes: The general large language model module 101 is used to output the result according to the prompt word and reflect on the output result according to the reflection instruction. The general large language model module 101 has the basic capabilities of the general large language model and can be connected to any large model with an MMLU evaluation score of 75 points or above.

[0016] The memory module 102 is used to save the input and output records of the general large language model module 101 and the corresponding reflection results in text form, and to sort the input and output records after receiving the sorting instruction. The specific process is as follows: 1. When the cumulative amount of new input and output records reaches the first threshold, all new input and output records and old input and output records are summarized to form one record as the old input and output record. At this time, the number of old input and output records is increased by 1, and at the same time, all new input and output records are deleted; 2. When the number of old input and output records reaches a second threshold, the earliest preset forgotten number of records among all old input and output records are forgotten.

[0017] Among them, the intelligent agents that undertake different tasks adapt to different tasks by setting different first thresholds, second thresholds and forgetting amounts. The forgetting amount can be set to a specific amount or to a proportion of all old input and output records.

[0018] Assume that the memory module originally has 10 old records and 0 new records, the first threshold is 5, the second threshold is 100, and the number of forgotten records is 20. When the cumulative number of new records reaches 5, 10+5 records are summarized to form one record as the old record. At this time, the number of old records is increased by 1, a total of 11. At the same time, 5 new records are deleted; when the number of old records reaches 100, the earliest 20 records among the 100 old records are forgotten.

[0019] The knowledge module 103 is used to store the knowledge information required for the task undertaken by the agent. For example, for an agent that undertakes the task of extracting information, if the extracted content requires further processing, the knowledge information in the knowledge module can be the scoring standard or judgment condition in the industry. Of course, the knowledge module 103 may also be empty. The knowledge information can be preset when the agent is created according to the different tasks undertaken by the agent, and can be modified later.

[0020] The instruction module 104 is used to generate prompt words and input them into the general large language model module based on the external information input by the user or other intelligent agents, the information in the memory module 102 and the knowledge information in the knowledge module 103; when a reflection instruction is built in, a reflection instruction is input into the general large language model module so that it outputs a result and then reflects on the result; and when a sorting instruction is built in, the sorting instruction is input into the memory module 102 after the general large language model module outputs the result.

[0021] Tool module 105 is a description of the functions and calling methods of the function tools provided by the previous agent in the workflow to the current agent, as well as a correspondence table between the function name and the function entity calling entry. It is a repository that specifies the range of callable functions for different agents by applying the workflow based on the toolcall function of the existing large model.

[0022] Based on the above medical record analysis process and intelligent agent structure, Figure 1 As shown, this embodiment provides a medical record analysis method, and the specific process is as follows: S101. Generate prompt words according to the content of the target medical record, the expected output content and requirements, and input them into the planning agent.

[0023] S102, the planning agent divides each process step of the medical record analysis process into multiple minimum subtasks according to the prompt words.

[0024] S103, the planning agent creates and allocates a corresponding task execution agent for each subtask, forming an agent workflow for executing the above-mentioned medical record analysis process, and at the same time, sends the prompt word content corresponding to the corresponding subtask to each task execution agent.

[0025] S104, each task execution agent generates a prompt word through its instruction module according to the prompt word content sent by the planning agent, and inputs the prompt word into its general large language model module to execute the corresponding subtask.

[0026] Among them, one of the multiple task execution agents used to execute multiple minimum subtasks of a process step is used to summarize the execution results of the remaining task execution agents, obtain the execution results of the process step, and send them to the planning agent.

[0027] In the above-mentioned intelligent agent workflow, the subsequent task execution agent may need to use the output of the previous task execution agent as input. The subsequent task execution agent can directly obtain the output from the corresponding previous task execution agent as input. Of course, when the subsequent task execution agent corresponds to different process steps from the previous task execution agent, the subsequent task execution agent can also obtain corresponding information from the planning agent. For example, the task execution agent of the treatment plan and knowledge base matching judgment process step can obtain the guide knowledge base treatment plan from the planning agent.

[0028] S105. The planning agent outputs the execution results of the process steps, such as outputting the execution results on a display screen.

[0029] like Figure 5As shown in the figure, taking the medical record data extraction process step as an example, the expected output content and requirements can be understood as the expected extraction content and extraction requirements. The planning agent divides it into 5 minimum subtasks according to the prompt words: basic information extraction, admission and discharge time extraction, treatment drug extraction, treatment drug use time extraction and summary format arrangement. The 5 minimum subtasks are respectively responsible for 5 task execution agents, among which summary format arrangement refers to summarizing and formatting the extracted basic information, admission and discharge time, treatment drugs and treatment drug use time. Similarly, the other 4 process steps can also be divided into multiple minimum subtasks, which will not be repeated here.

[0030] It is understandable that the five process steps can be handled by one planning agent or by five planning agents respectively.

[0031] It should be pointed out that if an agent is asked to perform the entire process steps or non-minimum subtasks, its large language model will be difficult to give accurate results at one time due to the complexity of the task. After being divided into minimum subtasks, the large language model can focus on performing a single subtask. The complexity of each subtask is greatly reduced, which reduces the burden on the model and allows it to focus more on the details of each specific task, making the large language model more efficient and accurate.

[0032] After the agent work pipeline is initially formed, it needs to be optimized and trained. The specific process is as follows. Figure 5 : S11. Take multiple medical record samples (such as 100) as target medical records and test the intelligent agent workflow through steps S101-S105.

[0033] S12. Create a judgment agent to determine whether the execution result of each process step is different from the corresponding quality control reference value. If so, the difference is output to the planning agent. Otherwise, the confusion without difference is output to the planning agent.

[0034] Specifically, it is necessary to construct judgment prompt words with the quality control reference value and the above execution results and input them into the judgment agent (the large model module of the agent needs to select an open source large model, because the closed source model cannot obtain parameter information during the model operation process). The "no difference" perplexity output by the large model is used to judge the effect of the agent's workflow (the lower the perplexity, the better the solution).

[0035] Among them, perplexity is an important indicator to measure the performance of language models, especially in the field of natural language processing (NLP). The lower the perplexity, the stronger the predictive ability of the model, which means that the model has a better understanding of the language. The calculation of perplexity is usually based on the probability distribution of the model. In this application, by requiring the model to output a fixed target string, the perplexity is used to evaluate the credibility of the model's output target string, and the output of the large language model is converted into a quantitative indicator (for example, originally only classification judgments such as [yes, no] can be output, now [yes, no] can be output, plus a perplexity value, which quantifies the credibility of the model's output target string in the judgment context).

[0036] The above-mentioned confusion of no difference represents how sure or confident the judging agent is that there is no difference between the execution result and the corresponding quality control reference value. It is a scoring value, and the lower the value, the greater the sureness or confidence.

[0037] The steps to calculate the perplexity are as follows: 1. Calculate the probability of each word: For a given test data set, the model generates a probability distribution for each word. Assume that the test data set contains N words, denoted as , , , .

[0038] The model is for each word The probability of generation is recorded as .

[0039] 2. Calculate the perplexity: The calculation formula of perplexity PP is:

[0040] in, Usually the logarithm is based on the natural logarithm ln.

[0041] The judgment agent can be shared by all process steps, or it can be divided into different agents to execute from the perspective of parallelism and clear division of labor. In practical applications, in order to increase the degree of parallel operation and improve operating efficiency, multiple agents can be responsible for quality control.

[0042] In order to provide more stable and reliable judgment results, reduce the random influence of a single model, and maintain a high level of vigilance when problems are discovered, this application has made the following improvements to the perplexity evaluation: 1. Multi-agent judgment mechanism to reduce the randomness of single model judgment: An odd number of judging agents (usually 5 or 7) are used to give their opinions simultaneously to simulate the expert evaluation process. Each judging agent independently calculates its confusion about the target string ("no difference" or "difference"), and the judgment results of multiple judging agents are used to determine the final judgment through a voting mechanism.

[0043] 2. Based on the Logits Fusion principle, the perplexity output by different judgment agents is merged to smooth the perplexity evaluation results: If the judgment results of an odd number of judgment agents are the same, then the geometric mean of the perplexity output by the odd number of judgment agents is combined using formula (1): (1) Where n is the number of judging agents, PP i is the confusion value of the i-th agent; If some of the judgment results of an odd number of judgment agents are the same and some are different, then the maximum perplexity among all the judgment agents whose judgment results are the same is taken as the perplexity of all the judgment agents whose judgment results are different. , and then combine the perplexity output by the odd number of judgment agents through formula (1); Among them, m is the number of judging agents whose judgment results are indifferent.

[0044] The reason for this design is that "difference" indicates a problem, and the most uncertain (highest confusion) judgment should be selected as the final result, thereby increasing the system's sensitivity to potential problems.

[0045] The following is an example of perplexity calculation: 1. Define the target string The target string is "no difference".

[0046] 2. Get the model's predicted probability for the target string Suppose the model predicts the following probability for the string "no difference": No: 0.7 Yes: 0.8 Difference: 0.6 Different: 0.75 3. Calculate the log probability of each character Take the natural logarithm ln of the probability of each character: ln(0.7) ≈ -0.357 ln(0.8) ≈ -0.223 ln(0.6) ≈ -0.511 ln(0.75) ≈ -0.287 4. Calculate the total log probability Add up the log probabilities of all characters:

[0047]

[0048] 5. Calculate the perplexity The calculation formula for perplexity is:

[0049] Where N is the length of the target string, so in this example, N=4.

[0050]

[0051] 6. Comprehensive judgment by multiple agents Suppose we use 3 judgment agents, and each agent independently calculates the perplexity as follows: The output of agent 1 is "no difference": PP1=1.411 Agent 2 outputs "no difference": PP2=1.325 The output of Agent 3 is judged to be "different": PP3=1.468 7. Logits Fusion merge confusion Since it is meaningless to take the perplexity of the difference when some agents have different outputs, the perplexity of the judgment that the output is "different" is first represented by the maximum perplexity of the judgments that the output is "not different":

[0052] Then use the geometric mean to fuse the results:

[0053] 8. Explaining perplexity For the judgment of "no difference", the final perplexity is 1.382, which represents the average judgment result of the three agents. The lower the perplexity, the more confident the model is in predicting the target string.

[0054] S13. The planning agent adds, deletes or modifies the task execution agent according to the content of the difference until the perplexity without difference meets the preset conditions. The optimization is completed. For example, when the reduction of perplexity before and after different iterations is less than 0.1% or the absolute value of the difference between the perplexity before and after is less than 50 (or other manually set thresholds), the optimization is completed.

[0055] Among them, the impact of adding, deleting or modifying a single-task execution agent on the execution result is evaluated in parallel (keeping other task execution agents unchanged, and multi-arm A / B testing), and finally the combination effect of the optimal task execution agent is tested.

[0056] When the planning agent needs to modify a task execution agent, on the one hand, the planning agent adjusts the parameters of the general large language model module of the task execution agent, and can copy and create an agent with the same function, but adjust the parameters, or set up multiple agents corresponding to the same subtask and with different parameters from the beginning, and finally make a choice based on the effect; on the other hand, the planning agent forwards the above-mentioned differences to the task execution agent, and the task execution agent reflects on the output task execution results based on the differences, and stores the reflection results in the memory module. When the subtask is executed again, the reflection results are input into its general large language model module as part of the prompt word.

[0057] Reflection results such as: "When I was processing....., I mistakenly put..... I should put....” The current general reflection technology has the defect of lacking creative behavior to escape the local minimum. Therefore, this application adds creative behavior during reflection to escape the local minimum, as follows: 1. Identify and exclude invalid reflection results: Calculate the similarity between the results of the previous and next rounds of reflection (the calculation method can be compared by the count of corresponding tokens, or semantic similarity). For example, the similarity between the results of the previous and next rounds of reflection sr0 and sr1 is 0.9, which exceeds the set artificial threshold. In this case, the new reflection will not be included in the memory module as an invalid reflection, so as to avoid the large model falling into a repetitive mode and entering an infinite loop, thus being unable to escape the local minimum.

[0058] 2. After multiple cycles of identifying similar reflections, inspiration is added to increase the randomness of the model output and expand the search space for the optimal value. Among them, inspiration is generated by a dedicated inspiration agent (the prompt word input to it is usually "You are an inspiration generator (coach), you generate innovative ideas based on the given tasks and reflection results, but do not give specific answers") based on the principle of few-shot examples. For example, in the drug extraction task, inspiration may include inspiring suggestions such as "consider different dosage forms of drugs" and "distinguish between prescription drugs and over-the-counter drugs". These inspirations will serve as additional contextual information to help the main model jump out of the local optimal solution.

[0059] Of course, if the optimization is not completed within 72 hours, manual intervention will be required for adjustment.

[0060] By organizing the intelligent agent workflow and comparing the execution results of each process step, the stability, transparency and controllability of the large language model output are improved.

[0061] The following is an example of an actual medical record to illustrate this step. If the input medical record sample is a picture or a PDF, Word or other document, it needs to be converted into a text format using OCR or PDF, Word tools. The specific process is as follows: Step 1: Convert the medical record sample into text format. The specific text after conversion is as follows: Discharge Record Name: *** Gender: Female Age: 65 Occupation: Retired Department: Thyroid and Breast Surgery Ward 2 Ward: (Ward) Thyroid and Breast Surgery Bed number: +10 Bed hospitalization medical record number: *** Date of admission: April 8, 2023 Date of discharge: April 27, 2023 Number of days in hospital: 19 days Admission and transfer department: Thyroid and Breast Surgery Ward II (Houhu) Admission diagnosis Discharge diagnosis Cervical lymphadenopathy 1. Diffuse large B-cell lymphoma (stage ⅡA) (involving the neck lymph nodes, ABC type) (IPI2 points, low- to medium-risk group) 2. Chemotherapy 3. Bone marrow suppression after chemotherapy Admission and treatment history: Patient ***, female, 65 years old. She had a lump on the right side of her neck for 3 months. Past medical history: denied hypertension, diabetes Disease, denied food and drug allergy history. Denied history of infectious diseases such as tuberculosis and hepatitis. Partial thyroidectomy in 1982: History of cesarean section. Physical examination: soft neck, trachea in the middle, a mass of about 8*6cm in size can be felt on the right side of the neck, medium in texture. Outpatient information: On March 4, 2023, an ultrasound-guided puncture biopsy of the right submandibular mass was performed at Wufan Central Hospital. Pathological examination: (puncture of the right submandibular mass) was considered to be reactive m lymphadenitis.

[0062] After admission, complete relevant auxiliary examinations, 2023-04-10 Myocardial enzyme spectrum four items α-hydroxybutyrate dehydrogenase 243U / L↑(72~182), acid dehydrogenase (rate method) 297U / L↑(103~227). 2023-04-10 Biochemical examination 2 *Total protein (double hyperplasia endpoint method) 52.8g / L↓(65~85), *Albumin (bromocresol green method) 38.2g / L↓(40~55), *Apolipoprotein B (immunoturbidimetry) 1.07g / L↑(0.69~1.05), Apolipoprotein E 88.8mg / L↑(30~50), Free fatty acid 0.62mmol / mouth (0.172~0.586), 2023-04-10 Complete blood count + five classifications: white blood cell count 4.43x10^9 / L15~9.5), hemoglobin concentration 116g / L (115~150), platelet count 186x10^9 / L (125~350), neutrophil count 2.35x10^9 / L (1.8~6.3): 2023-04-22 Plasma D-dimer determination (D-Dimer) D-dimer 01.815μg / mL↑(0~0.5); high-sensitivity C-reactive protein, serum troponin I determination (chemiluminescence method), coagulation test 2, erythrocyte sedimentation rate, coagulation routine, eight items of pre-transfusion examination, and hepatitis B DNA quantitative determination showed no obvious abnormalities. 2023-04-26, complete blood count + five categories: white blood cell count 10.17x10^9 / L↑(3.5~9.5), hemoglobin concentration 108g / L↓(115~150), platelet count 240x10^9 / L(125~350), neutrophil count 7.68x10^9 / L↑(L8~6.3): 2023-04-26, five renal function tests: * uric acid (uric acid Enzyme colorimetric method) 119μmol / L↓(155~357), creatinine (enzyme method) 31.6μmol / L↓(41~81): 2023-04-26, eleven liver function tests *Albumin (bromocresol method) 43.2g / L↓(40~55), albumin-globulin ratio 1.1↓(1.2~2.4), *Total protein (bidimensional end-point method) 62.3g / mouth↓(65~85): No obvious abnormalities were found in C-reactive protein and B2 microglobulin.

[0063] 2023-04-08DR: No obvious abnormalities were found in the heart, lungs and diaphragm.

[0064] 2023-04-08ECG: 1. Sinus rhythm, 2. Normal range ECG.

[0065] 2023-04-10 Color Doppler ultrasound of the thyroid and cervical lymph nodes: cystic nodule with crystals in the left lobe of the thyroid gland, considered to be an alveolar cyst, I-RADS2 category, a huge solid mass on the right side of the neck, and no obvious enlarged lymph nodes on the left side of the neck.

[0066] 2023-04-11 CT scan of soft tissue in the neck: 1. A huge mass in the right paracervical space. Considering the history, the source of the mass is lymph node or thyroid. (***Hospital) Discharge Record Name: *** Gender: Female Age: 65 Occupation: Retired Department: Thyroid and Breast Surgery Ward II Ward: (Ward) Thyroid and Breast Surgery Bed No.: +10 Bed Hospitalization Medical Record No.: ***Original space occupation. 2. Partial filling defect of right brachiocephalic vein, consider embolism; part of right internal jugular vein is not visualized, please combine clinical diagnosis.

[0067] 2023-04-19 Pathology: (Neck) non-Hodgkin diffuse aggressive B-cell lymphoma, consistent with diffuse large B-cell lymphoma-not specified (activated B-cell origin), immunohistochemistry: tumor cells: CD20, PAX-5, BCL-2, MUH-1 (+), C-MYC (+, 30%): P53 (mutant), Ki-67Li85%; the rest (CD6, CD1O, BCL-6, CyclinD1, CD21, CD23, ED3, CD43, PCK, S-100) are all (-), CD30 (-), molecular detection: EBER (-), 2023-04-21PET-CT: 1. A soft tissue density mass in the right paracervical space with abnormally increased metabolism is considered to be a sign of lymphoma infiltration.

[0068] 2. Small lymph nodes in zone IV of the left neck, with slightly increased metabolism, lymphoma infiltration is considered. 3. Multiple small lymph nodes in the remaining bilateral neck, bilateral axillary area, and bilateral inguinal area, with low metabolism, considered nonspecific changes, and follow-up examination is recommended. 4. Pure GG nodules in the anterior basal segment of the right lower lobe, with low metabolism, thin-layer CT follow-up examination is recommended. 5. Cystic and solid masses in the left adnexa, with low metabolism, are mostly considered benign lesions, and combined with specialist examinations: right adnexa teratoma. 6. No obvious signs of malignant tumor lesions were found in other parts of the body. 7. Brain atrophy; leukoaraiosis. 8. Left maxillary sinusitis; bilateral inferior turbinate hypertrophy; bilateral tonsil inflammation or physiological uptake. 9. Bilateral breast hyperplasia. 10. A few fibrous lesions in both lower lungs. 11. Multiple liver cysts; gallstones: a little gas accumulation in the lower part of the common bile duct: increased metabolism in the rectum and anorectal area, consider inflammation or physiological uptake. 12. Osteoporosis: multiple vertebral degeneration: left iliac bone island; bilateral low iliac joint degeneration; inflammatory lesions or physiological uptake of the muscles next to the left hip joint.

[0069] 2023-04-23 Cardiac ultrasound, cardiac function, and ventricular wall motion analysis: slight mitral valve regurgitation, slight tricuspid valve regurgitation, and pulmonary hypertension (mild).

[0070] 2023-4-24 Bone marrow cytology: No granulocytes were found in the bone marrow, the bone marrow showed trilineage hyperplasia, and the proportion of neutrophils and segmented granulocytes increased.

[0071] Immunophenotyping: The proportions of various cell groups in the bone marrow are roughly normal, the granulocytes are differentiated and mature, there is no obvious proliferation of primitive cell groups, and there is no increase in the proportion of B lymphocytes and abnormal phenotype. The immune phenotype is not characteristic.

[0072] 2023-04-25,DR: No substantial lesions were found in both lungs. Scoliosis was found.

[0073] The patient was hospitalized in the Department of Thyroid and Breast Surgery of our hospital due to a huge mass on the right side of the neck. After preoperative examination, a right cervical lymph node biopsy (right side) was performed on 2023-04-11. After surgery, anti-inflammatory, hemostatic, symptomatic supportive treatment and regular dressing changes were given. Postoperative pathology showed diffuse large B-cell lymphoma. The patient was transferred to the Department of Hematology on 2023-4-20 and was diagnosed with diffuse large B-cell lymphoma stage ⅡA. On 2023-04-25, a left upper PICC catheterization was performed, and no discomfort was reported after the operation. On 2023-4-22, CHOP chemotherapy was started (DEX13mgd1-5, CTX1000mgd5, VDS3mgd5, doxorubicin liposome 30mgd5). The process was smooth and the neck mass was reduced after treatment.

[0074] He is now being discharged from the hospital.

[0075] Discharge: The patient is in good general condition, with no complaints of special discomfort. The stitches on the neck wound have been removed, and there is no wet blood exudation. Physical examination: normal vital signs, 1 / nail of the neck wound healed, no redness, swelling, no bleeding, tenderness. Clear breath sounds in both lungs, normal heart sounds. Soft abdomen, no tenderness or rebound pain, liver and spleen not affected, normal bowel sounds.

[0076] Discharge instructions: 1. Avoid strenuous exercise and keep the wound dry; 2. Come to the hospital for the next course of treatment on 2023-5-15. If you feel unwell, please see a doctor in time.

[0077] Department outpatient arrangements: *** Chief Physician: Fujing Road Tuesday morning, Thursday morning, Houhu Friday morning: *** Deputy Chief Physician Doctor: ***Saturday morning, Houhu Thursday morning.

[0078] Step 1.1: This step is limited to process construction. The user input content is manually analyzed by experts to form quality control reference values ​​under the process framework of "medical record data extraction-key indicator calculation-actual treatment plan extraction-guideline knowledge base treatment plan matching-treatment plan and knowledge base matching judgment".

[0079] The quality control reference values ​​after manual analysis by experts are as follows: 1) Reference values ​​for medical record data extraction "Date of admission": "2023-04-08", "Discharge Date": "2023-04-27", "Diagnosis upon admission": "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "Start treatment date": "2023-04-22", "Treatment plan": "The patient received CHOP chemotherapy, including: DEX 13mgd1-5, CTX 1000mgd5, VDS 3mgd5, doxorubicin liposome 30mgd5.", "Effect of treatment": "After CHOP chemotherapy, the neck mass has shrunk, but the specific effect does not mention specific indicators such as CR, PR, etc.", "Disease outcome": "Disease condition has improved." 2) Reference values ​​for key indicator calculations "IPI score conclusion": "April 08, 2023, IPI2 points, low-medium risk group", "IPI scoring basis": "Medical record extraction results", "IPI Scoring Process": "Medical Record Extract" 3) Reference values ​​extracted from actual treatment plans "Start treatment date": "2023-04-22", "Diagnosis upon admission": "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "Actual treatment plan": "CHOP chemotherapy, including: DEX 13mgd1-5, CTX 1000mgd5, VDS 3mgd5, doxorubicin liposome 30mgd5." 4) Guideline knowledge base treatment plan matching results "Start treatment date": "2023-04-22", "Diagnosis upon admission": "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "Patient type for this treatment": "First-time patients", "Recommended treatment options": "[R-CHOP21, R-CDOP, R-CHOEP]", "Reason or basis for recommendation": "The patient type in this treatment is a treatment-naive patient, aged 65 years old, with an IPI score of 2 points, no heart failure, and currently a treatment-naive patient, and does not meet the conditions for transplantation. According to the first-line treatment regimen for diffuse large B-cell lymphoma in the "Reference Materials", the [R-CHOP21, R-CDOP, R-CHOEP] regimen in the first-line treatment is recommended." 5) Judgment results of the matching degree between treatment plan and knowledge base "Recent patient treatment": [ ("2023-04-22", "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "CHOP chemotherapy, specifically including: DEX13mgd1-5, CTX1000mgd5, VDS3mgd5, doxorubicin liposome 30mgd5.") ], "Is it a recommended solution": ["Not applicable"], "Recommended Ratio": "0 / 1", "Subsequent recommended regimen": "[R-CHOP21]", "Reasoning for the recommended regimen": "The patient is a treatment-naive patient, aged 65 years, with an IPI score of 2 points. There is a description of heart failure in the medical record, so the patient does not meet the transplant conditions. According to the first-line treatment regimen for diffuse large B-cell lymphoma in the "Reference Materials", considering that the patient is a treatment-naive patient and does not meet the transplant conditions, the R-CHOP21 regimen in the first-line treatment is recommended. This regimen includes rituximab and CHOP chemotherapy (cyclophosphamide, doxorubicin, vincristine, prednisone). The addition of rituximab can improve the treatment effect, especially for the treatment of B-cell lymphoma. Therefore, it is recommended to use the R-CHOP21 regimen in the future to optimize the treatment effect." Step 2: Use the original medical records from step 1 and the manual quality control reference values ​​from step 1.1 to automatically arrange and test the process (taking medical record data extraction as an example): First, the extraction of medical record data is taken as the goal, and the medical record text in step 1 is taken as the original medical record, and the required output extraction content and the extraction requirement list together constitute the prompt word input planning agent.

[0080] Examples of planning agent prompts are: [ { "role": "system", "content": "You are a doctor of medicine who is good at organizing and planning interns to extract medical record information", }, { "role": "user", "content": "This is the original medical record" + the original medical record text of step 1, }, {"role": "user", "content": "This is the target information that needs to be extracted" + the manual quality control standard in step 1.1}, {"role": "user", "content": "This is the information you extracted before" + historical output memory (initialized to empty, extracted from the memory module)}, {"role": "user", "content": "Please parse and output in JSON object format how to obtain the information elements in the target extraction. The following is an example: Original data: {" Discharge Record Name: *** Gender: Female Age: 65 Occupation: Retired Department: Thyroid and Breast Surgery Ward 2 Ward: (Ward) Thyroid and Breast Surgery Bed number: +10 Bed hospitalization medical record number: *** Date of admission: April 8, 2023 Date of discharge: April 27, 2023 Number of days in hospital: 19 days Admission and transfer department: Thyroid and Breast Surgery Ward II (Houhu) Admission diagnosis Discharge diagnosis Cervical lymphadenopathy 1. Diffuse large B-cell lymphoma (stage ⅡA) (involving the neck lymph nodes, ABC type) (IPI2 points, low- to medium-risk group) 2. Chemotherapy 3. Bone marrow suppression after chemotherapy } Target information: "Age": "65 years old", "Gender": "Female", "Main diagnosis": "Diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI 2 points, low-intermediate risk group)", "Date of diagnosis": "2023-04-18", } Output content: {"Age":{"Required":"Age of the patient","Precondition":"None"},"Gender":{"Required":"Gender of the patient","Precondition":" None"},"Main diagnosis":{"Requirement":"The patient's main diagnosis needs to be extracted","Precondition":"None"}}, ] After that, the agent construction program will construct an information extraction agent and an information aggregation agent for extracting the patient's age, gender, and main diagnosis through the instance of the new agent class and the output of the planning agent {"age":{"requirement":"patient's age","precondition":"none"},"gender":{"requirement":"patient's gender","precondition":"none"},"main diagnosis":{"requirement":"need to extract the patient's main diagnosis","precondition":"none"}}, respectively. The precondition is set to construct the order of information transmission between agents, which is not involved in this example. The above agents may get the following results after running, denoted as R1: "Age": "65 years old", "Gender": "Female", "Main diagnosis": "Diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI 2 points, low-intermediate risk group)", "Date of diagnosis": "2023-04-18" Subsequently, the result R1 and the manual quality control standard value in step 1.1 (denoted as R0) are input as prompt words into the equivalence judgment agent. If the perplexity calculated by the judgment agent does not reach the termination condition, the planning agent is prompted to further optimize and increase the content to be extracted or the corresponding detail requirements (here, the result R1 and the perplexity output by the judgment agent will also be added to the memory module of the planning agent, so that the agent can compare the effects before and after the planning change).

[0081] The prompt words for inputting the judgment agent are as follows: [ { "role": "system", "content": "You are a doctor of medicine. You are good at comparing different medical records and analyzing details. Please distinguish whether the following two paragraphs of text descriptions of the disease treatment process or patient indicators are different in the above content. Choose one of the two options [significant difference, no difference] as the output.", }, { "role": "user", "content": "This is the first description" + quality control standard value, }, {"role": "user", "content": "This is the second description" + agent pipeline output content}, ] Taking the above example, because the previously extracted plan did not include the extraction of "diagnosis time", the planning agent will add the extraction task and agent of the diagnosis time after comparison.

[0082] Among them, if the extraction has a prerequisite, the agent construction program will also search whether the extraction prerequisite already exists when constructing the agent. If not, a new corresponding agent will be created to extract the prerequisite. For example, calculating the IPI score of lymphoma requires information such as patient age, lymphoma stage, physical condition and lactate dehydrogenase test value. Therefore, when the IPI score is to be extracted, the extraction agent with the prerequisite requirements will be added accordingly. This also reflects the idea of ​​dividing the process steps into multiple minimum sub-tasks in this application.

[0083] The construction of the remaining process steps is similar to the above steps, except that the initial prompt words are set differently. They can be automatically formed through repeated iterations and will not be repeated here.

[0084] It is understandable that the process steps may involve the retrieval of the knowledge base, such as the two process steps of matching the guideline knowledge base with the treatment plan and judging the matching degree of the treatment plan with the knowledge base involve the retrieval of the knowledge base.

[0085] In an embodiment of the present application, the content of the knowledge base is decomposed into a knowledge tree, and a task execution agent is assigned to each node of the knowledge tree. The knowledge module and memory module of the task execution agent respectively store the knowledge text of the corresponding node and the summary and keywords of the knowledge text.

[0086] The leaf nodes of the knowledge tree are in segments, that is, the smallest node is a text segment. The tables in the knowledge base are expressed in markdown, HTML or XML format, the flowcharts are expressed in mermaid or DOT language, and the pictures and their annotations are processed as paragraphs to form an independent leaf node.

[0087] Figure 6 The knowledge tree formed by breaking down the content of the Clinical Diagnosis and Treatment Guidelines - Hematology Volume into chapters, sections and paragraphs is shown.

[0088] The main functions performed by the intelligent agents of the nodes of the knowledge tree are as follows: 1) Generate a summary and keywords for the knowledge text contained in this node, which can be called the summary origin and keyword origin.

[0089] 2) Collect the summaries and keywords generated by the child nodes, and generate summaries of the summaries and keywords, which can be called summary and keyword summary.

[0090] 3) Receive the search instruction, and combine the knowledge text, abstract, keywords contained in the node, the search results of the child nodes and the summary information of the nodes at the same level to determine whether the search results can be generated by this node.

[0091] 4) Summarize the search results generated by the child nodes to form the search results of this node (including search results and reflection content). The structure of the node search results is: {"Search results":"Fill in the knowledge base content contained in this node related to the user's answer","Reflection content":"Fill in the possible defects and points of attention retrieved in this node according to the user's query statement"}.

[0092] like Figure 7 As shown in the figure, when performing knowledge retrieval on the knowledge base, first find multiple target nodes that match the retrieval instructions according to the abstract and keywords (i.e. lock the target knowledge), and then the task execution agents of the multiple target nodes perform parallel calculations to determine the correctness of the knowledge content references, and summarize and report the correctly referenced knowledge content layer by layer to form the final knowledge retrieval results, so that knowledge acquisition in the medical knowledge text scenario is accurate.

[0093] Among them, the knowledge tree can be searched by the breadth-first algorithm according to the abstract and keyword respectively by text vector similarity or conventional index retrieval method to find multiple target nodes that match the search instruction, wherein the operation of the large language model of the intelligent agent of the target node is parallel rather than serial. In addition, if it is a large knowledge base, a key-value pair can be formed separately for the keyword and abstract of each node (such as {node2358 (node ​​unique number): keyword | abstract, node2359 (node ​​unique number): keyword | abstract}), and the node number can be obtained after retrieval using a traditional database or vector database, so as to apply the node number to batch lock multiple target nodes of the knowledge tree.

[0094] When judging the correctness of the knowledge content reference, the query in the search instruction, such as "Can anemia be treated with tacrolimus?", is used as input A, and the knowledge content contained in the node is used as input B (such as "Iron deficiency anemia (IDA) is the most common anemia in clinical practice, with the highest incidence in women of childbearing age and infants. ..."), and the following prompt word input large language model module is constructed: [ { "role": "system", "content": "You are a doctor of medicine. Please complete the following three tasks: 1) Judge whether the provided knowledge description meets the user's query requirements, and first select one of the two options [Relevant, Irrelevant] as the output. 2) At the same time, reflect on the possible defects and deficiencies of the output knowledge description. 3) Whether the existing knowledge is sufficient to answer the user's question, choose an answer from [Sufficient, Insufficient].

[0095] Here is a sample output: { "Relevance": "Relevance", “Reflection content”: “This knowledge points out that symptom A only occurs in a few acute situations. Please pay attention to identifying more common symptoms.” "adequacy": "insufficiency" }", }, { "role": "user", "content": "This is a knowledge description" + input B, }, {"role": "user", "content": "This is a user requirement" + input A}, ] The upper parent node will receive the output content of the child node to judge or filter the content of the child node and pass it upward.

[0096] The leaf nodes will also judge the correctness of the knowledge content references and output defect suggestions. For the intermediate nodes (i.e. nodes with child nodes as input), after receiving the content of the child node feedback, they will form the following sample prompt words to input the large language model module: [ { "role": "system", "content": "You are a doctor of medicine. Please complete the following tasks based on your knowledge, user questions and points of attention. 1) First, choose one of the two options [Relevant, Irrelevant] as the output. 2) Then output the knowledge related to the query in a concise form. 3) Then reflect on the possible defects and deficiencies of the output knowledge description. 4) Whether the existing knowledge is sufficient to answer the user's question, choose an answer from [Sufficient, Insufficient].

[0097] Here is a sample output: { "Relevance": "Relevance", "Related knowledge": "Tacrolimus can treat anemia caused by immune etiology, such as severe aplastic anemia" "Reflection content": "This section of knowledge refers to severe aplastic anemia, which may not be covered by other diseases" "sufficiency":"sufficiency" }", }, { "role": "user", "content": "This is a knowledge description" + the knowledge text paragraph or chapter that comes with this node, }, {"role": "user", "content": "This is a user question" + input A}, {"role": "user", "content": "This is a relevant point" + the child node outputs the conclusion as the knowledge content of the relevant node}, {"role": "user", "content": "This is a point of attention" + the reflection content description of the child node}, ] The intermediate nodes process the knowledge by applying the large language model through the above prompt words, and collect them upward layer by layer. In practical applications, you can set a threshold for the number of levels that the node considers irrelevant. For example, you can set the threshold for the number of levels to 2. When an intermediate node and its leaf node (i.e., the 2nd-layer node) are locked by search methods such as keywords, but both are judged to be irrelevant (the knowledge content is incorrectly cited), the branch formed by these two nodes will be pruned and closed, and the information will no longer be submitted upward, so that the node at the next level is regarded as a leaf node, thereby improving the usage efficiency. See Figure 8 .

[0098] Similarly, a threshold of sufficient levels can be set. For example, if two consecutive layers of nodes believe that sufficient knowledge retrieval results have been obtained (the knowledge content is correctly cited), the relevant content can be directly returned without passing the judgment upward.

[0099] From the above, it can be seen that in the method of this embodiment, the planning agent divides each process step of the medical record analysis process into multiple minimum sub-tasks according to the prompt words, and the planning agent assigns a corresponding task execution agent to each sub-task, thereby forming an agent workflow line. The medical record analysis process is executed by the agent workflow line, which not only adapts to the semantics of complex medical contexts, but also improves the efficiency and accuracy of medical record analysis.

[0100] Among them, the intelligent agent can make the large language model module output the execution results of the tasks it undertakes based on external information, the input and output records of the general large language model module in the memory module and the corresponding reflection results, and the knowledge information in the knowledge module, making the large language model module a vertical domain model with the ability to execute tasks in professional fields, thereby improving the cost-effectiveness.

[0101] At the same time, unlike the prior art of processing medical records through large language models, the input and output of each node (agent) in the agent workflow of the method of this embodiment is transparent, readable and interpretable, thereby ensuring the stability of the output quality.

[0102] In addition, the method of this embodiment automatically trains each task execution agent through a planning agent, and each task execution agent is only responsible for a single task with low complexity, so the training cost of each language model is low. In the prior art, retraining or fine-tuning based on a general large language model requires a large amount of domain-specific data to adjust its parameters to adapt to a specific field. However, due to the dispersed scale of vertical fields, the advantages of GPU parallel computing are relatively difficult to reflect. The lack of scale advantage leads to low computer utilization, and there is a lack of ready-made training data. Preparing new data requires additional costs, and the large model technology is iteratively updated quickly, and the cost-benefit of training and fine-tuning investment is low.

[0103] In addition, organizing the knowledge base in the form of a knowledge tree improves the retrieval accuracy of the knowledge base. At the same time, considering the characteristics of the medical knowledge tree with a shallow hierarchy but many branches, multiple nodes can process the knowledge base branches in parallel, ensuring the efficiency of retrieval. In the prior art, RAG is used to build a knowledge base, and the similarity of word vector search is used to perform knowledge retrieval. For long texts, the length of the text slice needs to be set, such as 100 words or 300 words. This slicing and searching method is prone to the phenomenon of taking things out of context, which is not obvious in texts centered on characters and events, such as novels, but is particularly prominent for the retrieval of medical knowledge. For example, medical materials are often divided into different chapters for description according to different disease names, and disease names are often used as chapter titles (few words, such as disease A, which is not repeated later or appears as a pronoun, such as "this disease" refers to disease A). When searching by similarity, most of the useful chapters will be missed when searching with the name of disease A as the condition when obtaining disease treatment methods. It is also easy to take the brief introduction of the target disease in the description chapter of non-target diseases (for example, the differential diagnosis part, such as "Compared with disease A (target disease), the ultrasound characteristics of disease B (non-target disease) are...") as the core knowledge, resulting in errors in application.

[0104] The medical record analysis device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution. Figure 2 A medical record analysis device provided by an embodiment of the present invention is shown. Figure 2 As shown, the device includes a first unit 11 , a second unit 12 , a third unit 13 , a fourth unit 14 and a fifth unit 15 .

[0105] The first unit 11 is used to generate prompt words according to the content of the target medical record, the content of the expected output and the requirements, and input them into the planning agent.

[0106] The second unit 12 is used for the planning agent to divide each process step of the medical record analysis process into a plurality of minimum subtasks according to the prompt words.

[0107] The third unit 13 is used for the planning agent to establish and allocate corresponding task execution agents for each subtask, forming an agent workflow for executing the above-mentioned medical record analysis process, and at the same time, sending prompt word content corresponding to the corresponding subtask to each task execution agent.

[0108] The fourth unit 14 is used for each task execution agent to generate a prompt word through its instruction module according to the prompt word content sent by the planning agent, and input the prompt word into its general large language model module to execute the corresponding subtask.

[0109] Among them, one of the multiple task execution agents used to execute multiple minimum subtasks of a process step is used to summarize the execution results of the remaining task execution agents, obtain the execution results of the process step, and send them to the planning agent.

[0110] In the above-mentioned intelligent agent workflow, the subsequent task execution agent may need to use the output of the previous task execution agent as input. The subsequent task execution agent can directly obtain the output from the corresponding previous task execution agent as input. Of course, when the subsequent task execution agent corresponds to different process steps from the previous task execution agent, the subsequent task execution agent can also obtain corresponding information from the planning agent. For example, the task execution agent of the process step of judging the matching degree between the treatment plan and the knowledge base can obtain the guide knowledge base treatment plan from the planning agent.

[0111] The fifth unit 15 is used for the planning agent to output the execution results of the process steps, such as outputting the execution results on a display screen.

[0112] like Figure 5 As shown in the figure, taking the medical record data extraction process step as an example, the expected output content and requirements can be understood as the expected extraction content and extraction requirements. The planning agent divides it into 5 minimum subtasks according to the prompt words: basic information extraction, admission and discharge time extraction, treatment drug extraction, treatment drug use time extraction and summary format arrangement. The 5 minimum subtasks are respectively responsible for 5 task execution agents, among which summary format arrangement refers to summarizing and formatting the extracted basic information, admission and discharge time, treatment drugs and treatment drug use time. Similarly, the other 4 process steps can also be divided into multiple minimum subtasks, which will not be repeated here.

[0113] It is understandable that the five process steps can be handled by one planning agent or by five planning agents respectively.

[0114] It should be pointed out that if an agent is asked to perform the entire process steps or non-minimum subtasks, its large language model will be difficult to give accurate results at one time due to the complexity of the task. After being divided into minimum subtasks, the large language model can focus on performing a single subtask. The complexity of each subtask is greatly reduced, which reduces the burden on the model and allows it to focus more on the details of each specific task, making the large language model more efficient and accurate.

[0115] After the agent work pipeline is initially formed, it needs to be optimized and trained. The specific process is as follows. Figure 5 : S11. Take multiple medical record samples (such as 100) as target medical records and test the intelligent agent workflow through steps S101-S105.

[0116] S12. Create a judgment agent to determine whether the execution result of each process step is different from the corresponding quality control reference value. If so, the difference is output to the planning agent. Otherwise, the confusion without difference is output to the planning agent.

[0117] Specifically, it is necessary to construct judgment prompt words with the quality control reference value and the above execution results and input them into the judgment agent (the large model module of the agent needs to select an open source large model, because the closed source model cannot obtain parameter information during the model operation process). The "no difference" perplexity output by the large model is used to judge the effect of the agent's workflow (the lower the perplexity, the better the solution).

[0118] Among them, perplexity is an important indicator to measure the performance of language models, especially in the field of natural language processing (NLP). The lower the perplexity, the stronger the predictive ability of the model, which means that the model has a better understanding of the language. The calculation of perplexity is usually based on the probability distribution of the model. In this application, by requiring the model to output a fixed target string, the perplexity is used to evaluate the credibility of the model's output target string, and the output of the large language model is converted into a quantitative indicator (for example, originally only classification judgments such as [yes, no] can be output, now [yes, no] can be output, plus a perplexity value, which quantifies the credibility of the model's output target string in the judgment context).

[0119] The above-mentioned confusion of no difference represents how sure or confident the judging agent is that there is no difference between the execution result and the corresponding quality control reference value. It is a scoring value, and the lower the value, the greater the sureness or confidence.

[0120] The steps to calculate the perplexity are as follows: 1. Calculate the probability of each word: For a given test data set, the model generates a probability distribution for each word. Assume that the test data set contains N words, denoted as , , , .

[0121] The model is for each word The probability of generation is recorded as .

[0122] 2. Calculate the perplexity: The calculation formula of perplexity PP is:

[0123] in, Usually the logarithm is based on the natural logarithm ln.

[0124] The judgment agent can be shared by all process steps, or it can be divided into different agents to execute from the perspective of parallelism and clear division of labor. In practical applications, in order to increase the degree of parallel operation and improve operating efficiency, multiple agents can be responsible for quality control.

[0125] In order to provide more stable and reliable judgment results, reduce the random influence of a single model, and maintain a high level of vigilance when problems are discovered, this application has made the following improvements to the perplexity evaluation: 1. Multi-agent judgment mechanism to reduce the randomness of single model judgment: An odd number of judging agents (usually 5 or 7) are used to give their opinions at the same time to simulate the expert evaluation process. Each judging agent independently calculates its confusion about the target string ("no difference" or "difference"), and the judgment results of multiple judging agents are determined by a voting mechanism to determine the final judgment.

[0126] 2. Based on the Logits Fusion principle, the perplexity output by different judgment agents is combined by geometric mean to smooth the perplexity evaluation results: If the judgment results of an odd number of judgment agents are the same, then the perplexity output by the odd number of judgment agents is combined using formula (1): (1) Where n is the number of judging agents, PP i is the confusion value of the i-th agent; If some of the judgment results of an odd number of judgment agents are the same and some are different, then the maximum perplexity among all the judgment agents whose judgment results are the same is taken as the perplexity of all the judgment agents whose judgment results are different. , and then combine the perplexity output by the odd number of judgment agents through formula (1); Among them, m is the number of judging agents whose judgment results are indifferent.

[0127] The reason for this design is that "difference" indicates a problem, and the most uncertain (highest confusion) judgment should be selected as the final result, thereby increasing the system's sensitivity to potential problems.

[0128] The following is an example of perplexity calculation: 1. Define the target string The target string is "no difference".

[0129] 2. Get the model's predicted probability for the target string Suppose the model predicts the following probability for the string "no difference": No: 0.7 Yes: 0.8 Difference: 0.6 Different: 0.75 3. Calculate the log probability of each character Take the natural logarithm ln of the probability of each character: ln(0.7) ≈ -0.357 ln(0.8) ≈ -0.223 ln(0.6) ≈ -0.511 ln(0.75) ≈ -0.287 4. Calculate the total log probability Add up the log probabilities of all characters:

[0130]

[0131] 5. Calculate the perplexity The calculation formula for perplexity is:

[0132] Where N is the length of the target string, so in this example, N = 4.

[0133]

[0134] 6. Comprehensive judgment by multiple agents Suppose we use 3 judgment agents, and each agent independently calculates the perplexity as follows: Agent 1 outputs "no difference": PP1=1.411 Agent 2 outputs "no difference": PP2=1.325 The output of Agent 3 is judged to be "different": PP3=1.468 7. Logits Fusion merge confusion Since it is meaningless to take the perplexity of the difference when some agents have different outputs, the perplexity of the judgment that the output is "different" is first represented by the maximum perplexity of the judgments that the output is "not different":

[0135] Then use the geometric mean to fuse the results:

[0136] 8. Explaining perplexity For the judgment of "no difference", the final perplexity is 1.382, which represents the average judgment result of the three agents. The lower the perplexity, the more confident the model is in predicting the target string.

[0137] S13. The planning agent adds, deletes or modifies the task execution agent according to the content of the difference until the perplexity without difference meets the preset conditions. The optimization is completed. For example, when the reduction of perplexity before and after different iterations is less than 0.1% or the absolute value of the difference between the perplexity before and after is less than 50 (or other manually set thresholds), the optimization is completed.

[0138] Among them, the impact of adding, deleting or modifying a single-task execution agent on the execution result is evaluated in parallel (keeping other task execution agents unchanged, and multi-arm A / B testing), and finally the combination effect of the optimal task execution agent is tested.

[0139] When the planning agent needs to modify a task execution agent, on the one hand, the planning agent adjusts the parameters of the general large language model module of the task execution agent, and can copy and create an agent with the same function, but adjust the parameters, or set up multiple agents corresponding to the same subtask and with different parameters from the beginning, and finally make a choice based on the effect; on the other hand, the planning agent forwards the above-mentioned differences to the task execution agent, and the task execution agent reflects on the output task execution results based on the differences, and stores the reflection results in the memory module. When the subtask is executed again, the reflection results are input into its general large language model module as part of the prompt word.

[0140] Reflection results such as: "When I was processing....., I mistakenly put..... I should put....” The current general reflection technology has the defect of lacking creative behavior to escape the local minimum. Therefore, this application adds creative behavior during reflection to escape the local minimum, as follows: 1. Identify and exclude invalid reflection results: Calculate the similarity between the results of the previous and next rounds of reflection (the calculation method can be compared by the count of corresponding tokens, or semantic similarity). For example, the similarity between the results of the previous and next rounds of reflection sr0 and sr1 is 0.9, which exceeds the set artificial threshold. In this case, the new reflection will not be included in the memory module as an invalid reflection, so as to avoid the large model falling into a repetitive mode and entering an infinite loop, thus being unable to escape the local minimum.

[0141] 2. After multiple cycles of identifying similar reflections, inspiration is added to increase the randomness of the model output and expand the search space for the optimal value. Among them, inspiration is generated by a dedicated inspiration agent (the prompt word input to it is usually "You are an inspiration generator (coach), you generate innovative ideas based on the given tasks and reflection results, but do not give specific answers") based on the principle of few-shot examples. For example, in the drug extraction task, inspiration may include inspiring suggestions such as "consider different dosage forms of drugs" and "distinguish between prescription drugs and over-the-counter drugs". These inspirations will serve as additional contextual information to help the main model jump out of the local optimal solution.

[0142] Of course, if the optimization is not completed within 72 hours, manual intervention will be required for adjustment.

[0143] By organizing the intelligent agent workflow and comparing the execution results of each process step, the stability, transparency and controllability of the large language model output are improved.

[0144] The following is an example of an actual medical record to illustrate this step. If the input medical record sample is a picture or a PDF, Word or other document, it needs to be converted into a text format using OCR or PDF, Word tools. The specific process is as follows: Step 1: Convert the medical record sample into text format. The specific text after conversion is as follows: Discharge Record Name: *** Gender: Female Age: 65 Occupation: Retired Department: Thyroid and Breast Surgery Ward 2 Ward: (Ward) Thyroid and Breast Surgery Bed number: +10 Bed hospitalization medical record number: *** Date of admission: April 8, 2023 Date of discharge: April 27, 2023 Number of days in hospital: 19 days Admission and transfer department: Thyroid and Breast Surgery Ward II (Houhu) Admission diagnosis Discharge diagnosis Cervical lymphadenopathy 1. Diffuse large B-cell lymphoma (stage ⅡA) (involving the neck lymph nodes, ABC type) (IPI2 points, low- to medium-risk group) 2. Chemotherapy 3. Bone marrow suppression after chemotherapy Admission and treatment history: Patient ***, female, 65 years old. She had a lump on the right side of her neck for 3 months. Past medical history: denied hypertension, diabetes Disease, denied food and drug allergy history. Denied history of infectious diseases such as tuberculosis and hepatitis. Partial thyroidectomy in 1982: History of cesarean section. Physical examination: soft neck, trachea in the middle, a mass of about 8*6cm in size can be felt on the right side of the neck, medium in texture. Outpatient information: On March 4, 2023, an ultrasound-guided puncture biopsy of the right submandibular mass was performed at Wufan Central Hospital. Pathological examination: (puncture of the right submandibular mass) was considered to be reactive m lymphadenitis.

[0145] After admission, complete relevant auxiliary examinations, 2023-04-10 Myocardial enzyme spectrum four items α-hydroxybutyrate dehydrogenase 243U / L↑(72~182), acid dehydrogenase (rate method) 297U / L↑(103~227). 2023-04-10 Biochemical examination 2 *Total protein (double hyperplasia endpoint method) 52.8g / L↓(65~85), *Albumin (bromocresol green method) 38.2g / L↓(40~55), *Apolipoprotein B (immunoturbidimetry) 1.07g / L↑(0.69~1.05), Apolipoprotein E 88.8mg / L↑(30~50), Free fatty acid 0.62mmol / mouth (0.172~0.586), 2023-04-10 Complete blood count + five classifications: white blood cell count 4.43x10^9 / L15~9.5), hemoglobin concentration 116g / L (115~150), platelet count 186x10^9 / L (125~350), neutrophil count 2.35x10^9 / L (1.8~6.3): 2023-04-22 Plasma D-dimer determination (D-Dimer) D-dimer 01.815μg / mL↑(0~0.5); high-sensitivity C-reactive protein, serum troponin I determination (chemiluminescence method), coagulation test 2, erythrocyte sedimentation rate, coagulation routine, eight items of pre-transfusion examination, and hepatitis B DNA quantitative determination showed no obvious abnormalities. 2023-04-26, complete blood count + five categories: white blood cell count 10.17x10^9 / L↑(3.5~9.5), hemoglobin concentration 108g / L↓(115~150), platelet count 240x10^9 / L(125~350), neutrophil count 7.68x10^9 / L↑(L8~6.3): 2023-04-26, five renal function tests: * uric acid (uric acid Enzyme colorimetric method) 119μmol / L↓(155~357), creatinine (enzyme method) 31.6μmol / L↓(41~81): 2023-04-26, eleven liver function tests *Albumin (bromocresol method) 43.2g / L↓(40~55), albumin-globulin ratio 1.1↓(1.2~2.4), *Total protein (bidimensional end-point method) 62.3g / mouth↓(65~85): No obvious abnormalities were found in C-reactive protein and B2 microglobulin.

[0146] 2023-04-08DR: No obvious abnormalities were found in the heart, lungs and diaphragm.

[0147] 2023-04-08ECG: 1. Sinus rhythm, 2. Normal range ECG.

[0148] 2023-04-10 Color Doppler ultrasound of the thyroid and cervical lymph nodes: cystic nodule with crystals in the left lobe of the thyroid gland, considered to be an alveolar cyst, I-RADS2 category, a huge solid mass on the right side of the neck, and no obvious enlarged lymph nodes on the left side of the neck.

[0149] 2023-04-11 CT scan of soft tissue in the neck: 1. A huge mass in the right paracervical space. Considering the history, the source of the mass is lymph node or thyroid. (***Hospital) Discharge Record Name: *** Gender: Female Age: 65 Occupation: Retired Department: Thyroid and Breast Surgery Ward II Ward: (Ward) Thyroid and Breast Surgery Bed No.: +10 Bed Hospitalization Medical Record No.: ***Original space occupation. 2. Partial filling defect of right brachiocephalic vein, consider embolism; part of right internal jugular vein is not visualized, please combine clinical diagnosis.

[0150] 2023-04-19 Pathology: (Neck) non-Hodgkin diffuse aggressive B-cell lymphoma, consistent with diffuse large B-cell lymphoma-not specified (activated B-cell origin), immunohistochemistry: tumor cells: CD20, PAX-5, BCL-2, MUH-1 (+), C-MYC (+, 30%): P53 (mutant), Ki-67Li85%; the rest (CD6, CD1O, BCL-6, CyclinD1, CD21, CD23, ED3, CD43, PCK, S-100) are all (-), CD30 (-), molecular detection: EBER (-), 2023-04-21PET-CT: 1. A soft tissue density mass in the right paracervical space with abnormally increased metabolism is considered to be a sign of lymphoma infiltration.

[0151] 2. Small lymph nodes in zone IV of the left neck, with slightly increased metabolism, lymphoma infiltration is considered. 3. Multiple small lymph nodes in the remaining bilateral neck, bilateral axillary area, and bilateral inguinal area, with low metabolism, considered nonspecific changes, and follow-up examination is recommended. 4. Pure GG nodules in the anterior basal segment of the right lower lobe, with low metabolism, thin-layer CT follow-up examination is recommended. 5. Cystic and solid masses in the left adnexa, with low metabolism, are mostly considered benign lesions, and combined with specialist examinations: right adnexa teratoma. 6. No obvious signs of malignant tumor lesions were found in other parts of the body. 7. Brain atrophy; leukoaraiosis. 8. Left maxillary sinusitis; bilateral inferior turbinate hypertrophy; bilateral tonsil inflammation or physiological uptake. 9. Bilateral breast hyperplasia. 10. A few fibrous lesions in both lower lungs. 11. Multiple liver cysts; gallstones: a little gas accumulation in the lower part of the common bile duct: increased metabolism in the rectum and anorectal area, consider inflammation or physiological uptake. 12. Osteoporosis: multiple vertebral degeneration: left iliac bone island; bilateral low iliac joint degeneration; inflammatory lesions or physiological uptake of the muscles next to the left hip joint.

[0152] 2023-04-23 Cardiac ultrasound, cardiac function, and ventricular wall motion analysis: slight mitral valve regurgitation, slight tricuspid valve regurgitation, and pulmonary hypertension (mild).

[0153] 2023-4-24 Bone marrow cytology: No granulocytes were found in the bone marrow, the bone marrow showed trilineage hyperplasia, and the proportion of neutrophils and segmented granulocytes increased.

[0154] Immunophenotyping: The proportions of various cell groups in the bone marrow are roughly normal, the granulocytes are differentiated and mature, there is no obvious proliferation of primitive cell groups, and there is no increase in the proportion of B lymphocytes and abnormal phenotype. The immune phenotype is not characteristic.

[0155] 2023-04-25,DR: No substantial lesions were found in both lungs. Scoliosis was found.

[0156] The patient was hospitalized in the Department of Thyroid and Breast Surgery of our hospital due to a huge mass on the right side of the neck. After preoperative examination, a right cervical lymph node biopsy (right side) was performed on 2023-04-11. After surgery, anti-inflammatory, hemostatic, symptomatic supportive treatment and regular dressing changes were given. Postoperative pathology showed diffuse large B-cell lymphoma. The patient was transferred to the Department of Hematology on 2023-4-20 and was diagnosed with diffuse large B-cell lymphoma stage ⅡA. On 2023-04-25, a left upper PICC catheterization was performed, and no discomfort was reported after the operation. On 2023-4-22, CHOP chemotherapy was started (DEX13mgd1-5, CTX1000mgd5, VDS3mgd5, doxorubicin liposome 30mgd5). The process was smooth and the neck mass was reduced after treatment.

[0157] He is now being discharged from the hospital.

[0158] Discharge: The patient is in good general condition, with no complaints of special discomfort. The stitches on the neck wound have been removed, and there is no wet blood exudation. Physical examination: normal vital signs, 1 / nail of the neck wound healed, no redness, swelling, no bleeding, tenderness. Clear breath sounds in both lungs, normal heart sounds. Soft abdomen, no tenderness or rebound pain, liver and spleen not affected, normal bowel sounds.

[0159] Discharge instructions: 1. Avoid strenuous exercise and keep the wound dry; 2. Come to the hospital for the next course of treatment on 2023-5-15. If you feel unwell, please see a doctor in time.

[0160] Department outpatient arrangements: *** Chief Physician: Fujing Road Tuesday morning, Thursday morning, Houhu Friday morning: *** Deputy Chief Physician Doctor: ***Saturday morning, Houhu Thursday morning.

[0161] Step 1.1: This step is limited to process construction. The user input content is manually analyzed by experts to form quality control reference values ​​under the process framework of "medical record data extraction-key indicator calculation-actual treatment plan extraction-guideline knowledge base treatment plan matching-treatment plan and knowledge base matching judgment".

[0162] The quality control reference values ​​after manual analysis by experts are as follows: 1) Reference values ​​for medical record data extraction "Date of admission": "2023-04-08", "Discharge Date": "2023-04-27", "Diagnosis upon admission": "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "Start treatment date": "2023-04-22", "Treatment plan": "The patient received CHOP chemotherapy, including: DEX 13mgd1-5, CTX 1000mgd5, VDS 3mgd5, doxorubicin liposome 30mgd5.", "Effect of treatment": "After CHOP chemotherapy, the neck mass has shrunk, but the specific effect does not mention specific indicators such as CR, PR, etc.", "Disease outcome": "Disease condition has improved." 2) Reference values ​​for key indicator calculations "IPI score conclusion": "April 08, 2023, IPI2 points, low-medium risk group", "IPI scoring basis": "Medical record extraction results", "IPI Scoring Process": "Medical Record Extract" 3) Reference values ​​extracted from actual treatment plans "Start treatment date": "2023-04-22", "Diagnosis upon admission": "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "Actual treatment plan": "CHOP chemotherapy, including: DEX 13mgd1-5, CTX 1000mgd5, VDS 3mgd5, doxorubicin liposome 30mgd5." 4) Guideline knowledge base treatment plan matching results "Start treatment date": "2023-04-22", "Diagnosis upon admission": "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "Patient type of this treatment": "First-time patients", "Recommended treatment options": "[R-CHOP21, R-CDOP, R-CHOEP]", "Reason or basis for recommendation": "The patient type in this treatment is a treatment-naive patient, aged 65 years old, with an IPI score of 2 points, no heart failure, and currently a treatment-naive patient, and does not meet the conditions for transplantation. According to the first-line treatment regimen for diffuse large B-cell lymphoma in the "Reference Materials", the [R-CHOP21, R-CDOP, R-CHOEP] regimen in the first-line treatment is recommended." 5) Judgment results of the matching degree between treatment plan and knowledge base "Recent patient treatment": [ ("2023-04-22", "Enlarged cervical lymph nodes, diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI2 points, low-intermediate risk group)", "CHOP chemotherapy, specifically including: DEX13mgd1-5, CTX1000mgd5, VDS3mgd5, doxorubicin liposome 30mgd5.") ], "Is it a recommended solution": ["Not applicable"], "Recommended Ratio": "0 / 1", "Subsequent recommended regimen": "[R-CHOP21]", "Reasoning for the recommended regimen": "The patient is a treatment-naive patient, aged 65 years, with an IPI score of 2 points. There is a description of heart failure in the medical record, so the patient does not meet the transplant conditions. According to the first-line treatment regimen for diffuse large B-cell lymphoma in the "Reference Materials", considering that the patient is a treatment-naive patient and does not meet the transplant conditions, the R-CHOP21 regimen in the first-line treatment is recommended. This regimen includes rituximab and CHOP chemotherapy (cyclophosphamide, doxorubicin, vincristine, prednisone). The addition of rituximab can improve the treatment effect, especially for the treatment of B-cell lymphoma. Therefore, it is recommended to use the R-CHOP21 regimen in the future to optimize the treatment effect." Step 2: Use the original medical records from step 1 and the manual quality control reference values ​​from step 1.1 to automatically arrange and test the process (taking medical record data extraction as an example): First, the extraction of medical record data is taken as the goal, and the medical record text in step 1 is taken as the original medical record, and the required output extraction content and the extraction requirement list together constitute the prompt word input planning agent.

[0163] Examples of planning agent prompts are: [ { "role": "system", "content": "You are a doctor of medicine who is good at organizing and planning interns to extract medical record information", }, { "role": "user", "content": "This is the original medical record" + the original medical record text of step 1, }, {"role": "user", "content": "This is the target information that needs to be extracted" + the manual quality control standard in step 1.1}, {"role": "user", "content": "This is the information you extracted before" + historical output memory (initialized to empty, extracted from the memory module)}, {"role": "user", "content": "Please parse and output in JSON object format how to obtain the information elements in the target extraction. The following is an example: Original data: {" Discharge Record Name: *** Gender: Female Age: 65 Occupation: Retired Department: Thyroid and Breast Surgery Ward 2 Ward: (Ward) Thyroid and Breast Surgery Bed number: +10 Bed hospitalization medical record number: *** Date of admission: April 8, 2023 Date of discharge: April 27, 2023 Number of days in hospital: 19 days Admission and transfer department: Thyroid and Breast Surgery Ward II (Houhu) Admission diagnosis Discharge diagnosis Cervical lymphadenopathy 1. Diffuse large B-cell lymphoma (stage ⅡA) (involving the neck lymph nodes, ABC type) (IPI2 points, low- to medium-risk group) 2. Chemotherapy 3. Bone marrow suppression after chemotherapy } Target information: "Age": "65 years old", "Gender": "Female", "Main diagnosis": "Diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI 2 points, low-intermediate risk group)", "Date of diagnosis": "2023-04-18", } Output content: {"Age":{"Required":"Age of the patient","Precondition":"None"},"Gender":{"Required":"Gender of the patient","Precondition":" None"},"Main diagnosis":{"Requirement":"The patient's main diagnosis needs to be extracted","Precondition":"None"}}, ] After that, the agent construction program will construct an information extraction agent and an information aggregation agent for extracting the patient's age, gender, and main diagnosis through the instance of the new agent class and the output of the planning agent {"age":{"requirement":"patient's age","precondition":"none"},"gender":{"requirement":"patient's gender","precondition":"none"},"main diagnosis":{"requirement":"need to extract the patient's main diagnosis","precondition":"none"}}, respectively. The precondition is set to construct the order of information transmission between agents, which is not involved in this example. The above agents may get the following results after running, denoted as R1: "Age": "65 years old", "Gender": "Female", "Main diagnosis": "Diffuse large B-cell lymphoma (stage IIA) (involving cervical lymph nodes, ABC type) (IPI 2 points, low-intermediate risk group)", "Date of diagnosis": "2023-04-18" Subsequently, the result R1 and the manual quality control standard value in step 1.1 (denoted as R0) are input as prompt words into the equivalence judgment agent. If the perplexity calculated by the judgment agent does not reach the termination condition, the planning agent is prompted to further optimize and increase the content to be extracted or the corresponding detail requirements (here, the result R1 and the perplexity output by the judgment agent will also be added to the memory module of the planning agent, so that the agent can compare the effects before and after the planning change).

[0164] The prompt words for inputting the judgment agent are as follows: [ { "role": "system", "content": "You are a doctor of medicine. You are good at comparing different medical records and analyzing details. Please distinguish whether the following two paragraphs of text descriptions of the disease treatment process or patient indicators are different in the above content. Choose one of the two options [significant difference, no difference] as the output.", }, { "role": "user", "content": "This is the first description" + quality control standard value, }, {"role": "user", "content": "This is the second description" + agent pipeline output content}, ] Taking the above example, because the previously extracted plan did not include the extraction of "diagnosis time", the planning agent will add the extraction task and agent of the diagnosis time after comparison.

[0165] Among them, if the extraction has a prerequisite, the agent construction program will also search whether the extraction prerequisite already exists when constructing the agent. If not, a new corresponding agent will be created to extract the prerequisite. For example, calculating the IPI score of lymphoma requires information such as patient age, lymphoma stage, physical condition and lactate dehydrogenase test value. Therefore, when the IPI score is to be extracted, the extraction agent with the prerequisite requirements will be added accordingly. This also reflects the idea of ​​dividing the process steps into multiple minimum sub-tasks in this application.

[0166] The construction of the remaining process steps is similar to the above steps, except that the initial prompt words are set differently. They can be automatically formed through repeated iterations and will not be repeated here.

[0167] It is understandable that the process steps may involve the retrieval of the knowledge base, such as the two process steps of matching the guideline knowledge base with the treatment plan and judging the matching degree of the treatment plan with the knowledge base involve the retrieval of the knowledge base.

[0168] In an embodiment of the present application, the content of the knowledge base is decomposed into a knowledge tree, and a task execution agent is assigned to each node of the knowledge tree. The knowledge module and memory module of the task execution agent respectively store the knowledge text of the corresponding node and the summary and keywords of the knowledge text.

[0169] The leaf nodes of the knowledge tree are in segments, that is, the smallest node is a text segment. The tables in the knowledge base are expressed in markdown, HTML or XML format, the flowcharts are expressed in mermaid or DOT language, and the pictures and their annotations are processed as paragraphs to form an independent leaf node.

[0170] Figure 6 The knowledge tree formed by breaking down the content of the Clinical Diagnosis and Treatment Guidelines - Hematology Volume into chapters, sections and paragraphs is shown.

[0171] The main functions performed by the intelligent agents of the nodes of the knowledge tree are as follows: 1) Generate a summary and keywords for the knowledge text contained in this node, which can be called the summary origin and keyword origin.

[0172] 2) Collect the summaries and keywords generated by the child nodes, and generate summaries of the summaries and keywords, which can be called summary and keyword summary.

[0173] 3) Receive the search instruction, and combine the knowledge text, abstract, keywords contained in the node, the search results of the child nodes and the summary information of the nodes at the same level to determine whether the search results can be generated by this node.

[0174] 4) Summarize the search results generated by the child nodes to form the search results of this node (including search results and reflection content). The structure of the node search results is: {"Search results":"Fill in the knowledge base content contained in this node related to the user's answer","Reflection content":"Fill in the possible defects and points of attention retrieved in this node according to the user's query statement"}.

[0175] like Figure 7 As shown in the figure, when performing knowledge retrieval on the knowledge base, first find multiple target nodes that match the retrieval instructions according to the abstract and keywords (i.e. lock the target knowledge), and then the task execution agents of the multiple target nodes perform parallel calculations to determine the correctness of the knowledge content references, and summarize and report the correctly referenced knowledge content layer by layer to form the final knowledge retrieval results, so that knowledge acquisition in the medical knowledge text scenario is accurate.

[0176] Among them, the knowledge tree can be searched by the breadth-first algorithm according to the abstract and keyword respectively by text vector similarity or conventional index retrieval method to find multiple target nodes that match the search instruction, wherein the operation of the large language model of the intelligent agent of the target node is parallel rather than serial. In addition, if it is a large knowledge base, a key-value pair can be formed separately for the keyword and abstract of each node (such as {node2358 (node ​​unique number): keyword | abstract, node2359 (node ​​unique number): keyword | abstract}), and the node number can be obtained after retrieval using a traditional database or vector database, so as to apply the node number to batch lock multiple target nodes of the knowledge tree.

[0177] When judging the correctness of the knowledge content reference, the query in the search instruction, such as "Can anemia be treated with tacrolimus?", is used as input A, and the knowledge content contained in the node is used as input B (such as "Iron deficiency anemia (IDA) is the most common anemia in clinical practice, with the highest incidence in women of childbearing age and infants. ..."), and the following prompt word input large language model module is constructed: [ { "role": "system", "content": "You are a doctor of medicine. Please complete the following three tasks: 1) Judge whether the provided knowledge description meets the user's query requirements, and first select one of the two options [Relevant, Irrelevant] as the output. 2) At the same time, reflect on the possible defects and deficiencies of the output knowledge description. 3) Whether the existing knowledge is sufficient to answer the user's question, choose an answer from [Sufficient, Insufficient].

[0178] Here is a sample output: { "Relevance": "Relevance", “Reflection content”: “This knowledge points out that symptom A only occurs in a few acute situations. Please pay attention to identifying more common symptoms.” "adequacy": "insufficiency" }", }, { "role": "user", "content": "This is a knowledge description" + input B, }, {"role": "user", "content": "This is a user requirement" + input A}, ] The upper parent node will receive the output content of the child node to judge or filter the content of the child node and pass it upward.

[0179] The leaf nodes will also judge the correctness of the knowledge content references and output defect suggestions. For the intermediate nodes (i.e. nodes with child nodes as input), after receiving the content of the child node feedback, they will form the following sample prompt words to input the large language model module: [ { "role": "system", "content": "You are a doctor of medicine. Please complete the following tasks based on your knowledge, user questions and points of attention. 1) First, choose one of the two options [Relevant, Irrelevant] as the output. 2) Then output the knowledge related to the query in a concise form. 3) Then reflect on the possible defects and deficiencies of the output knowledge description. 4) Whether the existing knowledge is sufficient to answer the user's question, choose an answer from [Sufficient, Insufficient].

[0180] Here is a sample output: { "Relevance": "Relevance", "Related knowledge": "Tacrolimus can treat anemia caused by immune etiology, such as severe aplastic anemia" "Reflection content": "This section of knowledge refers to severe aplastic anemia, which may not be covered by other diseases" "sufficiency":"sufficiency" }", }, { "role": "user", "content": "This is a knowledge description" + the knowledge text paragraph or chapter that comes with this node, }, {"role": "user", "content": "This is a user question" + input A}, {"role": "user", "content": "This is a relevant point" + the child node outputs the conclusion as the knowledge content of the relevant node}, {"role": "user", "content": "This is a point of attention" + the reflection content description of the child node}, ] The intermediate nodes process the knowledge by applying the large language model through the above prompt words, and collect them upward layer by layer. In practical applications, you can set a threshold for the number of levels that the node considers irrelevant. For example, you can set the threshold for the number of levels to 2. When an intermediate node and its leaf node (i.e., the 2nd-layer node) are locked by search methods such as keywords, but both are judged to be irrelevant (the knowledge content is incorrectly cited), the branch formed by these two nodes will be pruned and closed, and the information will no longer be submitted upward, so that the node at the next level is regarded as a leaf node, thereby improving the usage efficiency. See Figure 8 .

[0181] Similarly, a threshold of sufficient levels can be set. For example, if two consecutive layers of nodes believe that sufficient knowledge retrieval results have been obtained (the knowledge content is correctly cited), the relevant content can be directly returned without passing the judgment upward.

[0182] In summary, the medical record analysis device provided in the above embodiments can execute the medical record analysis methods provided in the above embodiments.

[0183] The same as above concept, Figure 2 The structure of the medical record analysis device shown can be realized as an electronic device. Figure 3 A schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention is shown.

[0184] Exemplarily, the electronic device includes a storage module 21 and a processor 22, the storage module 21 includes instructions loaded and executed by the processor 22, and when the instructions are executed, the processor 22 executes the steps of various exemplary embodiments of the present invention described in the above-mentioned medical record analysis method section of this specification.

[0185] It should be understood that the processor 22 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0186] A computer-readable storage medium is also provided in an embodiment of the present invention. The computer-readable storage medium stores one or more programs. When the one or more programs are executed by a processor, the steps of various exemplary embodiments of the present invention described in the medical record analysis method section are implemented.

[0187] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0188] As is known to those of ordinary skill in the art, the term computer-readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0189] Exemplarily, the computer-readable storage medium may be an internal storage module of the electronic device of the aforementioned embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device.

[0190] The electronic device and computer-readable storage medium provided by the aforementioned embodiments divide each process step of the medical record analysis process into multiple minimum subtasks according to the prompt words by the planning agent, and assign a corresponding task execution agent to each subtask by the planning agent, thereby forming an agent workflow line. The medical record analysis process is executed by the agent workflow line, which not only adapts to the semantics of complex medical contexts, but also improves the efficiency and accuracy of medical record analysis.

[0191] Among them, the intelligent agent can make the large language model module output the execution results of the tasks it undertakes based on external information, the input and output records of the general large language model module in the memory module and the corresponding reflection results, and the knowledge information in the knowledge module, making the large language model module a vertical domain model with the ability to execute tasks in professional fields, thereby improving the cost-effectiveness.

[0192] At the same time, unlike the prior art of processing medical records through large language models, the input and output of each node (agent) in the intelligent agent workflow of this embodiment is transparent, readable and interpretable, thereby ensuring the stability of the output quality.

[0193] In addition, this embodiment automatically trains each task execution agent through the planning agent, and each task execution agent is only responsible for a single task with low complexity, so the training cost of each language model is relatively low.

[0194] In addition, organizing the knowledge base in the form of a knowledge tree improves the retrieval accuracy of the knowledge base. At the same time, considering the characteristics of the medical knowledge tree that it has a shallow hierarchy but many branches, multiple nodes can process the knowledge base branches in parallel, ensuring the efficiency of retrieval.

[0195] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A medical record analysis method, which analyzes a target medical record to be analyzed through a preset medical record analysis process, characterized in that: include: S101, generating prompt words according to the content of the target medical record, the content of the expected output and the requirements, and inputting them into the planning agent; S102, the planning agent divides each process step of the medical record analysis process into a plurality of minimum subtasks according to the prompt words; S103, the planning agent establishes and allocates a corresponding task execution agent for each of the subtasks to form an agent workflow for executing the medical record analysis process, and at the same time, sends prompt word content corresponding to the corresponding subtask to each task execution agent; Each agent includes: A general large language model module is used to output results according to prompt words and reflect on the output results according to reflection instructions; A memory module, used for storing the input and output records of the general large language model module and the corresponding reflection results in text form; The knowledge module is used to store the knowledge information required for the tasks undertaken by the agent; An instruction module, used to generate prompt words according to information input by a user or other intelligent agent, information in a memory module and knowledge information in a knowledge module, and input a reflection instruction into the general large language model module when a reflection instruction is built in, so that the general large language model module outputs a result and then reflects on the result; S104, each task execution agent generates a prompt word through its instruction module according to the prompt word content sent by the planning agent, and inputs the prompt word into its general large language model module to execute the corresponding subtask; Among them, one of the multiple task execution agents for executing multiple minimum subtasks of a process step is used to summarize the execution results of the remaining task execution agents, obtain the execution result of the process step, and send it to the planning agent; S105: The planning agent outputs the execution results of the process steps.

2. A medical record analysis method according to claim 1, characterized in that: The medical record analysis process includes five process steps in sequence: medical record data extraction, key indicator calculation, actual treatment plan extraction, guideline knowledge base treatment plan matching, and treatment plan and knowledge base matching judgment.

3. A medical record analysis method according to claim 2, characterized in that: The agent workflow is optimized and trained by the following steps: S11, taking multiple medical record samples as target medical records respectively, and testing the agent workflow through S101-S105; S12, creating a judgment agent, through which the judgment agent judges whether the execution result of each process step is different from the corresponding reference value, if so, outputs the difference content to the planning agent, otherwise, outputs the perplexity without difference to the planning agent; S13, the planning agent adds, deletes or modifies the task execution agent according to the content of the difference, until the perplexity without the difference meets the preset condition, and the optimization is completed; Wherein, the modification task execution agent includes: The planning agent adjusts the parameters of the general large language model module of the task execution agent, and the planning agent forwards the content of the difference to the task execution agent, which reflects on the output task execution result according to the content of the difference and stores the reflection result in the memory module. When the subtask is executed again, the reflection result is input into its general large language model module as part of the prompt word.

4. A medical record analysis method according to claim 3, characterized in that: The task execution agent reflects on the output task execution result according to the difference content, and stores the reflection result in the memory module, further comprising: Identify and exclude invalid reflection results: Calculate the similarity between the results of the previous and next rounds of reflection. When the similarity exceeds the threshold, the new reflection result is regarded as invalid reflection and is not included in the memory module. When multiple cycles identify similar reflection results, heuristic suggestions are input to the task execution agent.

5. A medical record analysis method according to claim 3, characterized in that: The S12 further comprises: An odd number of judgment agents are used to give judgment results at the same time to determine whether there is a difference. The judgment results of multiple judgment agents are used to determine the final judgment result through a voting mechanism. Based on the Logits Fusion principle, the perplexity output by different judgment agents is merged: If the judgment results of the odd number of judgment agents are the same, then the perplexity output by the odd number of judgment agents is combined using formula (1): （1） Where n is the number of judging agents, PPi is the perplexity value of the i-th agent; If some of the judgment results of the odd number of judgment agents are the same and some are different, then the maximum perplexity among all the judgment agents whose judgment results are the same is taken as Each judgment result is different from the perplexity of the judgment agent. , and then combine the perplexities output by the odd number of judgment agents through formula (1); Among them, m is the number of judging agents whose judgment results are indifferent.

6. A medical record analysis method according to claim 2, characterized in that: The instruction module is further used to input the sorting instruction into the memory module after the general large language model module outputs the task result when the sorting instruction is built in. The memory module is also used to sort the input and output records after receiving the sorting instruction: When the cumulative amount of new input and output records reaches a first threshold, all new input and output records and old input and output records are summarized to form one record as the old input and output record. At this time, the number of old input and output records is increased by 1, and at the same time, all new input and output records are deleted; When the number of old input and output records reaches a second threshold, the earliest preset forgotten number of records among all old input and output records are forgotten.

7. A medical record analysis method according to claim 2, characterized in that: Decomposing the contents of the knowledge base used in the process steps into a knowledge tree, and assigning a task execution agent to each node of the knowledge tree, wherein the knowledge module and the memory module of the task execution agent respectively store the knowledge text of the corresponding node and the summary and keywords of the knowledge text; When performing knowledge retrieval on the knowledge base, first find multiple target nodes that match the search instructions according to the abstract and keywords, and then use multiple task execution agents corresponding to the multiple target nodes to perform parallel calculations to determine the correctness of the knowledge content references, and then summarize and report the correctly referenced knowledge content layer by layer to form the final knowledge retrieval results.

8. A medical record analysis device, which analyzes a target medical record to be analyzed through a preset medical record analysis process, characterized in that: include: The first unit is used for S101, generating prompt words according to the content of the target medical record, the content of the expected output and the requirements, and inputting them into the planning agent; The second unit is used for S102, wherein the planning agent divides each process step of the medical record analysis process into a plurality of minimum subtasks according to the prompt word; The third unit is used for S103, wherein the planning agent establishes and allocates a corresponding task execution agent for each of the subtasks to form an agent workflow for executing the medical record analysis process, and at the same time, sends the prompt word content corresponding to the corresponding subtask to each task execution agent; Each agent includes: A general large language model module is used to output results according to prompt words and reflect on the output results according to reflection instructions; A memory module, used for storing the input and output records of the general large language model module and the corresponding reflection results in text form; The knowledge module is used to store the knowledge information required for the tasks undertaken by the agent; An instruction module, used to generate prompt words according to information input by a user or other intelligent agent, information in a memory module and knowledge information in a knowledge module, and input a reflection instruction into the general large language model module when a reflection instruction is built in, so that the general large language model module outputs a result and then reflects on the result; The fourth unit is used for S104, each task execution agent generates a prompt word through its instruction module according to the prompt word content sent by the planning agent, and inputs the prompt word into its general large language model module to execute the corresponding subtask; Among them, one of the multiple task execution agents for executing multiple minimum subtasks of a process step is used to summarize the execution results of the remaining task execution agents, obtain the execution result of the process step, and send it to the planning agent; The fifth unit is used for S105, wherein the planning agent outputs the execution results of the process steps.

9. An electronic device, characterized in that: It comprises a storage module, wherein the storage module comprises instructions loaded and executed by a processor, and when the instructions are executed, the processor executes a medical record analysis method according to any one of claims 1-7.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by a processor, a medical record analysis method as described in any one of claims 1 to 7 is implemented.

Images