A method for interpreting laboratory tests in physical examination reports based on a hybrid expert model

Abstract

The present invention discloses a method for interpreting laboratory examinations in a physical examination report based on a hybrid expert model, and relates to the technical field of semantic analysis; the method comprises the following steps: step S1: constructing and obtaining a physical examination laboratory examination question and answer data set and a physical examination report laboratory examination field knowledge map; step S2: constructing and obtaining a hybrid expert model based on multi-round dialogues; step S3: constructing and obtaining a Chinese medical multi-round dialogue instruction data set, and fine-tuning the hybrid expert model based on the Chinese medical multi-round dialogue instruction data set and a hybrid low-rank adaptive algorithm to obtain a fine-tuned hybrid expert model; step S4: fine-tuning the fine-tuned hybrid expert model prompt based on a thinking chain to obtain a hybrid expert model after prompt fine-tuning; step S5: inputting the physical examination report into the hybrid expert model after prompt fine-tuning to obtain a laboratory examination interpretation result. Through steps S1 to S5, etc., the auxiliary interpretation of the laboratory examination of the physical examination report is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic analysis, and in particular to a method for interpreting laboratory examinations in physical examination reports based on a hybrid expert model. Background Art

[0002] In 1991, Schmidhuber published a paper in Neural Computation, proposing a fast weight programmer that uses a feedforward neural network and learns through gradient descent, which is equivalent to a linearized self-attention Transformer. In 2017, Google and the University of Toronto jointly proposed the original Transformer architecture consisting of an encoder and a decoder, laying the foundation for the subsequent development of large Transformer models. In 2020, OpenAI launched GPT-3, demonstrating excellent language comprehension capabilities. In March 2023, OpenAI's latest ultra-large-scale multimodal pre-trained large model GPT-4 has multimodal understanding and multi-type content generation capabilities. In June 2023, Baichuan Intelligence launched Baichuan-7B, the first Chinese-English language model with 7 billion parameters, which introduced large models into Chinese language scenarios, opened a new era of Chinese large models, and made up for the shortage of high-quality open source commercial models in China. In September of the same year, Baichuan 2 was released. In January 2024, Baichuan Intelligence released Baichuan 3, and built a medical data set of more than 100 billion tokens for training. Therefore, Baichuan 3 has a good level of medical knowledge and performs well in medical question answering and medical reasoning.

[0003] The above-mentioned large language models already have strong natural language understanding capabilities, and general models have also been launched in the field of Chinese medicine. However, in order to meet the different needs of multiple fields and multiple application scenarios, the scale and complexity of large models will continue to increase, and the sparse characteristics of the hybrid expert model MOE are needed to improve model training efficiency and reduce inference costs. In 1991, Michael Jordan, a big shot in the field of machine learning, and Geoffrey Hinton, the godfather of artificial intelligence, first proposed the hybrid expert model MoE. In 2017, Google proposed the use of hybrid expert models in the field of natural language processing, adding hybrid expert networks between long and short-term memory neural network layers, and achieved a thousand-fold performance improvement in machine translation. In 2020, Gshard introduced MoE technology into the Transformer architecture and provided an efficient distributed parallel computing architecture. In 2021, Google successively proposed SwitchTransformer and GLaM, which simplified the routing strategy of the hybrid expert model, further improved training efficiency, and saved computing time and cost.

[0004] The large hybrid expert model still needs fine-tuning under specific tasks to better adapt to the language environment of the professional field and align with human instructions. In June 2019, Tencent proposed to optimize the model through multiple rounds of dialogue to deepen the model's understanding of human intentions and promote specific tasks. In April 2021, Hu et al. proposed prompt tuning, and proposed to improve the effect of pre-trained models in zero-sample and small-sample scenarios by introducing external knowledge construction prompts. In June of the same year, Hu et al. proposed a low-rank adaptive re-parameterization efficient fine-tuning algorithm, which freezes the pre-trained model weights and injects the trainable rank decomposition matrix into each layer of the Transformer architecture, thereby greatly reducing the number of trainable parameters for downstream tasks, achieving the purpose of saving training costs and improving training efficiency. In January 2023, Google proposed thinking chain fine-tuning, inputting prompts with reasoning processes into large models, improving the ability to handle complex reasoning problems, and enabling the model to learn to think autonomously. In July 2023, Ghosa et al. proposed LoRA instruction fine-tuning, which means fine-tuning a large language model in a supervised manner by setting instructions. Compared with soft-prompt, the most obvious advantage is that it does not occupy the length of the input token and can help the model generalize to unseen tasks. In September 2023, Zadouri et al. first proposed a hybrid low-rank adaptive algorithm that combines efficient fine-tuning and hybrid expert models, and proposed efficient fine-tuning strategies for two hybrid expert models, MoV and MoLORA.

[0005] In summary, the development of large language models is currently in a period of rapid growth. Many large models have demonstrated excellent natural language understanding capabilities. Some large models have begun to focus on learning knowledge in the medical field, but there are no outstanding research results in the interpretation of chemical laboratory tests in medical examination reports. Summary of the invention

[0006] The present invention provides a method for interpreting laboratory tests in physical examination reports based on a hybrid expert model, thereby solving a technical problem.

[0007] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0008] A method for interpreting laboratory examinations in physical examination reports based on a hybrid expert model comprises the following steps: step S1: constructing a physical examination laboratory examination question and answer dataset and a physical examination report laboratory examination domain knowledge graph; step S2: constructing a hybrid expert model based on multi-round dialogue; step S3: constructing a Chinese medical multi-round dialogue instruction dataset, and fine-tuning the hybrid expert model based on the Chinese medical multi-round dialogue instruction dataset and a hybrid low-rank adaptive algorithm to obtain a fine-tuned hybrid expert model; step S4: fine-tuning the fine-tuned hybrid expert model prompt based on a thinking chain to obtain a hybrid expert model after prompt fine-tuning; step S5: inputting the physical examination report into the hybrid expert model after prompt fine-tuning to obtain the laboratory examination interpretation result.

[0009] A further technical solution is: in step S1, based on common problems in physical examination report laboratory tests, combined with open source Chinese medical data sets and medical laboratory professional title examination knowledge base, a physical examination laboratory test question and answer data set and a physical examination report laboratory test field knowledge graph are constructed.

[0010] A further technical solution is: in step S2, according to the five categories of physical examination laboratory examinations, five sub-expert models are constructed, a gated network is set, and a multi-round dialogue module is added to construct a hybrid expert model of expert network + gated network + multi-round dialogue module. The expert network includes five sub-expert models: routine examination expert, biochemical examination expert, immunological examination expert, microbiological examination expert and molecular biology examination expert.

[0011] A further technical solution is that in step S2, the gated network includes an expert activation module and an aggregation module, and the multi-round dialogue module includes four modules, namely, a natural language understanding module NLU, a dialogue state tracking module DST, a strategy learning module PL and a natural language generation module NLG.

[0012] A further technical solution is: in step S3, the physical examination laboratory examination question and answer dataset and the physical examination report laboratory examination field knowledge graph constructed in step S1 are combined with the existing open source Chinese medical dataset to form a multi-round dialogue command dataset in the medical field; a hybrid low-rank adaptive algorithm is used to fine-tune the hybrid expert model obtained in step S2.

[0013] A further technical solution is: in step S5, the indicators and values ​​in the PDF or picture report uploaded by the examinee are identified, and after data cleaning and preprocessing, they are compared with the structured knowledge in the knowledge base as a basis for quantitative diagnosis; the information on medical history and living habits obtained from multiple rounds of conversations is combined to obtain diagnostic results and treatment recommendations.

[0014] A further technical solution is: it also includes step S6 effect evaluation after step S5, step S6: using BLEU and ROUGE indicators, respectively, from the two dimensions of precision Precision and recall Recall, to evaluate the quality of the laboratory examination interpretation results output by the hybrid expert model.

[0015] The beneficial effects of adopting the above technical solution are:

[0016] A method for interpreting laboratory examinations in physical examination reports based on a hybrid expert model includes the following steps: step S1: constructing a physical examination laboratory examination question and answer dataset and a physical examination report laboratory examination domain knowledge map; step S2: constructing a hybrid expert model based on multi-round dialogues; step S3: constructing a Chinese medical multi-round dialogue instruction dataset, fine-tuning the hybrid expert model based on the Chinese medical multi-round dialogue instruction dataset and a hybrid low-rank adaptive algorithm to obtain a fine-tuned hybrid expert model; step S4: fine-tuning the fine-tuned hybrid expert model prompt based on a thinking chain to obtain a hybrid expert model after prompt fine-tuning; step S5: inputting the physical examination report into the hybrid expert model after prompt fine-tuning to obtain the laboratory examination interpretation result. Through steps S1 to S5, etc., the auxiliary interpretation of laboratory examinations in physical examination reports is realized.

[0017] Please refer to the detailed description of the specific implementation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a data flow diagram of the present invention;

[0019] Figure 2 It is a knowledge graph in the field of laboratory examination of physical examination report of the present invention;

[0020] Figure 3 It is a principle block diagram of the multi-round dialogue module of the present invention;

[0021] Figure 4 It is a data flow graph of the present invention based on fine-tuning of hybrid low-rank adaptive multi-round dialogue instructions;

[0022] Figure 5 It is a data flow diagram of the present invention based on the fine-tuning of the thought chain prompt;

[0023] Figure 6 yes Figure 2 Knowledge map of liver function in China;

[0024] Figure 7 yes Figure 6 A knowledge graph with multiple rounds of question and answer can be added. DETAILED DESCRIPTION

[0025] This application is based on a method that combines multi-round dialogues with a hybrid expert model to perform multi-strategy fine-tuning, aiming to professionally and accurately interpret the contents of laboratory tests in test reports and to complete the task of recommending treatment plans.

[0026] Physical examinations usually include physical examinations, chemical laboratory tests, and imaging examinations. For chemical laboratory tests, we have compiled a professional Q&A dataset and a special knowledge graph for laboratory tests based on medical expertise and common questions in daily consultations, centering on 25 examination items in five categories: routine, biochemical, immunological, microbiological, and molecular biology. A hybrid expert model architecture is adopted, covering five experts, namely, routine examination experts, biochemical examination experts, immunological examination experts, microbiological examination experts, and molecular biology examination experts. A multi-round dialogue module is added on the basis of the hybrid expert model to obtain multi-dimensional information. The newly created knowledge and the existing mature knowledge base are applied, and the model is tuned by fine-tuning the multi-round dialogue instructions based on hybrid low-rank adaptation + fine-tuning of thinking chain prompts, aiming to deepen the model's understanding of physical examination laboratory tests and better adapt to interpretation tasks in this field. For the physical examination reports input by the examinees, the PaddleOCR model library is used to detect and recognize PDFs and images, perform structured processing on the data, extract each indicator and the corresponding test information, and input the identified information and user query questions into the tuned hybrid expert model. The gated network dispatches the input to the corresponding experts, compares it with the normal values ​​of the indicators, and obtains multi-dimensional information through multiple rounds of dialogue, thereby completing the tasks of inferring diagnostic test results and outputting treatment recommendations, thus realizing the auxiliary interpretation of laboratory tests in physical examination reports.

[0027] The algorithm steps of this application are as follows: Based on the theme of laboratory examinations in physical examination reports, self-built professional data sets and knowledge graphs are combined with existing open source data sets. The hybrid expert model is optimized through multi-round dialogue command fine-tuning + thinking chain prompt fine-tuning. The physical examination report result information recognized by OCR and the examinee's inquiries are input into the optimized model together, and the final answer is output through a majority voting strategy.

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0030] Embodiment 1:

[0031] like Figure 1 to Figure 5 As shown, the present invention discloses a method for interpreting laboratory examinations in physical examination reports based on a hybrid expert model, comprising the following steps:

[0032] like Figure 1 As shown, this is the overall framework diagram for assisting the interpretation of laboratory tests in physical examination reports based on the above steps, that is, the overall framework diagram for assisting the interpretation of laboratory tests in physical examination reports based on a hybrid expert model with multi-round dialogues.

[0033] Step S1: Construct a physical examination laboratory test question and answer dataset and a knowledge graph in the field of physical examination report laboratory tests.

[0034] like Figure 2 As shown, the knowledge graph sample in step S1 is a demo display diagram of the knowledge graph in the field of physical examination report laboratory examination.

[0035] Based on the common problems in physical examination report laboratory tests, combined with the current open source Chinese medical data set and the medical laboratory professional title examination knowledge base, a physical examination laboratory examination question and answer data set and a physical examination report laboratory examination knowledge graph were constructed.

[0036] Physical examination laboratory tests mainly include five categories: routine examination, biochemical examination, immunological examination, microbiological examination and molecular biology examination. This application is subdivided into 25 examination items based on the five categories of examinations, as shown in Table 1 for details.

[0037] Table 1: 25 inspection items in five categories

[0038]

[0039] According to the 25 examination items in five categories in Table 1, professional doctors in tertiary hospitals compiled a professional question-and-answer dataset for physical examination and laboratory examination based on daily consultations and basic knowledge of clinical medicine and laboratory medicine, which contains a total of 213 items. An example is shown below.

[0040] Question: My alanine aminotransferase is 68. Do I need treatment?

[0041] Answer: Alanine aminotransferase is slightly elevated, not more than 2 times the upper limit of normal value. If there are no symptoms such as abdominal pain, nausea, loss of appetite (combined with the ultrasound / hepatitis B test report of this physical examination, there are no abnormalities), you can observe temporarily and ask whether there are behaviors such as staying up late for a long time, fatigue, excessive drinking, eating too greasy food, and using certain liver-damaging drugs recently. Check liver function again after improving the above conditions for 2 weeks; if alanine aminotransferase continues to increase, it means that there are other causes of liver damage, and you should go to a gastroenterology specialist for diagnosis and treatment.

[0042] After manually compiling the question-and-answer dataset, we used open source Chinese medical question-and-answer datasets and medical laboratory professional title examination knowledge base to expand the knowledge and build a complete medical physical examination laboratory examination benchmark dataset of over 9 million items. The open source Chinese medical question-and-answer datasets include MedDialog-CN, IMCS-V2, CHIP-MDCFNPC, MedDG, cMedQA2, and Chinese-medical-dialogue-data.

[0043] like Figure 2 As shown in the figure, after the question and answer dataset is compiled, professional doctors in tertiary hospitals build a knowledge graph in the field of laboratory examinations in physical examination reports based on their knowledge reserves and clinical consultation experience, combined with the above-compiled question and answer dataset, and referring to the currently publicly available Chinese medical knowledge graph CMeKG.

[0044] like Figure 6 Shown is the knowledge graph of liver function.

[0045] like Figure 7 As shown, a knowledge graph can be added with multiple rounds of question and answer.

[0046] Step S2: Construct a hybrid expert model based on multi-round dialogues.

[0047] like Figure 3 As shown, the task-oriented multi-round dialogue architecture diagram, the multi-round dialogue architecture in step S2.

[0048] According to the five categories of physical examination laboratory examinations, five sub-expert models are constructed, a gating network is set to determine the expert group activated by the examinee's input, a multi-round dialogue module is added for key information acquisition and human-computer interaction, and a hybrid expert model of expert network + gating network + multi-round dialogue module is constructed.

[0049] Physical examination laboratory tests mainly include five categories and 24 different examination items. In order to improve the professionalism of the large model in interpreting various examinations and reduce the difficulty and reasoning cost of large-scale data training, this application adopts a hybrid expert architecture to better obtain the examination results and key diagnostic information of the physical examination report, and add a multi-round dialogue module. The hybrid expert model consists of three parts, namely the expert network, the gated network and the multi-round dialogue module, which are described in detail below.

[0050] Step S201: construct an expert network.

[0051] According to the five major categories of physical examination chemical laboratory tests, five sub-expert models are constructed to obtain routine examination experts, biochemical examination experts, immunological examination experts, microbiological examination experts and molecular biology examination experts. The five sub-expert models form an expert network, and each sub-expert model serves as an expert for each examination item.

[0052] Step S202: Obtain a gated network.

[0053] For the examinee's questions, it is necessary to selectively input the expert model for training, so the gating network is introduced to route the examinee's questions to the required n expert models. The gating network consists of two modules, namely the expert activation module and the aggregation module.

[0054] For the expert activation module, that is, the gating network chooses which expert to trust when receiving user input. For example, when the physical examination report and inquiry content of the physical examinee are blood sugar and blood lipids, they are biochemical indicators and should be routed to the biochemical examination expert for training and output. When the input content includes blood routine and blood sugar and blood lipids, it involves two indicators, routine examination and biochemical examination. Therefore, two models of routine examination experts and biochemical examination experts are selected to be responsible for the interpretation of the physical examination report. Because the laboratory examination of the physical examination report requires accurate and detailed interpretation, and the hybrid expert network in this application only involves five expert models, in order to avoid information loss, the result of Softmax gating is directly used as the basis for activating each expert, and the gating mechanism of the first k experts is no longer used. The expert is Top K. The Softmax gating network multiplies the input by the trainable weight matrix W g , and then apply the Softmax function, expressed as formula (1).

[0055] G(x) = Softmax(x·W g )(1)

[0056] In formula (1), G(x) is the activation probability of the gated network output, x is the input, and W g is the weight matrix.

[0057] The aggregation module, namely the aggregation mechanism PoolingMethod, uses a weighted sum fusion strategy to make predictions based on the above expert activation status and expert output results, and outputs the final result, which is expressed as formula (2).

[0058]

[0059] In formula (2), x is the input, y is the input, G(x) i is the activation probability of the i-th expert output by the gating network, E i (x) is the output of the i-th expert.

[0060] Step S203: Obtain a multi-round dialogue module.

[0061] In order to make a more accurate diagnosis of laboratory test results, it is necessary to further understand the medical history, symptoms, living habits, etc. At the same time, the examinee often needs to further inquire about the answers. Therefore, it is necessary to set up a multi-round dialogue module. On the one hand, it promotes the model to obtain key information, and on the other hand, it increases the model's memory of historical dialogues and promotes the interaction between the examinee and the model.

[0062] Multi-round dialogues are divided into task-oriented dialogues TOD and open domain dialogues ODD. This application has specific tasks in the medical field, and requires the model to interpret the laboratory test items in the physical examination report. Therefore, a task-oriented multi-round dialogue mechanism is used. The multi-round dialogue module includes four modules: natural language understanding module NLU, dialogue state tracking module DST, policy learning module PL and natural language generation module NLG. The dialogue state tracking module and policy learning module are the core modules of the task-oriented multi-round dialogue module. The combination is called the dialogue manager DM, which is responsible for the overall control of the multi-round dialogue.

[0063] Natural language understanding module: This module mainly extracts the examinee's question intention through understanding natural language and fills in the slots for key information. For example, the examinee's question is: My alanine aminotransferase is 90, is it serious? What should I do? The question intention obtained is to diagnose the severity of symptoms + give treatment suggestions. To answer this question, set the slots: physical examination result value, medical history, and living habits. The physical examination value is filled in in the first round of dialogue: 90.

[0064] Dialogue state tracking module: Based on the intent identified by the natural language understanding module and the filled alanine aminotransferase check value, it tracks the dialogue state of the current round in combination with the historical input dialogue.

[0065] Strategy learning module: Based on the learned medical knowledge, the judgment conditions of laboratory test results, and the current dialogue status output by the joint dialogue tracking module, the next action strategy is derived in order to complete the task of diagnosing the test results and giving treatment recommendations. That is, the key information that the model should obtain in the future to complete the task.

[0066] Natural language generation module: Gives a preliminary answer based on the current information and asks the examinee questions about unfilled slots. For example: The transaminase level is 2 times higher than the normal value. Do you have a history of viral hepatitis (hepatitis B or hepatitis C)?

[0067] The above are the four modules of multi-round dialogue, and the first round of dialogue is demonstrated using a biochemical indicator as an example. Next, the examinee answers the questions of the large model and raises new questions of his own. The multi-round dialogue module continuously identifies the examinee's inquiry intention to update the slots, and adopts a task-oriented inquiry strategy to obtain key information to fill the slots. Finally, by filling in the complete slot information, combined with the normal range of laboratory examination indicators and judgment basis, the examinee is given various suggestions on diagnosis, treatment methods, etc.

[0068] Step S3: Construct a Chinese medical multi-round dialogue command dataset, and fine-tune the hybrid expert model based on the Chinese medical multi-round dialogue command dataset and a hybrid low-rank adaptive algorithm to obtain a fine-tuned hybrid expert model.

[0069] like Figure 4 As shown, the fine-tuning process in step S3 is performed based on hybrid low-rank adaptive multi-round dialogue instruction fine-tuning.

[0070] The physical examination laboratory examination question and answer dataset and the physical examination report laboratory examination domain knowledge graph constructed in step S1 are combined with the existing open source Chinese medical dataset to form a multi-round dialogue command dataset in the medical field. A hybrid low-rank adaptive algorithm is used to fine-tune the key layers of the hybrid expert model obtained in step S2, so that the hybrid expert model has a better understanding of the examinee's inquiry instructions and adapts to multi-round dialogue scenarios in the medical field.

[0071] Hybrid low-rank adaptive algorithm, the research community began to combine the PEFT method represented by LoRA with the MoE framework, and proposed new PEFT methods such as MoV, MoLORA, LoRAMOE and MOLA, which further improved the probability of fine-tuning large models compared to the original version of LORA.

[0072] For the above-mentioned multi-round dialogue hybrid expert model, a multi-round inquiry dialogue command dataset is used to perform low-rank adaptive LoRA multi-round dialogue command fine-tuning to make it more suitable for medical question-answering scenarios and have the ability to assist in the interpretation of physical examination reports and laboratory tests in the Chinese context.

[0073] Step S301: Construct and obtain a Chinese medical multi-round dialogue instruction dataset.

[0074] Using the self-built benchmark dataset in step S1, 213 manually compiled question-answer pairs + over 9 million open source Chinese medical question-answer pairs, after data cleaning and preprocessing, a Chinese medical multi-round dialogue instruction dataset was constructed and used for fine-tuning. The data volume is 9 million + 213, and the average number of inquiry dialogue rounds is 3.

[0075] An example of the Chinese medical multi-turn dialogue command dataset is as follows:

[0076] Instructions: As a doctor specializing in physical examinations, please compare the biochemical test results of the examinee with normal values, conduct multiple rounds of dialogue to obtain effective information, and give diagnosis and treatment recommendations.

[0077] Physical examiner: My alanine aminotransferase is elevated, what’s going on?

[0078] Large model: Alanine aminotransferase is one of the sensitive indicators reflecting liver function. Increased alanine aminotransferase indicates liver damage.

[0079] Physical examiner: What is the cause of the increased alanine aminotransferase?

[0080] Large model: There are many reasons for the increase of alanine aminotransferase, namely diseases that cause liver damage, such as various acute and chronic viral hepatitis, cirrhosis, liver cancer, etc., as well as staying up late, fatigue, excessive drinking, use of various drugs, etc., resulting in fatty liver, alcoholic liver disease, drug-induced hepatitis, etc., which may cause an increase in alanine aminotransferase.

[0081] Physical examinee: My alanine aminotransferase is over 200, what should I do?

[0082] Large model: Usually, an increase of 2 times the upper limit of normal for alanine aminotransferase indicates severe liver damage, and it is necessary to go to a gastroenterology specialist for diagnosis and treatment in time to find out the cause and provide treatment. Do you have a history of viral hepatitis (hepatitis B or hepatitis C)?

[0083] Examiner: No.

[0084] Big model: Do you have a history of liver cirrhosis?

[0085] Physical examiner: Neither do I.

[0086] Large model: Do you drink alcohol regularly? There are several options below, please choose according to your actual situation.

[0087] a. Regular drinking history for more than 5 years, with an average daily alcohol consumption of more than 100 ml (male) / 50 ml (female).

[0088] b. Regular drinking history for more than 5 years, with an average daily red wine consumption of more than 350-400 ml (male) / 125-200 ml (female).

[0089] c. A history of heavy drinking in the past two weeks, with an average daily consumption of more than 200 ml of liquor or 700-800 ml of red wine.

[0090] d. Drink alcohol occasionally or in small amounts or not at all.

[0091] Physical examinee: d Drink alcohol occasionally or in small amounts or do not drink alcohol.

[0092] Large model: OK, have you taken any medicines in the past three months? There are several options below, please choose according to your actual situation.

[0093] a Acetaminophen (paracetamol).

[0094] bAnti-tuberculosis drugs.

[0095] cCyclophosphamide / Cyclosporin / Methotrexate and other chemotherapy drugs

[0096] d. Traditional Chinese medicine with unknown ingredients.

[0097] eNo special medication.

[0098] Subject: d. Taking Chinese medicine, the ingredients are unknown.

[0099] Large model: You cannot rule out drug-induced liver damage at this time. It is recommended that you go to the gastroenterology department for diagnosis and treatment in time.

[0100] Step S302: fine-tune the hybrid expert model based on the Chinese medical multi-round dialogue instruction dataset and the hybrid low-rank adaptive algorithm to obtain a fine-tuned hybrid expert model.

[0101] After the high-quality multi-round dialogue command dataset is constructed, the hybrid low-rank adaptive algorithm MoLoRA is applied for fine-tuning.

[0102] Low-rank adaptive fine-tuning adds small and low-rank matrices to the key layers of the model to adjust the model parameters, which freezes the original parameters of the pre-trained model, efficiently trains low-dimensional weight parameters through re-parameterization, and finally adds the parameters. The large language model contains many layers, and the weight matrix of each layer is generally full-rank. When adapting to specific tasks in the specified field, low-dimensional subspaces can be used for learning. Similarly, this application fine-tunes the model in the medical field, and weight updates can also be performed in low-dimensional subspaces. Taking one of the pre-trained expert models as an example, the weight matrix is ​​expressed as Keep frozen; update the matrix representation of the pre-trained model weights as Gradient updateable.

[0103] The low-rank adaptive algorithm can be expressed as:

[0104] W0+ΔW=W0+BA (3)

[0105] In formula (3), W0 is the weight matrix, ΔW is the matrix for updating the weight of the pre-trained model, r《min(d,k).

[0106] The weights of the original low-rank adaptive fine-tuning adapter are dense, and each sample will activate and update all parameters in LoRA. Here, the hybrid low-rank adaptive fine-tuning module MoLoRA, MoE+LoRA is a combination of low-rank adaptive fine-tuning and hybrid expert model. The hybrid expert model contains five expert models, namely E1, E2, E3, E4, E5 and a gating network R, and configures five LoRAs, namely L1, L2, L3, L4, and L5, as lightweight trainable experts for the five expert models. The gating network uses a token-level soft routing mode, and acts on LoRA in parallel according to the weight assigned to each expert model, determines the activation probability of each LoRA module, and then sums the fine-tuned results.

[0107]

[0108] In formula (4), E mix is the parameter matrix after MoLoRA fine-tuning, w i is the weight of the activated expert, E i Pre-train parameters for each expert model, L i is a trainable low-dimensional parameter matrix.

[0109] y=E mix (x)(5)

[0110] In formula (5), x is the model input and y is the model output.

[0111] The above-mentioned fine-tuning can deepen the model's understanding of physical examination laboratory examinations. The instructions enhance the model's understanding of the examinee's diverse inputs, and multiple rounds of dialogue make the model more adaptable to the interaction between the person and the intelligent agent.

[0112] Step S4: Prompt fine-tuning the fine-tuned hybrid expert model based on the thought chain to obtain the fine-tuned hybrid expert model.

[0113] like Figure 5 As shown, the prompt fine-tuning based on the thinking chain, the fine-tuning process in step S4.

[0114] Using the thinking chain prompt template built by professional doctors + the question bank of medical examination-related professional qualification examinations, the thinking chain automatic generation framework is applied to generate a prompt fine-tuning dataset, and the hybrid expert model after the instruction fine-tuning obtained in step S3 is fine-tuned for the thinking chain prompt, aiming to enable the model to learn rich reasoning paths and have thinking ability.

[0115] After fine-tuning in step S3, the hybrid expert model, i.e., the language model, has experience in multiple rounds of dialogue in the field of medical physical examination laboratory examinations, but it still lacks thinking ability. When faced with problems that require multi-step reasoning, humans usually think about or write out the steps to solve the problem, and obtain the required information from the questions or the outside world during the problem-solving process. Therefore, by analogy with human thinking and problem-solving behavior, we add prompts based on the thinking chain to fine-tune the model, so that it has logical reasoning skills and can then output answers through thinking.

[0116] Professional clinicians from tertiary hospitals focus on five aspects: task description, small sample example prompts, context, role positioning, and answer tone. Set up the content template of the thinking chain prompt, see Table 2.

[0117] Table 2: Thinking chain prompt content template

[0118]

[0119] According to the above template, based on the 213 question and answer data sets manually compiled by the tertiary hospitals in step S1, 16 reasoning examples are compiled. In order to expand the richness of laboratory test reasoning, the knowledge base of the examination questions + answer analysis of the professional title examination "Clinical Biochemistry Laboratory" and the intermediate and senior professional title examination "Clinical Biochemistry and Biochemistry Laboratory" in clinical medicine laboratory is used as the basis. Through the Auto-CoT framework, the model learns from various questions and automatically constructs 50 examples with questions + reasoning chains, providing rich reasoning chains, and covering the reasoning methods of laboratory test results more comprehensively. A total of 66 example sets are generated through the two methods, 16+50=66, which are input into the hybrid expert model together with the questions asked by the examiner, and act on the input embedding InputEmbedding module to fine-tune the thinking chain.

[0120] There are differences in the thought chain examples written by different doctors, and the answer parsing process of different question banks is also different. Therefore, when the examinee inputs a question, the hybrid expert model learns different solution ideas, samples and generates multiple thought chains from the model decoder, and then outputs multiple interpretations. Different thought chains are represented by r i , different answers obtained according to different thinking chains are expressed as α i ∈Α, different reasoning paths and answer combinations are represented as (r i ,α i), following the principle of self-consistency, two fusion strategies are designed to determine the final answer y.

[0121] Majority voting strategy: The answer with the most votes or the relative majority is selected as the final answer, as shown in formula (6).

[0122]

[0123] In formula (6), y is the model output, α i is the output of the i-th thinking chain, and a is the output answer value.

[0124] Weighted summation strategy: The weighted sum of each answer is taken as the final answer, as shown in formula (8).

[0125]

[0126] In formula (7), P(r i ,α i |prompt,question) is the weight of the ith answer, K is the number of thought chains, P(t k prompt,question,t1,t2,L,t k-1 ) is the probability of getting the kth answer given the question, prompt, and output of the first k-1 answers. It is a conditional probability.

[0127]

[0128] In formula (8), y is the final answer and m is the number of output answers.

[0129] In the validation set performance, the two fusion strategies have comparable accuracy, and the majority voting method is slightly better than the weighted sum method. Therefore, the majority voting strategy is used as the criterion for selecting the final answer.

[0130] After fine-tuning the thinking chain, the hybrid expert model can reason and think according to the prompted problem-solving ideas and steps when faced with new question input from the examinee, which greatly improves the reasoning effect and thus improves the accuracy and professionalism of the model's diagnosis results.

[0131] Step S5: Input the physical examination report into the hybrid expert model after prompt fine-tuning to obtain the laboratory examination interpretation results.

[0132] PaddleOCR recognition technology is used to identify key indicators and values ​​of PDF and image reports uploaded by the examinee. After data cleaning and preprocessing, it is compared with the structured knowledge in the knowledge base as a basis for quantitative diagnosis. Combined with information such as medical history and living habits obtained through multiple rounds of conversations, diagnosis results and treatment recommendations are obtained through integration.

[0133] When the examinee inquires about the laboratory test results of the physical examination report, some of them will convert the results into text and numerical values ​​and send them directly, while others will directly send the physical examination report PDF or picture. Based on Baidu's open source ultra-lightweight OCR model library PaddleOCR, we adapt the physical examination report layout and content presentation to recognize the physical examination report. The recognition process is as follows:

[0134] Step S501: text detection.

[0135] The layout format of the physical examination report is analyzed, and the location of the text information in the physical examination report is detected based on the image segmentation scheme through the differentiable binarization algorithm DifferentiableBinarization (DB for short). The title and pictures in the report as well as the table name, column name and value areas in the list are marked with borders.

[0136] Step S502: text recognition.

[0137] According to the text area marked by the detected border, it is extracted as text and numerical values. Specifically, the Convolutional Recurrent Neural Network (CRNN) is used to input the text box marked in the text detection stage and output the recognized text, numerical value and symbol content. Then, the Connectionist Temporal Classification algorithm (CTC) is used to divide the above output text content into multiple time periods to solve the problem that the input sequence and the input sequence are of different lengths and difficult to align. It proposes a new alignment method, which divides the output sequence into multiple time periods and realizes sequence alignment by formulating certain processing rules. It does not need to be strictly aligned in length, thereby solving the problem of difficulty in alignment caused by different lengths of input and output sequences.

[0138] Step S503: Structural processing.

[0139] The identified content is converted into an Excel document, which contains inspection items and test values, so as to compare them with normal values ​​in the knowledge base and draw quantitative conclusions. At the same time, the identified results can also be added to Excel to improve the objectivity and accuracy of the analysis.

[0140] The information after the above identification and comparison is used as the basis for quantification. Based on the questions asked by the examinee, key information is obtained through task-oriented multi-round dialogues. The examinee interacts flexibly with the model and uses the thinking chain method to conduct reasoning and thinking, and finally outputs professional answers.

[0141] Embodiment 2:

[0142] The present invention discloses a method for interpreting laboratory tests in a physical examination report based on a hybrid expert model, based on steps S1 to S5 of embodiment 1, further comprising the following steps:

[0143] Step S6: Effect evaluation.

[0144] The BLEU and ROUGE indicators were used to evaluate the quality of the laboratory examination interpretation results output by the hybrid expert model from the two dimensions of precision and recall.

[0145] The laboratory test results output by the large model were evaluated using two indicators: Bilingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE). BLEU measures the quality of interpretation based on precision, while ROUGE measures the quality of interpretation based on recall.

[0146] BLEU mainly evaluates the similarity between the answers output by the large model and the answers given by professional doctors, focusing on the accuracy of the generated responses. The higher the BLEU value, the lower the probability that the model will hallucinate and fabricate false knowledge, and the more accurate and professional the answers will be. The formula is as follows:

[0147]

[0148] In formula (9), BLEU is the similarity between the answer output by the large model and the answer of the professional doctor, n is n consecutive words or characters, n-gram is a sequence of n consecutive words or characters, P n Indicates the matching degree, W n It represents the weight of each n-gram, and BP is a penalty factor, which is obtained by calculating the length difference between the output of the large model and the answer of the professional doctor, as shown below.

[0149]

[0150] In formula (10), l c is the length of the model output, l s The length of the standard answer for professional doctors.

[0151] ROUGE mainly evaluates whether the answer output by the big model contains the key information of the professional doctor's answer, focusing on the completeness of the generated response. The higher the ROUGE value, the smaller the probability that the big model misses important knowledge and the more comprehensive the answer. The formula is as follows.

[0152]

[0153] In formula (11), R lcs It is the ratio of the length of the common subsequence to the length of the doctor's interpretation answer, which is a recall probability. X is the answer output by the large model, Y is the interpretation answer of the professional doctor, and LCS(X,Y) is the length of the longest common subsequence.

[0154]

[0155] In formula (12), P lcs It is the ratio of the common subsequence length to the length of the answer output by the large model.

[0156]

[0157] In formula (13), F lcs It is the final score of RUGE, which measures the completeness of the large model output. β is a manually set value.

[0158] Embodiment 3:

[0159] The present invention discloses a method for interpreting laboratory tests in physical examination reports based on a hybrid expert model, comprising the following steps:

[0160] Training was performed on 7 A800s (80G), using pycharm for implementation.

[0161] The first step is data preparation.

[0162] A dataset of 213 questions and answers was compiled by professional doctors from the physical examination department of a tertiary hospital. The dataset was combined with an open source Chinese medical question and answer dataset and a medical laboratory professional title examination knowledge base to expand the knowledge and build a complete medical physical examination laboratory examination benchmark dataset.

[0163] Chinese medical question answering datasets include MedDialog-CN, IMCS-V2, CHIP-MDCFNPC, MedDG, cMedQA2, and Chinese-medical-dialogue-data.

[0164] The second step is model fine-tuning.

[0165] Fine-tuning of multi-round dialogue instructions based on hybrid low-rank adaptation: Based on the prepared benchmark dataset, after data cleaning and preprocessing, a Chinese medical multi-round dialogue instruction dataset is constructed for fine-tuning, with a data volume of more than 9 million and an average number of inquiry dialogue rounds of 3. Enter the finetune_basemodel directory to install the dependency library, add a low-rank matrix to the Multi-head Self-Attention part of the model encoding layer, set the lora rank r to 128, the scaling factor to 256, the Adam optimizer learning rate to 0.01, and train on the dataset for 10 epochs. Use soft router, that is, all experts have the possibility to participate in the training reasoning process, and then sum all the results.

[0166] Fine-tuning of prompts based on thought chains: 16 inference examples were manually compiled, and a dataset of 50 thought chain prompts was generated using the Auto-COT framework based on the professional qualification examination question bank. Hard prompts were used to act on the input embedding module of the model to improve the model's reasoning ability.

[0167] The third step is model training.

[0168] For the problem of input from the examinee, if there is an input of the physical examination report in PDF or image form, text recognition is required in PDF or image. Install and configure the PaddleOCR related environment, download the PaddleOCR Chinese lightweight OCR model to test some image recognition results, call tools / infer / predict_system.py to complete report recognition, perform structured processing on the recognized text, combine it with the normal range to get the quantitative result, and input it into the hybrid expert model together with the text information directly input by the examinee, and start the multi-round dialogue mechanism at the same time to output the final interpretation.

[0169] Step 4: Effect evaluation.

[0170] The code defines a string generated_text containing the generated text and a list of strings reference_texts containing the reference texts interpreted by the experts.

[0171] The sentence_bleu function in Python's Natural Language Toolkit abbreviation nltk library is used to calculate the Bleu index between the text generated by the model and the text interpreted by the expert. The ROUGE index between the model output and the expert interpretation is calculated using Python's rouge library. The code creates a Rouge object and uses the get_scores method to calculate the ROUGE index between the generated text and the reference text.

[0172] This application focuses on physical examination chemical laboratory tests. First, it organizes and compiles professional data sets and special knowledge graphs, builds a hybrid expert model, covers five experts in routine, biochemistry, immunology, microbiology and molecular biology, and adds a multi-round dialogue module to obtain multi-dimensional information. Then, it is tuned through the method of "mixed low-rank adaptive multi-round dialogue instruction fine-tuning + thinking chain prompt fine-tuning" to deepen the model's understanding of physical examination laboratory tests. After the model tuning is completed, the examinee inputs the report and questions. The large model realizes the detection and recognition of PDF and image reports based on the PaddleOCR model library, extracts various indicator information and user query question input, and the gated network dispatches the input to the corresponding expert to complete the task of reasoning diagnostic test results and outputting treatment recommendations, thereby realizing the auxiliary interpretation of physical examination report laboratory tests.

Claims

1. A method for interpreting laboratory tests in physical examination reports based on a hybrid expert model, characterized in that: The method comprises the following steps: step S1: constructing and obtaining a physical examination laboratory examination question and answer dataset and a knowledge graph in the field of laboratory examination of physical examination reports; step S2: constructing and obtaining a hybrid expert model based on multi-round dialogues; step S3: constructing and obtaining a Chinese medical multi-round dialogue instruction dataset, and fine-tuning the hybrid expert model based on the Chinese medical multi-round dialogue instruction dataset and a hybrid low-rank adaptive algorithm to obtain a fine-tuned hybrid expert model; step S4: fine-tuning the prompt of the fine-tuned hybrid expert model based on the thinking chain to obtain a hybrid expert model after prompt fine-tuning; step S5: inputting the physical examination report into the hybrid expert model after prompt fine-tuning to obtain the laboratory examination interpretation result; in step S1, based on the common problems in the laboratory examination of the physical examination report, combined with the open source Chinese medical dataset and the medical laboratory professional title examination knowledge base, a physical examination laboratory examination question and answer dataset and a knowledge graph in the field of laboratory examination of the physical examination report are constructed in combination; In the step S2, according to the five categories of physical examination laboratory examinations, five sub-expert models are constructed, a gated network is set, and a multi-round dialogue module is added to construct a hybrid expert model of expert network + gated network + multi-round dialogue module. The expert network includes five sub-expert models, namely, routine examination expert, biochemical examination expert, immune examination expert, microbiological examination expert and molecular biology examination expert; the gated network includes an expert activation module and an aggregation module, and the multi-round dialogue module includes four modules, namely, a natural language understanding module NLU, a dialogue state tracking module DST, a strategy learning module PL and a natural language generation module NLG; In step S3, the physical examination laboratory test question and answer dataset and the physical examination report laboratory test domain knowledge graph constructed in step S1 are combined with the existing open source Chinese medical dataset to form a multi-round dialogue command dataset in the medical field; and a hybrid low-rank adaptive algorithm is used to fine-tune the hybrid expert model obtained in step S2.

2. A method for interpreting laboratory tests in physical examination reports based on a hybrid expert model according to claim 1, characterized in that: In step S5, the indicators and values ​​in the PDF or picture report uploaded by the examinee are identified, and after data cleaning and preprocessing, they are compared with the structured knowledge in the knowledge base as a basis for quantitative diagnosis; the information on medical history and living habits obtained through multiple rounds of conversations is combined to obtain diagnostic results and treatment recommendations.

3. The method for interpreting laboratory tests in physical examination reports based on a hybrid expert model according to claim 1, characterized in that: It also includes step S6 effect evaluation after step S5, Step S6: Use the BLEU and ROUGE indicators to evaluate the quality of the laboratory examination interpretation results output by the hybrid expert model from the two dimensions of precision and recall.

Images