Llm-based multidisciplinary consultation digital avatar system and method

Abstract

This invention relates to the field of medical data processing technology, and discloses a multidisciplinary consultation digital avatar system and method based on LLM (Limited Ledger Model). The method includes: acquiring target medical record information; analyzing the target medical record information using multiple expert digital avatar units to obtain multiple expert analysis results, each expert digital avatar unit including a first agent and a second agent; and making a comprehensive decision based on the multiple expert analysis results and reasoning chain information to obtain the target analysis result. This invention relies on the traceability of reasoning chain information to ensure the rigor of reasoning, and improves the accuracy and efficiency of the analysis results by integrating multiple expert opinions through comprehensive decision-making.

Description

Technical Field

[0001] This invention relates to the field of medical data processing technology, and more specifically, to a multidisciplinary consultation digital avatar system and method based on LLM. Background Technology

[0002] In contemporary medical practice, multidisciplinary team (MDT) consultation is the core model for dealing with complex and difficult medical cases. By integrating the knowledge and experience of experts from different professional fields, it aims to provide patients with a comprehensive and individualized reference plan.

[0003] Currently, there are attempts to utilize artificial intelligence technology to assist decision-making processes, particularly using deep learning models to analyze patient data to support decision-making. However, existing technologies still have significant limitations: they fail to fully simulate the workings of multidisciplinary team (MDT) teams in the real world, cannot accurately reproduce the process of knowledge sharing among experts around multidimensional information in complex medical records, and decision-making is often limited to single-dimensional data, making it difficult to meet the needs of multidimensional integration, which greatly limits the accuracy and efficiency of the analysis results. Summary of the Invention

[0004] In view of this, the present invention provides a digital clone method for multidisciplinary consultation based on LLM to solve the problems of poor accuracy and efficiency of analysis results in multidisciplinary consultation.

[0005] In a first aspect, the present invention provides a multidisciplinary consultation digital avatar method based on LLM, comprising: acquiring target medical record information; analyzing the target medical record information using multiple expert digital avatar units to obtain multiple expert analysis results, each expert digital avatar unit comprising a first agent and a second agent, the first agent being used to decompose and reason about the target medical record information to obtain reasoning chain information, the reasoning chain information being text information expressing the intermediate reasoning process from the target medical record information to the analysis result; the second agent being used to analyze the target medical record information and the corresponding reasoning chain information to obtain expert analysis results; and making a comprehensive decision based on the multiple expert analysis results and the reasoning chain information to obtain the target analysis result.

[0006] In one optional implementation, the expert digital avatar unit is pre-trained by an initial first agent and an initial second agent, respectively. The training process of the initial first agent includes: constructing a corresponding first training dataset and a first test dataset based on the type of the expert digital avatar unit. The first training dataset includes training medical record information and training expert reasoning chain information for several users, and the first test dataset includes test medical record information and test expert reasoning chain information for several users; constructing first prompt information for the first training dataset using an instruction fine-tuning strategy; inputting the training medical record information and the first prompt information for several users into the initial first agent to obtain initial expert reasoning chain information; fine-tuning the initial first agent based on the training expert reasoning chain information and the initial expert reasoning chain information to obtain a fine-tuned first agent; using the fine-tuned first agent to predict test medical record information to obtain predicted expert reasoning chain information; performing similarity analysis on the predicted expert reasoning chain information and the test expert reasoning chain information to obtain a first analysis result; and iteratively updating the fine-tuned first agent based on the first analysis result until a first agent conforming to the corresponding expert thinking mode is obtained.

[0007] In one optional implementation, the training process of the initial second agent includes: constructing a corresponding second training dataset and a second test dataset based on the type of expert digital avatar unit. The second training dataset includes training medical record information of several users, training expert inference chain information, and training expert analysis results. The second test dataset includes testing medical record information of several users, testing expert inference chain information, and testing expert analysis results. A second prompt message is constructed for the second training dataset using an instruction fine-tuning strategy. The training medical record information of several users, training expert inference chain information, and the second prompt message are input into the initial second agent to obtain the initial expert analysis results. The initial second agent is fine-tuned based on the training expert analysis results and the initial expert analysis results to obtain a fine-tuned second agent. The fine-tuned second agent is used to predict the testing medical record information and the testing expert inference chain information to obtain the predicted expert analysis results. The accuracy of the predicted expert analysis results and the testing expert analysis results is analyzed to obtain the second analysis results. The fine-tuned second agent is iteratively updated based on the second analysis results until a second agent conforming to the corresponding expert's thinking pattern is obtained.

[0008] In one optional implementation, multiple expert digital avatars are used to analyze the target medical record information to obtain multiple expert analysis results. The method further includes: determining a third and a fourth prompt based on the type of the expert digital avatar; based on the third prompt, using multiple expert digital avatars to perform deconstruction and reasoning analysis on the target medical record information to obtain multiple first expert reasoning chain information; and based on the fourth prompt, using multiple expert digital avatars to analyze the target medical record information and the corresponding first expert reasoning chain information to obtain multiple expert analysis results.

[0009] In an optional implementation, the LLM-based multidisciplinary consultation digital avatar method provided by the present invention further includes: using multiple first expert digital avatar units to decompose and reason about the target medical record information to obtain multiple expert reasoning chain information; each first expert digital avatar unit includes a first intelligent agent, which is used to decompose and reason about the target medical record information to obtain expert reasoning chain information; and using a second expert digital avatar unit to make a comprehensive decision on the target medical record information and the multiple expert reasoning chain information to obtain the target analysis result.

[0010] In one optional implementation, multiple expert digital avatars are used to analyze the target medical record information to obtain multiple expert analysis results. The method further includes: retrieving multiple types of reference case information from a pre-built knowledge base based on the types of the multiple expert digital avatars; using a first agent in each of the multiple expert digital avatars to perform deconstruction and reasoning analysis on the target medical record information based on the multiple types of reference case information, obtaining multiple expert reasoning chain information; and using a second agent in each of the multiple expert digital avatars to analyze the target medical record information and the corresponding expert reasoning chain information to obtain multiple expert analysis results.

[0011] In one optional implementation, multiple expert digital avatars are used to analyze the target medical record information to obtain multiple expert analysis results. The method further includes: setting prompts with fixed format requirements based on the type of the expert digital avatar; using multiple expert digital avatars to output the target medical record information according to the prompts with fixed format requirements to obtain multiple fixed-format output texts; and parsing the text structure of the multiple fixed-format output texts to obtain multiple expert reasoning chain information and multiple expert analysis results.

[0012] In one alternative implementation, the inference chain information is a structured tag sequence containing multiple tag pairs ordered according to the inference process.

[0013] In one alternative implementation, the reasoning chain information is a visual reasoning flowchart to graphically represent the logical relationships in the reasoning process.

[0014] Secondly, the present invention provides a multidisciplinary consultation digital clone system based on LLM, comprising: a memory and a processor, wherein the memory and the processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the multidisciplinary consultation digital clone method based on LLM described in the first aspect or any corresponding embodiment.

[0015] This embodiment provides a multidisciplinary consultation digital avatar method based on LLM, which ensures data integrity by acquiring target medical record information covering multiple dimensions. Then, it utilizes expert digital avatars constructed for each domain expert to output multi-domain expert analysis results in parallel. Each expert digital avatar contains a first agent and a second agent. Specifically, the first agent of each expert digital avatar generates traceable reasoning chain information based on the target medical record information, replicating the expert's professional thinking logic. Next, the second agent generates analysis results based on the target medical record information and the corresponding reasoning chain information, accurately replicating each expert's personalized decision-making logic. By parallelly calling the multi-domain expert digital avatars, the reasoning chain information and analysis results from each expert's perspective are output synchronously, achieving efficient convergence of multi-domain opinions. Finally, by integrating the reasoning chain information and expert analysis results from all expert digital avatars, the collaborative logic of multidisciplinary consultation is simulated, and a comprehensive multidisciplinary consultation report is generated as the target analysis result for reference. This application ensures data integrity by acquiring multi-dimensional medical record information, integrates multi-domain expert perspectives to reduce limitations imposed by a single perspective, guarantees the rigor of reasoning by relying on the traceability of information in the reasoning chain, and improves the accuracy of analysis results by integrating multiple expert opinions through comprehensive decision-making. It replaces the time-consuming offline multidisciplinary consultation process by parallel invocation of expert digital avatars, automatically integrating all reasoning chains and generating reports, thereby improving the overall efficiency of complex case management. This application separates the first and second intelligent agents, forcing them to execute clinical thinking and decision-making step by step, realistically and completely replicating the two-stage cognitive path from analysis to decision-making in multidisciplinary consultations with corresponding expert styles. This makes the entire consultation process transparent and traceable, greatly enhancing doctors' trust in and willingness to adopt the analysis results. Furthermore, the intelligent agents, based on expert decision-making logic, can quickly reason, shortening the analysis cycle and improving processing efficiency, ultimately providing efficient and comprehensive auxiliary judgment basis for clinical practice. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the LLM-based multidisciplinary consultation digital clone method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the training process of the first intelligent agent according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the training process of the second intelligent agent according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] According to an embodiment of the present invention, an embodiment of a multidisciplinary consultation digital clone method based on LLM is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0020] This embodiment provides a multidisciplinary consultation digital avatar method based on LLM, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 1 This is a flowchart of a multidisciplinary consultation digital avatar method based on LLM according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the target medical record information.

[0021] The target medical record information in this embodiment includes the chief complaint, present illness, past medical history, family history, psychological assessment scales such as HAMD (Hamilton Depression Rating Scale) and PANSS (Positive and Negative Syndrome Scale), laboratory tests, and imaging results.

[0022] Step S102: Analyze the target medical record information using multiple expert digital avatar units to obtain multiple expert analysis results. Each expert digital avatar unit includes a first intelligent agent and a second intelligent agent. The first intelligent agent is used to decompose and reason about the target medical record information to obtain reasoning chain information. The reasoning chain information is text information that expresses the intermediate reasoning process from the target medical record information to the analysis result. The second intelligent agent is used to analyze the target medical record information and the corresponding reasoning chain information to obtain the expert analysis result.

[0023] In this embodiment, corresponding expert digital avatars are constructed for each field of expertise. Each expert digital avatar includes a first intelligent agent (Agent1) and a second intelligent agent (Agent2). Both agents are trained based on the decision-making logic of the expert in that field, accurately replicating the expert's personalized decision-making characteristics (such as the expert's unique reasoning preferences, judgment habits, and logical emphasis when analyzing medical records). Taking the mental health field as an example, expert types include experts in affective disorders, psychopathology, and sleep. After obtaining the target medical record information, it is then input into each expert digital avatar for analysis.

[0024] Taking a digital avatar of an affective disorder expert as an example, the first built-in intelligent agent first deconstructs and analyzes the target medical record information to obtain the expert's reasoning chain information. This reasoning chain information can be understood as clinical thinking (Chain-of-Thought, CoT): a logically clear chain of reasoning developed by the expert around the target medical record. Subsequently, a second intelligent agent combines the target medical record information and the corresponding expert reasoning chain information for further analysis, yielding the expert analysis results from the affective disorder expert. In this way, expert analysis results from digital avatars of other experts, such as psychopathologists and sleep specialists, can be obtained, ultimately achieving parallel output of expert opinions from multiple fields.

[0025] Step S103: Based on the analysis results of multiple experts and the information of the reasoning chain, a comprehensive decision is made to obtain the target analysis result.

[0026] This embodiment integrates the analysis results of all expert digital avatars (including the reasoning chain information CoT generated by the first intelligent agent and the expert analysis results obtained by the second intelligent agent), merges the reasoning processes and conclusions of these multi-domain experts into a comprehensive multidisciplinary consultation report, and submits it to the attending physician as the target analysis result for reference, simulating the collaborative logic of multidisciplinary consultation, so as to comprehensively assist clinical judgment.

[0027] The technical framework of this application is not limited to psychiatry. As long as multidisciplinary consultation data (including background data, clinical reasoning (CoT), and decision-making results) in the relevant field is available, the same Agent1-Agent2 architecture can be reused to quickly build expert digital avatars in fields such as cardiovascular medicine, oncology, and neurology. This provides a standardized technical path for building a cross-disciplinary intelligent consultation platform in the future.

[0028] This embodiment provides a multidisciplinary consultation digital avatar method based on LLM, which ensures data integrity by acquiring target medical record information covering multiple dimensions. Then, it utilizes expert digital avatars constructed for each domain expert to output multi-domain expert analysis results in parallel. Each expert digital avatar contains a first agent and a second agent. Specifically, the first agent of each expert digital avatar generates traceable reasoning chain information based on the target medical record information, replicating the expert's professional thinking logic. Next, the second agent generates analysis results based on the target medical record information and the corresponding reasoning chain information, accurately replicating each expert's personalized decision-making logic. By parallelly calling the multi-domain expert digital avatars, the reasoning chain information and analysis results from each expert's perspective are output synchronously, achieving efficient convergence of multi-domain opinions. Finally, by integrating the reasoning chain information and expert analysis results from all expert digital avatars, the collaborative logic of multidisciplinary consultation is simulated, and a comprehensive multidisciplinary consultation report is generated as the target analysis result for reference. This application ensures data integrity by acquiring multi-dimensional medical record information, integrates multi-domain expert perspectives to reduce limitations imposed by a single perspective, guarantees the rigor of reasoning by relying on the traceability of information in the reasoning chain, and improves the accuracy of analysis results by integrating multiple expert opinions through comprehensive decision-making. It replaces the time-consuming offline multidisciplinary consultation process by parallel invocation of expert digital avatars, automatically integrating all reasoning chains and generating reports, thereby improving the overall efficiency of complex case management. This application separates the first and second intelligent agents, forcing them to execute clinical thinking and decision-making step by step, realistically and completely replicating the two-stage cognitive path from analysis to decision-making in multidisciplinary consultations with corresponding expert styles. This makes the entire consultation process transparent and traceable, greatly enhancing doctors' trust in and willingness to adopt the analysis results. Furthermore, the intelligent agents, based on expert decision-making logic, can quickly reason, shortening the analysis cycle and improving processing efficiency, ultimately providing efficient and comprehensive auxiliary judgment basis for clinical practice.

[0029] The following application scenario illustrates the LLM-based multidisciplinary consultation digital avatar method provided in this invention: A hospital's psychiatry department admitted a patient presenting with depressed mood, insomnia, poor concentration, occasional paranoid delusions, and difficulty in diagnosis. The intelligent multidisciplinary consultation system was used to assist in the diagnosis: The doctor first entered the patient's medical records into the system. The system then invoked three digital avatars—an affective disorder expert, a psychopathology expert, and a sleep medicine expert—to analyze the patient's condition. The first agent of the affective disorder expert avatar analyzed the patient's significantly depressed mood and decreased interest, high HAMD score, but also noted accompanying delusions. The system requires differentiation between major depressive disorder with psychotic symptoms and bipolar depression. Based on this, the second agent concludes that "major depressive disorder with psychotic symptoms is highly probable." The first agent, representing a psychopathology expert, analyzes that "fixed delusions accompanied by emotional apathy suggest early signs of schizophrenia," and the second agent concludes that "schizophrenia cannot be ruled out; a comprehensive PANSS assessment is recommended." The first agent, representing a sleep medicine expert, analyzes that "insomnia is the initial symptom and affects daytime function; primary or accompanying symptoms need evaluation," and the second agent concludes that "secondary insomnia is closely related to mood disorders." The system integrates the reasoning chains (CoT) and analysis results of the three experts to generate a report. The attending physician, after reviewing the report, decides to conduct further examinations, ultimately diagnosing "major depressive disorder with psychotic symptoms" and developing a treatment plan. The system simulates a multi-expert consultation within minutes, providing multi-dimensional professional insights to assist doctors in making a comprehensive judgment and significantly improving diagnostic efficiency.

[0030] In some optional embodiments, each expert digital avatar unit in step S102 is pre-trained by the initial first agent and the initial second agent, respectively.

[0031] Among them, such as Figure 2 As shown, the training process of the first agent includes: Step 1: Construct the corresponding first training dataset and first test dataset based on the type of expert digital avatar unit. The first training dataset includes training medical record information and training expert inference chain information for several users. The first test dataset includes testing medical record information and testing expert inference chain information for several users.

[0032] The training data in this embodiment uses psychiatry as an example, constructing dedicated datasets for different types of mental illnesses, such as affective disorders, psychotic symptoms, and sleep disorders. Each dataset contains, for example, 150 real multidisciplinary consultation cases that have undergone ethical review and anonymization. Each case contains three parts of structured data: background data, clinical reasoning, and analysis results. The background data consists of complete medical record information, including chief complaint, present illness, past medical history, family history, psychological assessment scales (such as HAMD, PANSS), laboratory tests, and imaging results. The clinical reasoning (CoT) consists of discussion texts of multiple experts discussing the patient in a multidisciplinary meeting, labeled according to roles (such as affective disorder expert A, psychopathology expert B, and sleep expert C), and summarized and refined by professionals into a logically clear chain of reasoning. The analysis results are the final diagnostic conclusions formed through expert consensus, conforming to the International Classification of Diseases 11th Revision (ICD-11) or the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) standards. The above datasets are divided into a first training dataset (100 cases) and a first test dataset (50 cases) in a 7:3 ratio.

[0033] The high-quality and structured dataset in this embodiment forms the basis for building trustworthy expert digital avatars. By classifying and modeling by disease type, it ensures that each expert digital avatar has domain-specific expertise. Clinical reasoning annotations enable the model to learn the expert's reasoning process rather than just the result, thus improving interpretability.

[0034] Step 2: Use the instruction fine-tuning strategy to construct the first prompt information for the first training dataset.

[0035] Taking a psychiatrist as an example, an instruction fine-tuning strategy is used to construct a prompt for the first training dataset: "You are an experienced psychiatrist. Based on the following patient's medical record information, please elaborate on your clinical analysis and reasoning process: [Insert background]". The expected model output is text that is highly similar to the CoT of real clinical reasoning.

[0036] Step 3: Input the training medical record information and first prompt information of several users into the initial first intelligent agent to obtain the initial expert reasoning chain information.

[0037] In this embodiment, the initial first agent can use a general large language model (LLM) (such as ChatGPT, DeepSeek, Tongyi Qianwen, etc.) as the base model. Each training medical record and the first prompt information in the first training dataset are input into the initial first agent, and the corresponding initial inference chain information of the psychiatric expert is output.

[0038] Step four: Fine-tune the initial first agent based on the training expert inference chain information and the initial expert inference chain information to obtain the fine-tuned first agent.

[0039] This embodiment uses the training expert's reasoning chain information and the initial expert's reasoning chain information to fine-tune the initial first agent using a basic large language model, optimizing its parameters so that it can stably generate "clinical thinking" consistent with the expert's style from "background data". This allows the fine-tuned first agent to master the expert's clinical thinking pattern, which is key to the digital avatar's professionalism.

[0040] The fine-tuning measures in this embodiment can be further optimized using reinforcement learning from human feedback (RLHF), or by using more complex prompt engineering without fine-tuning. Instruction fine-tuning directly modifies model parameters, resulting in stable and persistent effects, making it suitable for building specialized agents. Reinforcement learning, on the other hand, can further improve alignment and achieve zero-cost prompt engineering.

[0041] Step 5: Use the fine-tuned first intelligent agent to predict the test medical record information to obtain the inference chain information of the prediction expert.

[0042] In this embodiment, the performance of the fine-tuned first agent is analyzed using a reserved first test dataset. Specifically, the test medical record information in the first test dataset is input into the fine-tuned first agent to generate the inference chain information of the prediction expert.

[0043] Step six: Perform similarity analysis on the reasoning chain information of the prediction expert and the reasoning chain information of the test expert to obtain the first analysis result.

[0044] This embodiment performs semantic similarity analysis between the predicted expert reasoning chain information and the real test expert reasoning chain information in the first test dataset. For example, it uses BERTScore (BERT-based Evaluation Score), ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation - Longest Common Subsequence), SimCSE (Simple Contrastive Sentence Embedding), etc., to obtain the first analysis result.

[0045] Step 7: Based on the first analysis results, iteratively update the first intelligent agent until a first intelligent agent that conforms to the corresponding expert's thinking pattern is obtained.

[0046] In this embodiment, the first intelligent agent is fine-tuned and iterated multiple times based on the first analysis result in step six until a preset performance threshold is reached (e.g., the similarity of expert reasoning chain information > 0.85), at which point the update stops, and the first intelligent agent is obtained.

[0047] The initial training of the first agent described above teaches it to generate the expert's typical clinical reasoning process (CoT) based solely on the patient's background data. The first agent simulates the expert's ability to "think." The training process for the first agent of other types of expert digital avatars is the same and will not be elaborated upon here.

[0048] This embodiment constructs a high-quality, labeled dataset; then, it fine-tunes the design prompts using instructions; subsequently, it inputs training medical record information and first prompt information into the initial first agent to generate initial expert reasoning chain information; based on the initial expert reasoning chain information and the training expert reasoning chain information, it fine-tunes and optimizes the initial first agent to complete the initial training, obtaining the fine-tuned first agent; then, it uses test medical record information from the test set to perform performance analysis on the fine-tuned first agent, obtaining the predicted expert reasoning chain information, and further verifies the similarity between the predicted expert reasoning chain information and the test expert reasoning chain information through similarity analysis indicators; finally, it iterates and updates until the target is met, obtaining the first agent. The training process of this embodiment ensures that the agent can reproduce expert reasoning logic (not just the result), structured data and clinical thinking annotations improve interpretability, and disease classification modeling enhances domain specialization; iterative verification ensures that the reasoning ability is close to that of real experts, providing reliable intelligent assistance for multidisciplinary consultations.

[0049] In some alternative embodiments, such as Figure 3 As shown, the training process of the second agent for each expert digital avatar in step S102 includes: Step 1: Construct a second training dataset and a second test dataset based on the type of expert digital avatar unit. The second training dataset includes training medical record information of several users, training expert inference chain information, and training expert analysis results. The second test dataset includes testing medical record information of several users, testing expert inference chain information, and testing expert analysis results.

[0050] In this embodiment, 150 examples of data were constructed to train the initial second agent. The dataset was divided into a second training dataset (100 examples) and a second test dataset (50 examples) in a 7:3 ratio.

[0051] Step 2: Use the instruction fine-tuning strategy to construct the second prompt information for the second training dataset.

[0052] Taking a psychiatric expert as an example, an instruction fine-tuning strategy is used to construct a prompt for the second training dataset: "You are a psychiatric expert. Please provide your final analytical opinion based on the following patient's medical records and your clinical analysis process: [Insert background][Insert clinical thinking]". The model output is expected to be consistent with the actual analysis.

[0053] Step 3: Input the training medical record information of several users, the training expert inference chain information, and the second prompt information into the initial second intelligent agent to obtain the initial expert analysis results.

[0054] In this embodiment, the initial second agent can use a general large language model (LLM) (such as ChatGPT, DeepSeek, Tongyi Qianwen, etc.) as the base model. Each training medical record, training expert inference chain information, and second prompt information from the second training dataset are input into the initial second agent, and the corresponding initial expert analysis results of the psychiatric expert are output.

[0055] Step four: Fine-tune the initial second agent based on the training expert analysis results and the initial expert analysis results to obtain the fine-tuned second agent.

[0056] This embodiment uses the training expert analysis results and the initial expert analysis results to fine-tune the initial second agent based on the basic large language model, optimizing its parameters so that it can make accurate analyses based on a complete reasoning chain. This allows the fine-tuned second agent to master the ability of experts "how to draw conclusions," which is key to the professionalism of the digital avatar.

[0057] Step 5: Use the fine-tuned second agent to predict the test medical record information and the test expert reasoning chain information to obtain the prediction expert analysis results.

[0058] In this embodiment, a reserved second test dataset is used to perform performance analysis on the fine-tuned second agent after training. Specifically, the test medical record information and test expert inference chain information in the second test dataset are input into the fine-tuned second agent to generate prediction expert analysis results.

[0059] Step six: Perform an accuracy analysis on the prediction expert analysis results and the test expert analysis results to obtain the second analysis result.

[0060] This embodiment compares the accuracy of the predicted expert analysis results with the actual expert analysis results in the second test dataset, such as calculating precision, recall, F1 score, etc., to obtain the first analysis result.

[0061] Step 7: Based on the second analysis results, iteratively update the fine-tuned second agent until a second agent that conforms to the corresponding expert's thinking pattern is obtained.

[0062] In this embodiment, the second intelligent agent is fine-tuned and iterated multiple times based on the second analysis results in step six until a preset performance threshold is reached (e.g., analysis accuracy > 90%), at which point the update is stopped, and the second intelligent agent is obtained.

[0063] The training described above for the initial second agent teaches it the ability to draw conclusions and generate typical analytical results characteristic of an expert. The training process for other types of expert digital avatars is the same and will not be elaborated upon here.

[0064] This embodiment constructs a second training dataset and a second test dataset for training the initial second agent. A fine-tuning strategy is used to construct second prompt information for the second training dataset, enabling it to make conclusions. Next, training medical record information from several users, training expert reasoning chain information, and the second prompt information are input into the initial second agent to obtain initial expert analysis results. This transforms the reasoning process into analysis results, ensuring the consistency and professionalism of the decision-making. Based on the training expert analysis results and the initial expert analysis results, the initial second agent is fine-tuned and optimized, completing the initial training of the first agent. Then, the fine-tuned second agent is used to predict test medical record information and test expert reasoning chain information to obtain predicted expert analysis results. Accuracy analysis is performed on the predicted expert analysis results and the test expert analysis results to obtain second analysis results. Based on the second analysis results, the fine-tuned second agent is iteratively updated until a second agent conforming to the corresponding expert's thinking pattern is obtained, thus completing the performance evaluation of the second agent. The training process in this embodiment ensures that the intelligent system can replicate the ability of experts to "draw conclusions," and the structured data and clinical reasoning annotations enhance interpretability. Iterative verification ensures that the ability to draw conclusions is close to that of real experts, providing reliable intelligent assistance for multidisciplinary consultations.

[0065] After training the first and second intelligent agents, they together constitute a digital clone of an expert. In practical applications, the output of the first intelligent agent is one of the necessary inputs for the second intelligent agent. This separation design allows the thought process and decision-making results to be separated and auditable, greatly improving the transparency and credibility of the process.

[0066] In some optional embodiments, multiple expert digital avatars are used to analyze the target medical record information to obtain multiple expert analysis results. Another method is that each expert digital avatar is trained by a general large language model (LLM) (such as ChatGPT, DeepSeek, Tongyi Qianwen, etc.), and two independent models are not used to train the first agent and the second agent.

[0067] Step 1: Determine the third and fourth prompt messages based on the type of the expert digital clone unit.

[0068] In this embodiment, different prompts are designed to switch roles during the reasoning phase: When performing functions such as the first agent, the determined third prompt message is: "You are a psychiatrist. Please only output your clinical analysis and reasoning process (Chain-of-Thought) for the following case, and do not give the final decision result." When performing functions such as the second agent, the fourth prompt message is determined as follows: "You are a psychiatrist. Please provide a final decision based on the following case data and your analysis process." Step two: Based on the third prompt information, multiple expert digital clones are used to deconstruct and analyze the target medical record information to obtain multiple first expert reasoning chain information.

[0069] Based on the third prompt information, this embodiment directly uses model training to obtain each expert digital clone unit to decompose and reason about the target medical record information, thereby obtaining multiple first expert reasoning chain information.

[0070] Step 3: Based on the fourth prompt information, multiple expert digital clones are used to analyze the target medical record information and the corresponding first expert reasoning chain information to obtain multiple expert analysis results.

[0071] In this embodiment, based on the fourth prompt information, multiple expert digital clones are used to analyze the target medical record information and the corresponding first expert reasoning chain information to obtain multiple expert analysis results.

[0072] Each of the aforementioned expert digital avatar units is pre-trained using a separate model. The training process for each expert digital avatar unit includes: constructing a corresponding third training dataset and a third test dataset based on the type of the expert digital avatar unit. The third training dataset includes training medical record information, training expert inference chain information, and training expert analysis results for several users. The third test dataset includes testing medical record information, testing expert inference chain information, and testing expert analysis results for several users. Two prompts are constructed for the third training dataset using an instruction fine-tuning strategy, resulting in third and fourth prompts. The analysis model is trained based on the training medical record information and the third prompts for several users to obtain initial expert inference chain information. Initial expert analysis results are obtained based on the training medical record information, training expert inference chain information, and the fourth prompts for several users. The initial expert inference chain information is then processed. A similarity analysis is performed on the inference chain information of the training experts to obtain the third analysis result. A similarity analysis is then performed on the initial expert analysis result and the training expert analysis result to obtain the fourth analysis result. Based on the third and fourth analysis results, a comprehensive calculation (e.g., average, standard deviation, etc.) is performed to fine-tune the analysis model, resulting in a fine-tuned analysis model. The fine-tuned analysis model is then used to predict test medical record information to obtain the prediction expert inference chain information and the prediction expert analysis result. An accuracy analysis is then performed on the prediction expert inference chain information and the test expert inference chain information to obtain the fifth analysis result. An accuracy analysis is then performed on the prediction expert analysis result and the test expert analysis result to obtain the sixth analysis result. Based on the fifth and sixth analysis results, a comprehensive calculation (e.g., average, standard deviation, etc.) is performed to iteratively update the analysis model until an expert digital clone unit that conforms to the corresponding expert's thinking mode is obtained.

[0073] This embodiment presents a second approach to analyzing target medical record information. First, based on the type of expert digital avatar unit, a third prompt (outputting only the reasoning process, without providing a decision result) and a fourth prompt (providing a final decision opinion based on case data and analysis process) are determined, and the model role is switched through differentiated prompts. Next, the third prompt drives a single-model expert digital avatar unit trained by a general large language model to decompose and reason about the target medical record information, obtaining multiple first expert reasoning chains. Based on the fourth prompt, the same model unit combines the target medical record information with the corresponding first expert reasoning chain information to further analyze and obtain multiple expert analysis results. This embodiment, through a single-model and prompt switching approach, realizes a complete process of background data-clinical thinking-decision. The separation of clinical thinking and decision output meets the requirements of transparency in reasoning for intelligent multidisciplinary consultation. Furthermore, this embodiment saves model deployment resources, reduces operational complexity, and facilitates unified knowledge updates. It can reproduce multi-expert consultation logic at a lower cost, providing a feasible path for core clinical support objectives.

[0074] In some optional embodiments, the implementation of step S102 above further includes: Step 1: Utilize multiple first expert digital avatar units to deconstruct and reason about the target medical record information to obtain multiple expert reasoning chain information; each first expert digital avatar unit includes a first intelligent agent, which is used to deconstruct and reason about the target medical record information to obtain expert reasoning chain information.

[0075] In this embodiment, each first expert digital clone unit includes only a first intelligent agent to deconstruct and reason about the target medical record information to obtain expert reasoning chain information. Its training process is the same as the training steps of the first intelligent agent of the expert digital clone unit, learning the reasoning ability of real experts.

[0076] Step two: Use the second expert digital avatar unit to make a comprehensive decision on the target medical record information and information from multiple expert reasoning chains to obtain the target analysis results.

[0077] This embodiment constructs a second expert digital avatar unit to comprehensively analyze the expert reasoning chain information output by all the first expert digital avatar units and the corresponding target medical record information. The task of the second expert digital avatar unit is to integrate multiple viewpoints and output a consensus result. The second expert digital avatar unit can be trained based on the real reasoning chain information and case information of several experts, and its performance can be analyzed with the comprehensive analysis results.

[0078] This embodiment represents a third approach to analyzing target medical record information. First, multiple first expert digital avatars are used to deconstruct and reason about the target medical record information, obtaining multiple expert reasoning chains. Then, a second expert digital avatar unit is used to synthesize all expert reasoning chains and corresponding medical record information to make a comprehensive decision, yielding the target analysis result. This embodiment more closely resembles the consensus-building process in real multidisciplinary consultations, and can even replace some of the attending physician's decision-making functions. It can automatically generate "consultation consensus," improving the level of intelligence and reducing the burden on doctors to integrate information.

[0079] In some optional embodiments, the implementation of step S102 above further includes: Step 1: Based on the types of multiple expert digital avatar units, retrieve reference case information from a pre-built knowledge base to obtain information on multiple types of reference cases.

[0080] Based on the type of the expert digital avatar (e.g., affective disorder expert, psychopathology expert, etc.), this embodiment retrieves relevant medical literature or similar medical records from pre-built databases such as psychiatric clinical guidelines, typical cases, and expert consensus knowledge bases to obtain reference case information (i.e., "external knowledge" to assist reasoning) for multiple types of experts.

[0081] Step 2: Based on multiple types of reference case information, the first intelligent agent in multiple expert digital avatar units is used to decompose and reason about the target medical record information to obtain multiple expert reasoning chain information.

[0082] In this embodiment, the retrieved reference case information is used as "context" and input into the first intelligent agent of multiple expert digital avatars. This allows the agent to combine the target medical record information with deconstruction and reasoning to generate more accurate expert reasoning chain information (i.e., simulating the reasoning process of expert clinical thinking). When training the first intelligent agent, the reference case information is added to the model training to enhance the model's medical knowledge support.

[0083] Step 3: The second agents in multiple expert digital avatar units analyze the target medical record information and the corresponding expert reasoning chain information to obtain multiple expert analysis results.

[0084] In this embodiment, multiple expert digital avatars are used as second agents to combine the target medical record information and the reasoning chain information generated by the first agent to further analyze and obtain expert analysis results.

[0085] This embodiment represents the fourth approach to analyzing target medical record information. First, based on the expert digital avatar type, medical literature or similar cases matching that type are retrieved from a pre-built knowledge base of psychiatric clinical guidelines, typical cases, and expert consensus, yielding type reference case information. Then, the reference cases are input into a first intelligent agent, which combines the target medical record information with deconstructed reasoning to generate more accurate expert reasoning chain information. Finally, a second intelligent agent integrates the target medical record information and the expert reasoning chain information to analyze and derive the expert analysis results. This embodiment, by retrieving authoritative knowledge bases, enhancing the generation of expert reasoning chain information, and integrating and analyzing the output, achieves a complete process from background data to clinical thinking to decision-making, significantly improving the professionalism and accuracy of expert reasoning chain information. Specifically, knowledge base retrieval provides evidence-based support for reasoning, reducing the risk of model "illusion"; the integration of reference cases during training enhances medical knowledge support, making the expert reasoning chain information more aligned with clinical norms. Ultimately, this provides more reliable and traceable intelligent assistance for multidisciplinary consultations, strengthening the clinical usability of the expert digital avatar.

[0086] In some optional embodiments, the implementation of step S102 above further includes: Step one, based on the type settings of the expert digital clone unit, includes prompts with fixed format requirements.

[0087] In this embodiment, the expert digital avatar unit is trained using a general-purpose big language model, eliminating the need to construct a first and second intelligent agent. Specifically, based on the type of expert digital avatar, such as an expert in emotional disorders or a psychopathology expert, prompts with fixed format requirements are set. For example, the output is explicitly required to be divided into two parts: clinical thinking and decision analysis, corresponding to the reasoning process and the final decision result, respectively.

[0088] Step two: Using multiple expert digital clones, the target medical record information is output according to the prompts in a fixed format, resulting in multiple fixed-format output texts.

[0089] This embodiment utilizes multiple expert digital avatar units to input target medical record information and generate output according to the fixed format prompts in step one, resulting in multiple fixed-format texts, each of which contains the reasoning process and decision analysis process of clinical thinking.

[0090] Step 3: Perform text structure parsing on multiple fixed-format output texts to obtain multiple expert reasoning chain information and multiple expert analysis results.

[0091] This embodiment performs text structure parsing on each fixed-format text obtained in step two. For example, it extracts the text content of the clinical thinking segment as reasoning chain information and extracts the content of the diagnostic conclusion segment as the analysis result, ultimately obtaining reasoning chain information and expert analysis results from multiple experts.

[0092] The expert digital avatar unit in this embodiment can be fine-tuned and trained through instructions, enabling the model to learn to complete two-stage tasks in a single generation. Subsequently, it can automatically parse the text structure and extract inference chain information and expert analysis results.

[0093] This embodiment represents the fifth approach to analyzing target medical record information. First, based on the expert digital avatar type, it sets prompts with fixed format requirements, explicitly requiring the output to be divided into two segments: "clinical reasoning" and "decision analysis," corresponding to the reasoning process and the final decision result, respectively, without constructing independent first and second intelligent agents. Then, using multiple expert digital avatar units, the target medical record information is generated and output according to the fixed format prompts, resulting in multiple fixed-format texts with segmented structures. Finally, the text is structurally parsed, extracting the "clinical reasoning" segment as expert reasoning chain information and the "decision analysis" segment as expert analysis results. This embodiment, through a minimalist architecture of "single model + fixed-format prompts + text parsing," achieves the process of background data-clinical reasoning-decision, while still separating the output of reasoning and decisions, meeting the interpretability requirements of intelligent multidisciplinary consultation for transparent reasoning. Its advantages lie in its simplest architecture and lowest deployment cost. It eliminates the need to maintain multiple models or dual intelligent agents; through fine-tuning of instructions, a single large language model can learn to generate and output segments at once, and the system automatically parses and separates the expert reasoning chain information from the diagnosis, significantly improving development and application efficiency.

[0094] In some optional embodiments, the reasoning chain information generated by the first agent in step S102 above can be free text, that is, a complete description of the reasoning process from medical record information to preliminary conclusion in natural language. For example: "The patient complains of depressed mood and insomnia for 3 months, poor concentration, and occasional paranoid delusions; the HAMD score of 28 points indicates severe depression. It is necessary to differentiate between bipolar depression (which can be temporarily ruled out if there is no history of mania) and severe depression with psychotic symptoms (delusions are the key supporting point). Based on the coexistence of mood and psychotic symptoms in the present medical history, the latter is initially suspected and further investigation is needed."

[0095] In some optional embodiments, the reasoning chain information can also be a structured tag sequence containing multiple tag pairs ordered according to the reasoning process. Each tag pair focuses on a core element of the reasoning, such as: [Differential diagnosis: Bipolar depression, Major depressive disorder] - [Key symptom: Delusions] - [Scale support: HAMD=28] - [Preliminary judgment: Psychotic symptoms need to be ruled out]. The tag pairs are arranged in the order of "problem statement - evidence extraction - logical judgment," clearly presenting the reasoning framework.

[0096] In some optional embodiments, the reasoning chain information can also be a visual reasoning flowchart, which can be automatically generated as a decision tree or mind map to graphically display the logical relationships in the reasoning process. For example, a decision tree has a "core diagnostic question" as the root node (such as "differentiation between depressed mood and delusions"), branches such as "symptom characteristics", "scale results", "medical history exclusion items", etc., and leaf nodes are preliminary judgments; a mind map presents the relationship of "main question - sub-question - basis - conclusion" in a hierarchical layout, such as the central theme "diagnostic direction" with branches such as "depression-related" and "psychotic-related", which intuitively show the logical dependencies.

[0097] Because the inference chain information takes many forms, the second agent needs to adjust its analysis logic and training strategy for different forms of inference chain information. For example, it should focus on semantic parsing for free text, key-value extraction for structured tags, and graphical logic transformation for visual flowcharts, so as to ensure that it can effectively generate the expected analysis results based on the form.

[0098] All three forms of reasoning chain information in this embodiment can achieve interpretability of the reasoning process from background to clinical thinking. Structured tag output is more conducive to parsing and subsequent analysis, while flowcharts can more intuitively display the reasoning process.

[0099] The LLM-based multidisciplinary consultation digital avatar system provided in this embodiment, also known as the intelligent multidisciplinary consultation system, can be deployed in different environments. For example, it supports three adaptive application forms: the first form is a plug-in module embedded in the hospital's electronic medical record system, allowing doctors to launch the intelligent multidisciplinary consultation system analysis with a single click when viewing medical records; the second form is an independent web or mobile consultation platform, supporting cross-institutional expert digital avatar collaboration; and the third form is a doctor's personal "virtual assistant," which only calls upon a single expert's digital avatar for single-point consultation. Regardless of the deployment form, the core first and second intelligent agents' fine-tuning and inference mechanisms remain unchanged, enabling AI-assisted diagnosis. Its advantage lies in its adaptability to different medical institution deployment environments (such as electronic medical record integration, independent platforms, or lightweight assistants), significantly improving the system's versatility.

[0100] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0101] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A digital cloning method for multidisciplinary consultation based on LLM, characterized in that, The method includes: Obtain the target medical record information; Multiple expert digital avatar units are used to analyze the target medical record information to obtain multiple expert analysis results. Each expert digital avatar unit includes a first intelligent agent and a second intelligent agent. The first intelligent agent is used to deconstruct and reason about the target medical record information to obtain reasoning chain information. The reasoning chain information is text information that expresses the intermediate reasoning process from the target medical record information to the analysis result. The second intelligent agent is used to analyze the target medical record information and the corresponding reasoning chain information to obtain the expert analysis result. Based on the combined analysis results of the multiple experts and the information from the reasoning chain, a comprehensive decision is made to obtain the target analysis result.

2. The method according to claim 1, characterized in that, The expert digital avatar unit is pre-trained from an initial first agent and an initial second agent, respectively. The training process of the initial first agent includes: Based on the type of the expert digital avatar unit, a corresponding first training dataset and a first test dataset are constructed. The first training dataset includes training medical record information and training expert inference chain information of several users. The first test dataset includes test medical record information and test expert inference chain information of several users. A first prompt message is constructed for the first training dataset using a fine-tuning instruction strategy; The training medical record information and the first prompt information of the aforementioned users are input into the initial first intelligent agent to obtain the initial expert reasoning chain information; The initial first agent is fine-tuned based on the training expert inference chain information and the initial expert inference chain information to obtain the fine-tuned first agent; The first intelligent agent is used to predict the test medical record information to obtain the inference chain information of the prediction expert; A similarity analysis is performed on the inference chain information of the prediction expert and the inference chain information of the test expert to obtain a first analysis result; Based on the first analysis result, the first intelligent agent is iteratively updated until a first intelligent agent that conforms to the corresponding expert's thinking mode is obtained.

3. The method according to claim 2, characterized in that, The training process for the initial second agent includes: Based on the type of the expert digital avatar unit, a corresponding second training dataset and a second test dataset are constructed. The second training dataset includes training medical record information of several users, training expert inference chain information, and training expert analysis results. The second test dataset includes testing medical record information of several users, testing expert inference chain information, and testing expert analysis results. A second prompt message is constructed for the second training dataset using a fine-tuning instruction strategy. The training medical record information of the aforementioned users, the training expert inference chain information, and the second prompt information are input into the initial second intelligent agent to obtain the initial expert analysis results; Based on the training expert analysis results and the initial expert analysis results, the initial second agent is fine-tuned to obtain the fine-tuned second agent. The second intelligent agent is used to predict the test medical record information and the test expert reasoning chain information to obtain the prediction expert analysis results. Accuracy analysis is performed on the prediction expert analysis results and the test expert analysis results to obtain a second analysis result; Based on the second analysis results, the fine-tuned second agent is iteratively updated until a second agent that conforms to the corresponding expert's thinking pattern is obtained.

4. The method according to claim 1, characterized in that, The method of analyzing the target medical record information using multiple expert digital avatar units to obtain multiple expert analysis results also includes: The third and fourth prompt messages are determined based on the type of the expert digital avatar unit; Based on the third prompt information, the target medical record information is decomposed and analyzed by the multiple expert digital clone units to obtain multiple first expert reasoning chain information. Based on the fourth prompt information, the multiple expert digital clone units are used to analyze the target medical record information and the corresponding first expert reasoning chain information to obtain the analysis results of the multiple experts.

5. The method according to claim 1, characterized in that, The method further includes: Multiple first expert digital avatar units are used to deconstruct and reason about the target medical record information to obtain multiple expert reasoning chain information; each first expert digital avatar unit includes a first intelligent agent, which is used to deconstruct and reason about the target medical record information to obtain the expert reasoning chain information. The target medical record information and the information from multiple expert reasoning chains are used to make a comprehensive decision to obtain the target analysis result.

6. The method according to claim 1, characterized in that, The method of analyzing the target medical record information using multiple expert digital avatar units to obtain multiple expert analysis results also includes: Based on the types of the multiple expert digital avatar units, multiple types of reference case information are obtained by retrieving from a pre-built knowledge base; Based on the multiple types of reference case information, the first intelligent agent in the multiple expert digital avatar units is used to decompose and reason about the target medical record information to obtain multiple expert reasoning chain information. The second agent in each of the multiple expert digital avatar units analyzes the target medical record information and the corresponding expert reasoning chain information to obtain the analysis results of the multiple experts.

7. The method according to claim 1, characterized in that, The method of analyzing the target medical record information using multiple expert digital avatar units to obtain multiple expert analysis results also includes: The type settings for the expert digital avatar unit include prompts with fixed format requirements; The target medical record information is output according to the prompts in the fixed format requirements using the multiple expert digital avatar units, resulting in multiple fixed format output texts; The text structure of the multiple fixed-format output texts is parsed to obtain the multiple expert reasoning chain information and the multiple expert analysis results.

8. The method according to claim 1, characterized in that, The inference chain information is a structured tag sequence, which contains multiple tag pairs ordered according to the inference process.

9. The method according to claim 1, characterized in that, The reasoning chain information is a visual reasoning flowchart, which graphically displays the logical relationships in the reasoning process.

10. A multidisciplinary consultation digital avatar system based on LLM, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the LLM-based multidisciplinary consultation digital clone method as described in any one of claims 1-9.

Images