Multi-agent diagnosis planning device and method based on consultation thought flow

Abstract

The present application relates to the field of artificial intelligence and biomedical technology, and provides a multi-agent diagnosis planning device and method based on consultation thought process, the multi-agent diagnosis planning device comprises: a diagnosis planning agent, a pathological section analysis agent, an in-hospital information aggregation agent, an information search agent, a knowledge base search agent, a criticism agent, an arbitration agent and a risk assessment agent; each agent realizes data interaction through a dynamic priority message bus, the criticism agent and the arbitration agent form a progressive check closed loop: after the criticism agent outputs a questioning evidence chain, the arbitration agent triggers diagnosis correction only when the number of questioning items is greater than 3, otherwise, the arbitration agent outputs the final diagnosis based on preset rules. By constructing a multi-agent collaborative diagnosis framework, the present application simulates the thought process of multi-disciplinary expert cooperation in clinical consultation, and realizes the full-process intelligentization from medical data acquisition, analysis to diagnosis decision.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and biomedical technology, and in particular to a multi-agent diagnostic planning device and method based on a consultation thinking process. Background Technology

[0002] In the field of modern medical diagnostics, the achievement of precision diagnosis relies heavily on the collaborative integration of multi-dimensional information, including the patient's clinical data throughout the hospital cycle, the microscopic morphological characteristics of pathological sections, the latest medical literature evidence, and treatment guidelines. With the explosive growth of medical data and the deepening of clinical specialties, the traditional consultation model, which is mainly based on manual consultation, is gradually revealing its core pain points such as low efficiency, strong subjectivity, and insufficient information integration. There is an urgent need to build an automated and standardized diagnostic planning system through artificial intelligence technology.

[0003] Current AI-assisted diagnostic technologies in healthcare still suffer from the following end-to-end deficiencies, failing to meet the precision and standardization requirements of clinical consultations: First, the collaborative logic lacks clinical adaptability; multi-agent scheduling lacks case-type differentiation rules, making it impossible to prioritize urgent / critical information (such as tumor pathology slides); second, pathological analysis lacks collaboration and dynamic monitoring; toolsets operate independently, lacking result cross-verification mechanisms and the ability to track lesion temporal changes, resulting in a lack of stability assessment; third, there is an imbalance between information integration efficiency and compliance, with high latency in in-hospital data acquisition, non-standard desensitization, and low accuracy in external literature retrieval, failing to form a comprehensive, high-quality evidence chain; fourth, diagnostic verification and risk quantification are lacking, with no critical questioning mechanism, vague arbitration rules, and no quantification of diagnostic uncertainty, failing to provide doctors with risk references; fifth, there is no closed-loop iterative optimization capability; after diagnostic results are generated, the diagnostic plan cannot be automatically iterated and optimized based on verification feedback, requiring manual intervention for adjustments.

[0004] Therefore, there is an urgent need to build a multi-agent diagnostic planning technology based on the consultation thinking process to solve the problems of information silos, inefficient collaboration, and high diagnostic risks in existing technologies, and to meet the clinical needs of full-dimensional information integration, standardized diagnostic planning, and precise risk assessment. Summary of the Invention

[0005] This invention provides a multi-agent diagnostic planning device and method based on the consultation thinking process. By constructing a multi-agent collaborative diagnostic framework, this invention simulates the thinking process of multidisciplinary experts collaborating in clinical consultation, and realizes intelligentization of the entire process from medical data collection and analysis to diagnostic decision-making.

[0006] This invention provides a multi-agent diagnostic planning device based on a consultation thinking process, including: a diagnostic planning agent, a pathological slide analysis agent, an in-hospital information aggregation agent, an information search agent, a knowledge base search agent, a critique agent, an arbitration agent, and a risk assessment agent; each agent interacts with data through a dynamic priority message bus; the dynamic priority message bus is configured to adjust the agent calling priority in real time according to the urgency of the case and the completeness of the information;

[0007] The diagnostic planning agent is configured as follows: driven by a deep learning model pre-trained on a pathological image dataset based on a visual Transformer architecture, it receives the case ID, formulates a diagnostic plan through the diagnostic stage-agent mapping table, and calls the analysis information of the pathological slide analysis agent, the hospital information aggregation agent, the information search agent, and the knowledge base search agent, and generates an initial diagnosis by combining the case information of the case ID;

[0008] The pathological slide analysis agent is configured to: receive the slide ID, call preset tools to analyze the pathological slide, and output structured results;

[0009] The intelligent agent for aggregating hospital information is configured to: receive patient IDs and obtain patient information within the hospital through database queries and system interface calls;

[0010] The information search agent is configured to receive search commands and access certified medical professional databases and academic search engines to obtain external information.

[0011] The knowledge base search agent is configured to receive search instructions and generate supporting documents by semantically retrieving the built-in knowledge base.

[0012] The critical agent is configured to receive initial diagnostic and analytical information and generate a critical report; the analytical information includes structured results, patient in-hospital information, external information, and supporting literature.

[0013] The arbitration agent is configured as follows: it receives initial diagnosis, analysis information and critique report, judges the validity of diagnosis based on arbitration rules based on pathological diagnosis standards, evidence completeness and literature support, outputs a diagnosis result that is judged to be valid, and triggers diagnosis correction for the diagnosis result that is judged to be invalid when there are more than 3 objections in the critique report.

[0014] The risk assessment agent is configured to receive initial diagnostic and analysis information, utilize a machine learning risk prediction model trained based on the patient's time-series clinical data to assess the risk of disease progression and treatment, simultaneously generate a diagnostic uncertainty index, and feed back the risk assessment results in real time to the diagnostic planning agent and arbitration agent in JSON format via a standardized API interface to assist in further diagnosis and decision-making.

[0015] Furthermore, the diagnostic planning agent includes a persistent memory module configured to record information throughout the entire case analysis process, and selectively perform at least one of the following actions based on the case type and the current information completeness:

[0016] The system calls upon the hospital's information aggregation intelligence until it determines that there is sufficient diagnostic information. The criteria for determining sufficient information are: the information covers medical history, imaging reports, and pathological slide analysis results, and the missing information rate is <5%. The missing information rate is calculated as: number of missing key fields / total number of preset required fields × 100%.

[0017] A dynamic priority invocation order is adopted to invoke the pathology slide analysis intelligent agent to perform slide analysis. The dynamic priority invocation order is as follows: tumor cases are invoked first, and non-tumor cases first invoke the hospital information aggregation intelligent agent, and then invoke the pathology slide analysis intelligent agent.

[0018] Invoke the information search agent and the knowledge base search agent to obtain external information and supporting documents;

[0019] The initial diagnosis is generated by comprehensively calling the analysis information.

[0020] Furthermore, in the configuration of the pathological slide analysis agent, the preset tools include a database query tool, a whole slide classification and recognition tool, a tissue segmentation tool, a target detection tool, a region description tool, and a dynamic monitoring tool; the logic for calling the preset tools is as follows: the tool is called through the StdIO transmission mechanism, and the trigger condition is: the pathological slide analysis agent automatically retryes if there is no data feedback within 10 seconds after receiving the slide ID, and the transmission format of the tool call result is JSON;

[0021] The database query tool is configured to: locate and read slide data; trigger a scan notification when the slide data is not yet digitized; the slide data format conforms to the DICOM standard; the scan notification is sent via text pop-up and voice prompt; and the triggering delay is less than 5 minutes.

[0022] The whole-slice classification and recognition tool is configured as follows: a multi-instance learning (MIL) model based on a multi-stage attention mechanism outputs disease probability and coordinates of suspicious regions; the coordinates of suspicious regions are physical coordinates in mm.

[0023] The tissue segmentation tool is configured to: be based on the U-Net++ deep learning model, and output the tissue type and distribution parameters within the segmented region; the distribution parameters include area proportion and morphological irregularity.

[0024] The target detection tool is configured to: identify and count specific cell types within the detection area based on the YOLOv8 model;

[0025] The region description tool is configured to generate textual descriptions of slice regions based on the PathVLM visual language model; the textual descriptions of slice regions include tissue morphology, cell characteristics, and diagnostic tendencies.

[0026] The dynamic monitoring tool is configured as follows: based on time series analysis and LSTM machine learning algorithm, it dynamically monitors and analyzes the changes of pathological sections at different time points and outputs a lesion stability score; the changes include changes in cell proliferation rate and lesion area; the lesion stability score is derived by weighting the cell morphology change rate and the lesion area variation coefficient, with the cell morphology change rate and the lesion area variation coefficient having weight ratios of 60% and 40%, respectively; the lesion stability score ranges from 0 to 100.

[0027] Furthermore, the intelligent information aggregation body within the institute includes database query tools, system interface call tools, and information structuring modules;

[0028] The database query tool is configured to automatically generate SQL statements to query patient data; the SQL statements comply with medical data privacy regulations and undergo field anonymization.

[0029] The system interface call tool is configured to: obtain standardized medical information through the PACS / HIS / LIS system interface; after receiving the request, the system interface shall complete the data query, processing and return the result within 15 seconds;

[0030] The information structuring module is configured to integrate multi-source data into a unified format diagnostic support document; the fields of the diagnostic support document include anonymized patient ID, basic information, medical history, imaging report, and laboratory results.

[0031] Furthermore, upon receiving search instructions, supporting documents are generated through the built-in knowledge base of semantic retrieval, including:

[0032] The search instructions are transformed into feature vectors using a PubMedBERT medical terminology embedding model fine-tuned based on the UMLS terminology database.

[0033] Semantic retrieval is performed in the built-in knowledge base by calculating the cosine similarity between feature vectors; the cosine similarity threshold is ≥0.85.

[0034] Extract the built-in knowledge base texts corresponding to the Top-N similarity feature vectors as supporting evidence; the built-in knowledge base texts include high-impact factor journal articles from the past three years; N≥5 in Top-N; high impact factor≥5;

[0035] The system uses a fine-tuned large model based on GPT-4 in the medical field to generate structured answers based on supporting evidence, and then generates supporting literature based on the structured answers. The structured answers include a summary of supporting evidence and an analysis of their relevance to the search instructions.

[0036] Furthermore, the critical agent is configured to perform the following operations:

[0037] Based on the diagnostic criteria, the sufficiency of the initial diagnosis in supporting the diagnostic conclusion was verified to obtain the verification conclusion. The diagnostic criteria were: among the supporting evidence for the initial diagnosis, the confidence level of the pathological slide analysis results was <70%, or key differential disease types were omitted.

[0038] Based on the verification conclusions and analysis information, differential diagnosis suggestions are proposed;

[0039] Generate a critical report that includes challenges to the chain of evidence and recommendations for further investigation.

[0040] Furthermore, the arbitration agent is configured to execute the following judgment logic using a finely tuned LLaMA-2 model in the pathology field to determine the validity of the diagnosis:

[0041] The completeness of the examination items required for diagnosis should be checked; the standard for completeness is: the missing rate of core examination items <10%;

[0042] Verify the exclusion criteria for differential diagnosis; the exclusion criteria are: excluding diseases with a high probability of ≥90% in the differential diagnosis.

[0043] Assess the degree of support from literature evidence for the diagnostic conclusion; the minimum standard for support is: ≥2 articles from high-impact factor literature in the past three years.

[0044] Verify whether the number of logical chain nodes in the diagnostic reasoning logic chain is greater than the preset number, which is 3 to 5.

[0045] A multi-agent diagnostic planning method based on a consultation thinking process includes:

[0046] S1. The diagnostic planning agent receives the case ID and initializes the diagnostic process; the diagnostic process includes case type labeling and agent call priority setting.

[0047] S2. Perform multi-agent cooperative analysis:

[0048] S21. Call the hospital information aggregation intelligent agent to obtain the patient's hospital information;

[0049] S22. Call the pathology slide analysis agent to perform multi-dimensional analysis of digital slides;

[0050] S23. Call the knowledge base / information search agent to obtain supporting documents and external information;

[0051] S24. Generate an initial diagnosis including a chain of evidence; the chain of evidence is ordered according to clinical decision weights, in the following order: pathological evidence with a weight of 40%, clinical evidence with a weight of 30%, and literature evidence with a weight of 30%.

[0052] S3. The critical agent submits a critical report on the initial diagnosis;

[0053] S4. The arbitration agent receives and, based on the initial diagnosis, the critical report, and the diagnostic uncertainty index generated by the risk assessment agent, judges the validity of the diagnosis according to the judgment logic.

[0054] S5. If the arbitration agent determines that the diagnosis is valid, the final diagnosis is output; if the arbitration agent determines that the diagnosis is invalid, the arbitration judgment result and related information are fed back to step S1, triggering iterative optimization of the diagnosis process; after updating the diagnosis plan based on the arbitration judgment result and related information, step S1 executes steps S2, S3 and S4 again.

[0055] S6. Repeat steps S2 to S5 until the diagnostic termination condition is met.

[0056] Furthermore, the pathological slide analysis agent is invoked to perform multi-dimensional analysis of digital slides, including:

[0057] After receiving the slice ID, verify the availability of the digital slice. The verification criteria for digital slice availability are: the digital slice is in DICOM format, the horizontal pixel count is not less than 2000, and there is no data corruption.

[0058] The following steps are performed sequentially on the available slices: full slice classification using a multi-instance learning (MIL) model based on a multi-stage attention mechanism; tissue segmentation using a U-Net++ deep learning model; object detection using a YOLOv8 model; and region description using a PathVLM visual language model. This generates a multi-dimensional analysis report that includes disease probability distribution, coordinates of suspicious regions, tissue composition ratio, and morphological description.

[0059] Furthermore, the diagnostic termination criteria include a combination of at least three of the following:

[0060] (1) The completeness of the evidence chain for the arbitration intelligent agent is ≥90% and the exclusion rate of key identification diagnosis is 100%; the calculation method for completeness is: number of valid evidence / total number of required evidence × 100%;

[0061] (2) The fluctuation range of the diagnostic uncertainty index output by the risk assessment agent is less than 5% for three consecutive iterations; the fluctuation range is calculated as: |current index - previous index| / previous index × 100%;

[0062] (3) The latest evidence returned by the knowledge base search agent matches the current diagnostic conclusion with a degree of greater than 95%, and the number of high-impact factor literatures in the past three years is greater than or equal to 3; the degree of matching is calculated based on cosine similarity.

[0063] (4) The lesion stability score output by the dynamic monitoring tool of the pathological section analysis agent is >85 points and there are no new high-risk characteristics;

[0064] (5) The coefficient of variation of the key clinical indicators obtained by the in-hospital information aggregation intelligent agent for two consecutive tests is <15%; key clinical indicators include tumor markers and inflammatory markers; the formula for calculating the coefficient of variation is: standard deviation / mean × 100%;

[0065] The maximum number of iterations is set to 3 to 5. If the above combination of judgments is not satisfied even after reaching the maximum number of iterations, the latest thinking result generated in the current iteration is output.

[0066] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0067] First, it deeply integrates the multi-agent collaboration mechanism with the consultation thinking process, and realizes the intelligentization of the entire process of pathological diagnosis from data collection and analysis to conclusion generation through modular tool configuration. This breaks through the limitations of single algorithms or human experience in the traditional diagnosis model and improves the accuracy and efficiency of diagnosing complex cases.

[0068] Secondly, each intelligent agent tool adopts industry-leading deep learning and natural language processing technologies, such as the multi-stage attention mechanism MIL model, U-Net++ segmentation network, YOLOv8 object detection algorithm and PathVLM visual language model, to ensure the multi-dimensional accuracy of pathological slide analysis. At the same time, it combines time series analysis to realize dynamic monitoring of lesions, providing more comprehensive quantitative evidence for diagnosis.

[0069] Third, the hospital's information aggregation intelligent agent seamlessly connects with the medical system through standardized interfaces, enabling efficient integration and structured processing of multi-source patient data. It strictly adheres to medical data privacy regulations, providing complete clinical background information support for diagnosis while ensuring data security.

[0070] Fourth, the knowledge base search agent is based on a professional medical terminology embedding model and semantic retrieval technology to accurately locate high-value literature evidence. It also uses a fine-tuned large model in the medical field to generate structured answers, so that the diagnostic conclusions have a solid evidence-based medical foundation and effectively reduce the risk of misdiagnosis caused by the lag in knowledge updates.

[0071] Fifth, the critical and arbitration agents construct a rigorous diagnostic quality control system. Through diagnostic standard verification, differential diagnostic suggestions, and multi-dimensional judgment logic execution, the diagnostic process is dynamically monitored and optimized to ensure the integrity of the diagnostic evidence chain, the rigor of the reasoning logic, and the reliability of the conclusions.

[0072] Sixth, the diagnostic process design incorporates an iterative optimization mechanism and clear termination conditions. Through cyclical execution of intelligent agent collaborative analysis, critical evaluation, and arbitration judgment, combined with a maximum loop count limit, it ensures diagnostic sufficiency while avoiding ineffective iterations, balancing diagnostic accuracy and efficiency, and providing a scientific and standardized intelligent solution for clinical pathology diagnosis.

[0073] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0074] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0075] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0076] Figure 1 A schematic diagram of a multi-agent diagnostic planning device based on a consultation thinking process;

[0077] Figure 2 This is a schematic diagram illustrating the composition of a toolset invoked via the model context protocol.

[0078] Figure 3 This is a schematic diagram illustrating the steps of a multi-agent diagnostic planning method based on a consultation-based thinking process. Detailed Implementation

[0079] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0080] This invention provides a multi-agent diagnostic planning device based on a consultation thinking process, such as... Figure 1As shown, it includes: a diagnostic planning agent, a pathology slide analysis agent, an in-hospital information aggregation agent, an information search agent, a knowledge base search agent, a critique agent, an arbitration agent, and a risk assessment agent; each agent interacts with data through a dynamic priority message bus; the dynamic priority message bus is configured to adjust the agent calling priority in real time according to the urgency of the case and the completeness of the information;

[0081] The diagnostic planning agent is configured as follows: driven by a deep learning model pre-trained on a pathological image dataset based on a visual Transformer architecture, it receives the case ID, formulates a diagnostic plan through the diagnostic stage-agent mapping table, and calls the analysis information of the pathological slide analysis agent, the hospital information aggregation agent, the information search agent, and the knowledge base search agent, and generates an initial diagnosis by combining the case information of the case ID;

[0082] The pathological slide analysis agent is configured to: receive the slide ID, call preset tools to analyze the pathological slide, and output structured results;

[0083] The intelligent agent for aggregating hospital information is configured to: receive patient IDs and obtain patient information within the hospital through database queries and system interface calls;

[0084] The information search agent is configured to receive search commands and access certified medical professional databases and academic search engines to obtain external information.

[0085] The knowledge base search agent is configured to receive search instructions and generate supporting documents by semantically retrieving the built-in knowledge base.

[0086] The critical agent is configured to receive initial diagnostic and analytical information and generate a critical report; the analytical information includes structured results, patient in-hospital information, external information, and supporting literature.

[0087] The arbitration agent is configured as follows: it receives initial diagnosis, analysis information and critique report, judges the validity of diagnosis based on arbitration rules based on pathological diagnosis standards, evidence completeness and literature support, outputs a diagnosis result that is judged to be valid, and triggers diagnosis correction for the diagnosis result that is judged to be invalid when there are more than 3 objections in the critique report.

[0088] The specific logic for judging the validity of a diagnosis is as follows:

[0089] a) Completeness rule for examination items: The missing rate of core examination items is lower than the first threshold (e.g., <10%); the first threshold is set between 5% and 15% according to the case type, with 10% being the preferred value; the basis for this value is: in clinical practice, it is believed that when the missing rate is lower than 10%, it does not affect the judgment of the disease subject.

[0090] b) Differential diagnosis exclusion rules: exclude differential diseases with a probability higher than the second threshold (e.g., ≥90%);

[0091] c) Literature support rule: The number of high-impact factor literatures supported in the past three years reaches the third threshold (e.g., ≥2 articles);

[0092] d) Diagnostic uncertainty rule: The diagnostic uncertainty index is below the fourth threshold (e.g., <30).

[0093] When conditions a), b), c), and d) are met simultaneously, the diagnostic result is output; otherwise, if there are more than 3 objections in the critique report, the diagnosis correction for the invalid diagnostic result is triggered.

[0094] The risk assessment agent is configured as follows: it receives initial diagnostic and analysis information, uses a machine learning risk prediction model trained based on the patient's time-series clinical data to assess the risk of disease progression and treatment, generates a diagnostic uncertainty index, and feeds back the risk assessment results in real time to the diagnostic planning agent and the arbitration agent in JSON format through a standardized API interface to assist in further diagnosis and decision-making; the diagnostic uncertainty index ranges from 0 to 100.

[0095] The specific implementation is as follows: For example, a 65-year-old male patient was admitted to the hospital due to persistent chest pain for 2 hours. Case ID: P20251001, Patient ID: PT789456, Section ID: S12345;

[0096] First, analyze the dynamic priority messages. Based on the case urgency (chest pain highly suspected to be myocardial infarction, urgency rating 90 / 100) and information completeness (initial information only contains symptoms and basic signs, completeness 30 / 100), set the intelligent agent calling priority in real time: pathological slide analysis intelligent agent (level 1) > hospital information aggregation intelligent agent (level 2) > knowledge base search intelligent agent (level 3) > information search intelligent agent (level 4);

[0097] Then, the diagnostic planning agent performs the following: receiving case ID: P20251001, calling the diagnostic stage-agent mapping table (mapping the chest pain investigation stage with pathological slide analysis, in-hospital information aggregation, and knowledge base search); driving the model based on the visual Transformer architecture (pre-trained on the TCGA pathological image dataset) to generate an initial diagnostic plan: prioritizing the acquisition of pathological slide analysis results and the patient's in-hospital historical data;

[0098] Then, multi-agent collaborative analysis was performed: First, the pathological slide analysis agent received slide ID: S12345, called the digital pathology analysis tool (supporting HE staining quantitative analysis), and output structured results: coagulative necrosis of cardiomyocytes in the anterior wall of the left ventricle (area approximately 25%), consistent with the pathological characteristics of acute myocardial infarction; then, the hospital information aggregation agent received patient ID: PT789456, and obtained hospital information through the HIS system interface: 10-year history of hypertension (maximum 180 / 110 mmHg), 5-year history of type 2 diabetes, and ST-T segment depression on an outpatient electrocardiogram 3 days ago; then, the knowledge base search agent received the search command: pathological characteristics of acute myocardial infarction and risk of hypertension combined with diabetes, and used the built-in UpToDate knowledge base for semantic retrieval to generate supporting literature: "2025 AHA Guidelines for the Diagnosis of Myocardial Infarction"; then, the information search agent received the search command: new advances in the treatment of myocardial infarction in 2025, called the PubMed database, and obtained the 2025 NEJM research content;

[0099] Next, the critical agent receives the initial diagnosis (acute ST-segment elevation myocardial infarction) and analysis information, and generates a critical report. The critical report includes the following questions:

[0100] Question 1: The pathological section only covers the anterior wall of the left ventricle and does not include the interventricular septum area;

[0101] Question 2: The hospital's information did not provide the medication history (such as anticoagulants) taken within 48 hours prior to the onset of illness;

[0102] Question 3: Supporting literature does not clearly define the quantitative relationship between necrotic area and prognosis in diabetic patients;

[0103] Question 4: The contraindications for the new drug in the external information were not matched with the patients' renal function data;

[0104] Next, the arbitration agent receives the initial diagnosis, analysis information, and critique report (4 or more objections), triggering a diagnosis correction. Based on the arbitration rules (pathological diagnostic criteria + completeness of evidence + support from literature), it is determined that supplementary data on ventricular septum slicing analysis and renal function tests are required.

[0105] Next, the risk assessment agent receives the initial diagnosis and analysis information, calls the LSTM risk prediction model trained based on time-series clinical data, and outputs: disease progression risk, treatment risk, and diagnostic uncertainty index; the disease progression risk is: 22% probability of re-infarction within 6 months; the treatment risk is: 18% risk of bleeding with conventional anticoagulation regimen; the diagnostic uncertainty index is: 65 / 100 (due to incomplete pathological section area and missing medication history); and is fed back to the diagnosis planning agent and arbitration agent in JSON format through a standardized API interface.

[0106] Finally, based on the arbitration results and risk assessment, the diagnostic planning agent re-invoked the pathology slide analysis agent (supplementing the interventricular septum slide ID: S12346) and the in-hospital information aggregation agent (obtaining renal function indicators: creatinine 135μmol / L). Ultimately, the arbitration agent output a revised diagnosis: acute ST-segment elevation myocardial infarction (involving the anterior wall of the left ventricle and the interventricular septum), and recommended the use of a novel P2Y12 inhibitor (adjusting the dose based on renal function).

[0107] The working principle of the above technical solution is as follows: When the case ID is received, the diagnostic planning agent (driven by a deep learning model pre-trained on a pathological image dataset based on the visual Transformer architecture) is launched. It first obtains basic case information based on the case ID and formulates a preliminary diagnostic plan in combination with the diagnostic stage-agent mapping table. The dynamic priority message bus adjusts the calling priority of each relevant agent in real time according to the urgency of the case (e.g., the urgency of acute and critical cases is high) and the completeness of the current information (e.g., the completeness is low if the initial information is less).

[0108] Based on the diagnostic plan and dynamic priorities, the diagnostic planning agent sequentially or in parallel invokes the pathology slide analysis agent, the hospital information aggregation agent, the information search agent, and the knowledge base search agent. The pathology slide analysis agent receives the slide ID and uses preset tools (such as digital pathology image analysis software) to analyze the pathology slide, outputting structured results including cell morphology, tissue structure, and suspected lesion areas. The hospital information aggregation agent receives the patient ID and retrieves the patient's electronic medical records, laboratory test results, past medical history, medication history, and other hospital information through hospital database queries and system interface calls. The information search agent executes search commands issued by the diagnostic planning agent (such as those targeting specific...). The system retrieves the latest research on symptoms or suspected diseases, and accesses certified medical databases (such as PubMedCentral and CochraneLibrary) and academic search engines to obtain the latest external medical information and research progress. The knowledge base search agent then uses semantic retrieval of built-in medical knowledge bases (such as clinical guidelines and disease databases) according to search instructions to generate relevant literature and guideline summaries that support the initial diagnostic approach. These structured results returned by each agent, along with the patient's in-hospital information, external information, and supporting literature, constitute the analysis information, which is fed back to the diagnostic planning agent. The diagnostic planning agent combines the original case information with this analysis information to generate an initial diagnosis.

[0109] Subsequently, the critical agent receives the initial diagnosis and all analysis information, examines it, and focuses on checking the completeness of the evidence chain, the rigor of logical reasoning, and the consistency between the diagnosis and various information, and then puts forward a critical report containing the points of contention.

[0110] The arbitration agent receives initial diagnosis and analysis information, as well as the critique report from the critique agent. Simultaneously, the risk assessment agent also receives initial diagnosis and analysis information, and uses a machine learning risk prediction model trained based on the patient's time-series clinical data to assess the patient's disease progression risk (such as the probability of deterioration and prognosis) and treatment risk (such as potential complications and adverse drug reaction risks). It also generates a diagnostic uncertainty index (ranging from 0 to 100, with higher values ​​indicating greater diagnostic uncertainty). The risk assessment results are fed back to the diagnosis planning agent and the arbitration agent in real time via a standardized API interface in JSON format, assisting them in further diagnosis and decision-making.

[0111] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, it is possible to organically integrate and coordinate intelligent agents from different professional fields by simulating the thinking process of multidisciplinary consultation, and realize the full-process automation and intelligence from case information acquisition, multi-dimensional data integration to diagnosis generation, critical verification and risk assessment.

[0112] In one embodiment, the diagnostic planning agent includes a persistent memory module configured to record information throughout the case analysis process and selectively perform at least one of the following actions based on the case type and the current information completeness:

[0113] The system calls upon the hospital's information aggregation intelligence until it determines that there is sufficient diagnostic information. The criteria for determining sufficient information are: the information covers medical history, imaging reports, and pathological slide analysis results, and the missing information rate is <5%. The missing information rate is calculated as: number of missing key fields / total number of preset required fields × 100%.

[0114] A dynamic priority invocation order is adopted to invoke the pathology slide analysis intelligent agent to perform slide analysis. The dynamic priority invocation order is as follows: tumor cases are invoked first, and non-tumor cases first invoke the hospital information aggregation intelligent agent, and then invoke the pathology slide analysis intelligent agent.

[0115] Invoke the information search agent and the knowledge base search agent to obtain external information and supporting documents;

[0116] The initial diagnosis is generated by comprehensively calling the analysis information.

[0117] The working principle of the above technical solution is as follows: The diagnostic planning agent records information throughout the entire case analysis process through a persistent memory module, and selectively executes actions based on the case type and the current information completeness: First, it determines whether the diagnostic information is sufficient. The standard for sufficient information is that it covers medical history, imaging reports, and pathological slide analysis results, and the information missing rate (number of missing key fields / total number of preset required fields × 100%) is < 5%. If it is insufficient, the internal information aggregation agent is called until sufficient information is available. When calling the pathological slide analysis agent, a dynamic priority calling order is adopted, that is, tumor cases are called first, and non-tumor cases call the internal information aggregation agent first and then the pathological slide analysis agent. At the same time, the information search agent and the knowledge base search agent are called to obtain external information and supporting literature. Finally, the initial diagnosis is generated by integrating the information returned by each agent.

[0118] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the entire case analysis process can be traced through the persistent memory module, ensuring the transparency and reproducibility of the diagnostic plan; by dynamically judging the completeness of information and adjusting the intelligent agent calling logic in combination with the case type, tumor cases are given priority to obtain pathological slide analysis results to quickly clarify the nature of the lesion, while non-tumor cases first integrate basic information within the hospital and then conduct specialized analysis, effectively improving the targeting and efficiency of information acquisition.

[0119] In one embodiment, such as Figure 2 As shown, the preset tools in the pathological slide analysis agent configuration include a database query tool, a whole slide classification and recognition tool, a tissue segmentation tool, a target detection tool, a region description tool, and a dynamic monitoring tool. The logic for calling the preset tools is as follows: the tool is called through the StdIO transmission mechanism. The trigger condition is that the pathological slide analysis agent will automatically retry if there is no data feedback within 10 seconds after receiving the slide ID. The transmission format of the tool call result is JSON.

[0120] The database query tool is configured to: locate and read slide data; trigger a scan notification when the slide data is not yet digitized; the slide data format conforms to the DICOM standard; the scan notification is sent via text pop-up and voice prompt; and the triggering delay is less than 5 minutes.

[0121] The whole-slice classification and recognition tool is configured as follows: a multi-instance learning (MIL) model based on a multi-stage attention mechanism outputs disease probability and coordinates of suspicious regions; the coordinates of suspicious regions are physical coordinates in mm.

[0122] The tissue segmentation tool is configured to: be based on the U-Net++ deep learning model, and output the tissue type and distribution parameters within the segmented region; the distribution parameters include area proportion and morphological irregularity.

[0123] The target detection tool is configured to: identify and count specific cell types within the detection area based on the YOLOv8 model;

[0124] The region description tool is configured to generate textual descriptions of slice regions based on the PathVLM visual language model; the textual descriptions of slice regions include tissue morphology, cell characteristics, and diagnostic tendencies.

[0125] The dynamic monitoring tool is configured as follows: based on time series analysis and LSTM machine learning algorithm, it dynamically monitors and analyzes the changes of pathological sections at different time points and outputs a lesion stability score; the changes include changes in cell proliferation rate and lesion area; the lesion stability score is derived by weighting the cell morphology change rate and the lesion area variation coefficient, with the cell morphology change rate and the lesion area variation coefficient having weight ratios of 60% and 40%, respectively; the lesion stability score ranges from 0 to 100.

[0126] The working principle of the above technical solution is as follows: the pathological slide analysis agent calls the toolset through the model context protocol (StdIO transmission mechanism). If no data feedback is received within 10 seconds of receiving the slide ID, it automatically retryes. The tool call results are transmitted in JSON format. The workflow, in a specific example, is as follows:

[0127] When conducting cancer screening at a hospital, the pathology slide analysis AI receives a slide ID from a suspected lung cancer patient. The database query tool begins to locate and read the slide data. If it finds that the slide data has not yet been digitized, the system will pop up a text window within 5 minutes, displaying "The slide data of the suspected lung cancer patient with the number [specific number] has not been digitized. Please scan it in time." At the same time, there will be a voice prompt to remind the staff to perform the scanning operation to obtain slide data that conforms to the DICOM standard.

[0128] After obtaining the slide data of the lung cancer patient, the whole slide classification and recognition tool started working based on the MIL model of the multi-stage attention mechanism. After analysis, it output that the probability of the patient having lung cancer is 80%, and at the same time, it gave the coordinates of the suspicious area, such as (12.5mm, 15.2mm). This indicates that there may be a lesion at the physical coordinate position on the slide, which provides direction for more accurate subsequent analysis.

[0129] For suspicious areas identified by the whole-slice classification and identification tool, the tissue segmentation tool analyzes the area based on the U-Net++ deep learning model. For example, it identifies the tissue types in the area as cancerous tissue and normal lung tissue, and outputs distribution parameters. The cancerous tissue accounts for 60% of the area and has a high degree of morphological irregularity. This data can help doctors make a preliminary judgment on the severity of the lesion.

[0130] After identifying the suspicious area and tissue type, the target detection tool uses the YOLOv8 model to identify and count specific cell types within the area. For example, if 500 cancer cells and 300 normal cells are detected in the area, doctors can further assess the condition based on the number and proportion of cancer cells.

[0131] The region description tool, based on the PathVLM visual language model, generates text descriptions for the slice regions analyzed above. For the suspicious region of this lung cancer patient, the description could be: In terms of tissue morphology, the cancer cells show irregular clusters with unclear boundaries from the surrounding normal lung tissue; the cell characteristics are that the cancer cells are large and have prominent nucleoli; the diagnosis is highly suspected to be lung cancer. Such a text description provides doctors with intuitive and detailed information.

[0132] After a period of treatment, the patient's pathological slides were analyzed again. The dynamic monitoring tool, based on time series analysis and LSTM machine learning algorithms, compared the pathological slides at different time points. For example, in the first test, the cancer cell proliferation rate was 10% per day, and the lesion area was 20 square millimeters. After one week of treatment, the second test showed that the cancer cell proliferation rate had decreased to 5% per day, and the lesion area had become 18 square millimeters. The dynamic monitoring tool calculated a lesion stability score of 70 points based on a weighted average of the cell morphology change rate and the lesion area variation coefficient (assuming a weight of 60% for cell morphology change rate and 40% for lesion area variation coefficient). These dynamic monitoring and analysis results were integrated into structured results, allowing doctors to judge the effectiveness of the treatment plan and adjust the treatment strategy in a timely manner.

[0133] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, the pathological slide analysis process can be fully automated and intelligently upgraded through multi-agent collaboration and precise calling of toolsets.

[0134] In one embodiment, the in-hospital information aggregation intelligent agent includes a database query tool, a system interface call tool, and an information structuring module;

[0135] The database query tool is configured to automatically generate SQL statements to query patient data; the SQL statements comply with medical data privacy regulations and undergo field anonymization.

[0136] The system interface call tool is configured to: obtain standardized medical information through the PACS / HIS / LIS system interface; after receiving the request, the system interface shall complete the data query, processing and return the result within 15 seconds;

[0137] The information structuring module is configured to integrate multi-source data into a unified format diagnostic support document; the fields of the diagnostic support document include anonymized patient ID, basic information, medical history, imaging report, and laboratory results.

[0138] The working principle of the above technical solution is as follows: the database query tool can automatically generate SQL statements according to the set rules to query patient data. In the process of generating SQL statements, it strictly follows the medical data privacy specifications and performs anonymization processing on fields involving patient privacy to protect patient privacy; for example, sensitive information such as patient name and ID number is replaced or obfuscated, and then the generated SQL statement is used to retrieve the required patient data from the corresponding database.

[0139] The system interface calling tool connects to the interfaces of systems such as PACS (Pharmaceutical Archiving and Communication System), HIS (Hospital Information System), and LIS (Laboratory Information System) to send requests to these systems to obtain standardized medical information. After receiving the request, these systems will quickly perform data query operations, process the queried data according to their respective system rules, and are required to return the processed results to the system interface calling tool within 15 seconds. For example, it can obtain image-related information from the PACS system, patient medical process information from the HIS system, and test report information from the LIS system.

[0140] The information structuring module is responsible for receiving multi-source data, including patient data obtained from database query tools and standardized medical information returned by system interface call tools. It integrates this data, which may come from different sources and have different formats, and generates a diagnostic support document according to a set unified format. The fields of this diagnostic support document include anonymized patient IDs, as well as key medical information such as basic patient information, medical history, imaging reports, and laboratory results. Anonymization further protects patient privacy, ensuring that the patient's real identity information will not be exposed when using the diagnostic support document in the future. The final result is a diagnostic support document with a unified format, comprehensive content, and protection of patient privacy, providing strong support for medical diagnosis.

[0141] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, multi-source medical data can be efficiently integrated through database query tools and system interface calling tools. At the same time, secure diagnostic support documents can be constructed through desensitization and anonymization processing, which not only solves the problems of scattered data and inconsistent formats in traditional diagnosis, but also strictly protects patient privacy.

[0142] In one embodiment, a search instruction is received, and supporting documents are generated through a semantic retrieval built-in knowledge base, including:

[0143] The search instructions are transformed into feature vectors using a PubMedBERT medical terminology embedding model fine-tuned based on the UMLS terminology database.

[0144] Semantic retrieval is performed in the built-in knowledge base by calculating the cosine similarity between feature vectors; the cosine similarity threshold is ≥0.85.

[0145] Extract the built-in knowledge base texts corresponding to the Top-N similarity feature vectors as supporting evidence; the built-in knowledge base texts include high-impact factor journal articles from the past three years; N≥5 in Top-N; high impact factor≥5;

[0146] The system uses a fine-tuned large model based on GPT-4 in the medical field to generate structured answers based on supporting evidence, and then generates supporting literature based on the structured answers. The structured answers include a summary of supporting evidence and an analysis of their relevance to the search instructions.

[0147] The working principle of the above technical solution is as follows: The knowledge base search agent is configured to first perform instruction conversion. Specifically, it uses the PubMedBERT medical terminology-specific embedding model, which is fine-tuned based on the UMLS terminology database, to convert the user's search instructions from natural language form into feature vectors that are easy for computers to process. This model uses the professional medical terminology knowledge of the UMLS terminology database to fine-tune PubMedBERT, so that it can extract key features more accurately and convert them into corresponding vector representations when processing medical-related search instructions.

[0148] Next, semantic retrieval is performed. Specifically, the transformed feature vectors are compared with the data in the built-in knowledge base, and semantic retrieval is performed by calculating cosine similarity. Cosine similarity is used to measure the similarity between two vectors in a direction. A similarity threshold of ≥0.85 is set. Only data with a similarity of ≥0.85 will be further considered, thereby filtering out data that is semantically similar to the search command.

[0149] Next, evidence extraction continues. Specifically, from the data filtered by semantic retrieval, the built-in knowledge base texts corresponding to the Top-N similar feature vectors are extracted as supporting evidence. Here, the built-in knowledge base texts are limited to articles from high-impact factor journals in the past three years, and the high impact factor is ≥5, which ensures the timeliness and authority of the evidence. At the same time, N≥5 ensures that there are a sufficient number of relevant texts as evidence, providing rich materials for generating subsequent answers.

[0150] Finally, there's the answer generation. Specifically, it calls a large model fine-tuned for the medical field based on GPT-4 to generate a structured answer based on the extracted supporting evidence. This structured answer includes an evidence summary, which concisely summarizes the extracted knowledge base text, making it easy for users to quickly understand the key points of the evidence. It also includes a correlation analysis with search queries, explaining the connection between this evidence and the search query to the user, making the answer logical and well-founded.

[0151] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the medical terminology-specific embedding model fine-tuned by the UMLS terminology database can significantly improve the accuracy of medical search command conversion, allowing the computer to more accurately understand the user's intent; setting a cosine similarity threshold of 0.85 in the semantic retrieval stage can effectively filter low-relevance data and reduce interference from invalid information; limiting the evidence source to high-impact factor (≥5) journal articles from the past three years and ensuring N≥5 not only guarantees the timeliness, authority, and richness of the evidence, but also lays a reliable foundation for answer generation; and the structured answer generated based on the GPT-4 medical field fine-tuned large model, which includes evidence summaries and correlation analysis, improves the efficiency and quality of information retrieval and processing in the medical diagnostic planning process.

[0152] In one embodiment, the critical agent is configured to perform the following operations:

[0153] Based on the diagnostic criteria, the sufficiency of the initial diagnosis in supporting the diagnostic conclusion was verified to obtain the verification conclusion. The diagnostic criteria were: among the supporting evidence for the initial diagnosis, the confidence level of the pathological slide analysis results was <70%, or key differential disease types were omitted.

[0154] Based on the verification conclusions and analysis information, differential diagnosis suggestions are proposed;

[0155] Generate a critical report that includes challenges to the chain of evidence and recommendations for further investigation.

[0156] The working principle of the above technical solution is as follows: The critical agent first verifies the degree of support of the initial diagnosis for the diagnostic conclusion based on the given diagnostic criteria. The diagnostic criteria are set as follows: if the confidence level of the pathological slide analysis results in the supporting evidence of the initial diagnosis is less than 70%, or if the key disease type is omitted, then it is determined that the case information does not support the diagnostic conclusion sufficiently. Through this clear quantification and condition setting, it is possible to accurately determine whether the diagnostic conclusion is supported by sufficient case information.

[0157] Secondly, based on the verification conclusions and analysis information, differential diagnosis suggestions are proposed to help doctors consider possible disease types more comprehensively and accurately.

[0158] Finally, a critical report is generated, which integrates differential diagnosis recommendations. This report includes a chain of challenged evidence, detailing the basis and logical evidence for questioning the existing diagnostic conclusion. It also provides suggestions for supplementary examinations to guide further clarification of the diagnosis, help doctors improve the diagnostic process, and enhance diagnostic accuracy.

[0159] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, through the quantitative evaluation of the degree of support of the diagnostic conclusion by the critical agent, the risk of misdiagnosis caused by insufficient evidence or omission of key information can be effectively avoided, and reliable quality control can be provided for diagnostic decision-making.

[0160] In one embodiment, the arbitration agent is configured to execute the following decision logic to determine the validity of a diagnosis using a finely tuned LLaMA-2 model in the pathology domain:

[0161] The completeness of the examination items required for diagnosis should be checked; the standard for completeness is: the missing rate of core examination items <10%;

[0162] Verify the exclusion criteria for differential diagnosis; the exclusion criteria are: excluding diseases with a high probability of ≥90% in the differential diagnosis.

[0163] Assess the degree of support from literature evidence for the diagnostic conclusion; the minimum standard for support is: ≥2 articles from high-impact factor literature in the past three years.

[0164] Verify whether the number of logical chain nodes in the diagnostic reasoning logic chain is greater than the preset number, which is 3 to 5.

[0165] The working principle of the above technical solution is as follows: The arbitration agent works based on the LLaMA-2 large model that has been fine-tuned in the field of pathology, executes judgment logic, and judges the validity of the diagnosis. First, it checks the completeness of the examination items required for diagnosis. The standard is that the missing rate of core examination items is <10%. It judges whether the examination items on which the current diagnosis is based are comprehensive enough. The completeness is measured by calculating the proportion of missing core examination items. If the missing rate is within the specified range, the completeness of the examination items meets the requirements.

[0166] Next, the differential diagnosis exclusion is verified. Based on the standard of excluding ≥90% of high-probability differential diseases, other possible high-probability diseases are investigated. This step ensures that diseases that are likely to interfere with the diagnosis are fully considered and excluded during the diagnostic process, thereby improving the accuracy of the diagnosis.

[0167] Next, the support of literature evidence for the diagnostic conclusion is evaluated. The minimum standard is ≥2 articles with high impact factors in the past three years. The reliability of the diagnostic conclusion at the academic level is judged, and the credibility of the diagnostic conclusion is enhanced by relying on high-quality literature support.

[0168] Next, check whether the number of logical chain nodes in the diagnostic reasoning logic chain is greater than the preset number, which is 3 to 5. Determine whether the reasoning logic of the diagnostic process is rigorous and complete enough. If the number of logical chain nodes meets the preset value, it means that the reasoning process is more detailed and can better support the diagnostic conclusion.

[0169] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the arbitration agent can effectively simulate the comprehensive judgment process of the expert team in clinical consultation by controlling the completeness of the examination items, the exclusion of differential diagnoses, the support of literature evidence, and the rigor of the reasoning logic chain in a multi-dimensional way based on the fine-tuning of the large model.

[0170] A multi-agent diagnostic planning method based on a consultation thinking process includes:

[0171] S1. The diagnostic planning agent receives the case ID and initializes the diagnostic process; the diagnostic process includes case type labeling and agent call priority setting.

[0172] S2. Perform multi-agent cooperative analysis:

[0173] S21. Call the hospital information aggregation intelligent agent to obtain the patient's hospital information;

[0174] S22. Call the pathology slide analysis agent to perform multi-dimensional analysis of digital slides;

[0175] S23. Call the knowledge base / information search agent to obtain supporting documents and external information;

[0176] S24. Generate an initial diagnosis including a chain of evidence; the chain of evidence is ordered according to clinical decision weights, in the following order: pathological evidence with a weight of 40%, clinical evidence with a weight of 30%, and literature evidence with a weight of 30%.

[0177] S3. The critical agent submits a critical report on the initial diagnosis;

[0178] S4. The arbitration agent receives and, based on the initial diagnosis, the critical report, and the diagnostic uncertainty index generated by the risk assessment agent, judges the validity of the diagnosis according to the judgment logic.

[0179] S5. If the arbitration agent determines that the diagnosis is valid, the final diagnosis is output; if the arbitration agent determines that the diagnosis is invalid, the arbitration judgment result and related information are fed back to step S1, triggering iterative optimization of the diagnosis process; after updating the diagnosis plan based on the arbitration judgment result and related information, step S1 executes steps S2, S3 and S4 again.

[0180] S6. Repeat steps S2 to S5 until the diagnostic termination condition is met;

[0181] In practice, we take a case of recurrent fever with lung shadows of unknown origin (Case ID: P20231001) as an example:

[0182] First, after receiving the case ID, the diagnostic planning agent marks the case type as infectious disease under investigation (medium to high complexity) and sets the priority of core agents: in-hospital information aggregation agent (level 1), pathological slide analysis agent (level 2), and knowledge base / information search agent (level 3).

[0183] Next, perform multi-agent collaborative analysis:

[0184] The hospital's information aggregation intelligent agent obtains the patient's complete blood count (white blood cell count 12×10⁻⁶) within the past 3 months. 9 A comprehensive medical data package including: / L, chest CT (nodular shadow in the lower lobe of the right lung), and history of diabetes (glycated hemoglobin 7.8%).

[0185] The pathological section analysis agent performed digital section analysis on the CT-guided lung puncture specimen and found granulomatous inflammation with a small number of acid-fast bacilli (pathological evidence weight 40%).

[0186] The knowledge base / information search agent retrieved the latest research published in The Lancet Infectious Diseases in 2025; combined with the patient's symptoms such as fever (body temperature 38.5℃), cough and sputum (clinical evidence weight 30%);

[0187] Initial diagnosis generated: Secondary pulmonary tuberculosis (chain of evidence: pathological evidence 40%, clinical evidence 30%, literature evidence 30%);

[0188] Next, the critical agent raised systematic questions: ① The number of acid-fast bacilli in the pathological sections was low (only 2 bacilli / 100 fields), could there be non-tuberculous mycobacteria? ② The patient's recent history of using glucocorticoids (to treat diabetic complications) was not included, could the immunosuppressed state affect the diagnosis? A critical report was then submitted.

[0189] Next, the risk assessment agent generated a diagnostic uncertainty index of 8.2 (out of 10, ≥7 indicates high uncertainty); the arbitration agent evaluated based on the judgment logic: the morphology of the acid-fast bacilli is consistent with the characteristics of Mycobacterium tuberculosis, but the medication history is not included in the clinical evidence chain, and the diagnostic validity score is 65 (<80 points, the judgment is invalid).

[0190] Next, the arbitration result is fed back to step S1, and the diagnostic planning agent updates the diagnostic plan: the medication history is added to the clinical evidence, and the priority of the knowledge base agent is increased to retrieve relevant literature on glucocorticoids and nontuberculous mycobacterial infections; steps S2-S4 are executed again.

[0191] The iteration of step S2 is as follows: supplement the acquisition of glucocorticoid medication records (prednisone 30mg / day for the past month); add a search showing that the risk of nontuberculous mycobacterial infection increases by 3.2 times for long-term hormone users; after updating the chain of evidence, the initial diagnosis is adjusted to possible pulmonary tuberculosis (nontuberculous mycobacterial infection to be ruled out); step S3: the critical agent's questioning is resolved; step S4: the risk assessment index drops to 4.5, and the arbitration score is 89 points (judged as valid);

[0192] Finally, if the diagnostic termination criteria are met (arbitration score ≥ 80 points and uncertainty index < 5 points), the final diagnosis is output: secondary pulmonary tuberculosis (compounded with type 2 diabetes and long-term use of glucocorticoids).

[0193] The working principle of the above technical solution is as follows: The principle of this multi-agent diagnostic planning method based on the consultation thinking process is to simulate the consultation thinking process and to diagnose diseases through the collaboration and interaction of multiple agents.

[0194] First, the diagnostic planning agent receives the case ID, labels the case type, and sets the priority for calling the core agent, laying the foundation for subsequent diagnostic work.

[0195] Next, multi-agent collaborative analysis: the in-hospital information aggregation agent is responsible for collecting comprehensive medical data of patients, including blood routine, chest CT, past medical history, etc., to provide rich basic information for diagnosis; the pathological slide analysis agent performs multi-dimensional analysis of digital slides of relevant specimens to obtain pathological evidence, which has a high weight in the diagnostic evidence chain, reaching 40%; the knowledge base / information search agent retrieves disease characteristics and the latest research results, and combines them with patient symptoms to form clinical evidence and literature evidence, each with a weight of 30%, providing support for diagnosis from different perspectives;

[0196] Next, an initial diagnosis is generated based on the evidence chains with different weights, and the evidence chains are sorted according to the clinical decision weights.

[0197] Next, the critical agent systematically questions the initial diagnosis, raising questions about the completeness of pathological and clinical information and forming a critical report to verify the accuracy and comprehensiveness of the diagnosis.

[0198] Next, the arbitration agent combines the initial diagnosis, the critique report, and the diagnostic uncertainty index generated by the risk assessment agent, and scores the diagnostic validity according to a specific judgment logic. If the score reaches a certain standard, it is judged as valid; otherwise, it is invalid.

[0199] Next, if the arbitration agent determines that the diagnosis is invalid, it feeds back the arbitration result and related information to the diagnosis planning agent. The diagnosis planning agent updates the diagnosis plan accordingly and performs multi-agent collaborative analysis, critical review and effectiveness evaluation again. Through continuous iteration, the chain of evidence and diagnosis results are adjusted to gradually improve the accuracy of the diagnosis.

[0200] Finally, repeat the above process until the diagnostic termination conditions are met (e.g., arbitration score ≥ 80 points and uncertainty index < 5 points), output the final diagnosis, and complete the entire diagnostic process.

[0201] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the thinking mode of multidisciplinary expert collaboration in a real consultation scenario can be simulated, and medical data from different dimensions can be organically integrated with professional analysis. With the help of clear evidence weight allocation and multi-round iterative optimization mechanism, the systematicness and rigor of the diagnostic process can be effectively improved.

[0202] In one embodiment, invoking a pathological slide analysis agent to perform multi-dimensional analysis of digital slides includes:

[0203] After receiving the slice ID, verify the availability of the digital slice. The verification criteria for digital slice availability are: the digital slice is in DICOM format, the horizontal pixel count is not less than 2000, and there is no data corruption.

[0204] The following steps are performed sequentially on the available slices: full slice classification using a multi-instance learning (MIL) model based on a multi-stage attention mechanism; tissue segmentation using a U-Net++ deep learning model; object detection using a YOLOv8 model; and region description using a PathVLM visual language model. This generates a multi-dimensional analysis report that includes disease probability distribution, coordinates of suspicious regions, tissue composition ratio, and morphological description.

[0205] The working principle of the above technical solution is as follows: First, when the pathological slide analysis agent is called to perform multi-dimensional analysis of digital slides, the system starts to work; the first step is to receive the slide ID, which is an identifier to identify a specific digital slide. The corresponding digital slide can be accurately found through the slide ID.

[0206] Next, the usability of digital slices is verified. The verification criteria include three aspects: First, the digital slices must be in DICOM format, which is a standard format widely used in the field of medical digital imaging and communication, facilitating the exchange and storage of medical images between different devices and systems; second, the horizontal pixel count must be no less than 2000, as sufficient pixels ensure clear image details and provide enough information for subsequent analysis; and third, no data corruption must be ensured, as data corruption will lead to inaccurate analysis results. Only digital slices that meet these three criteria are considered usable.

[0207] For available slices, a series of different analysis tasks will be performed sequentially, specifically:

[0208] The multi-instance learning (MIL) model based on a multi-stage attention mechanism is used for whole slice classification. The multi-stage attention mechanism allows the model to focus on different important regions of the slice at different stages. The MIL model takes multiple instances in the slice as input for overall classification, which can determine the possible disease categories and related probabilities of the whole slice and generate disease probability distribution information.

[0209] Tissue segmentation was performed using a U-Net++ deep learning model. U-Net++ is a model that performs well in the field of medical image segmentation. Its special network structure is conducive to accurately segmenting different tissue parts in the slice. By analyzing and calculating the proportion of each tissue in the slice, the tissue composition ratio information is obtained.

[0210] Object detection is performed using the YOLOv8 model. YOLOv8 is a fast and accurate object detection model that can quickly locate suspicious areas in pathological slides and provide their coordinate information.

[0211] Finally, the PathVLM visual language model is used for region description. This model combines visual information with language expression to provide morphological descriptions of previously detected suspicious regions, such as their shape, size, and edge features. These analysis results are integrated to generate a multidimensional analysis report that includes disease probability distribution, suspicious region coordinates, tissue composition ratio, and morphological description, providing doctors with comprehensive and valuable reference for diagnosing diseases.

[0212] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the pathological slides can be systematically analyzed through multi-agent collaboration, which can give full play to the technical advantages of different models. This not only improves the comprehensiveness and accuracy of the analysis, but also provides doctors with multi-dimensional and quantifiable diagnostic reference information, which helps doctors make diagnostic decisions more quickly and accurately, thereby improving the efficiency and quality of pathological diagnosis.

[0213] In one embodiment, the diagnostic termination criteria include a combination of at least three of the following:

[0214] (1) The completeness of the evidence chain for the arbitration intelligent agent is ≥90% and the exclusion rate of key identification diagnosis is 100%; the calculation method for completeness is: number of valid evidence / total number of required evidence × 100%;

[0215] (2) The fluctuation range of the diagnostic uncertainty index output by the risk assessment agent is less than 5% for three consecutive iterations; the fluctuation range is calculated as: |current index - previous index| / previous index × 100%;

[0216] (3) The latest evidence returned by the knowledge base search agent matches the current diagnostic conclusion with a degree of greater than 95%, and the number of high-impact factor literatures in the past three years is greater than or equal to 3; the degree of matching is calculated based on cosine similarity.

[0217] (4) The lesion stability score output by the dynamic monitoring tool of the pathological section analysis agent is >85 points and there are no new high-risk characteristics;

[0218] (5) The coefficient of variation of the key clinical indicators obtained by the in-hospital information aggregation intelligent agent for two consecutive tests is <15%; key clinical indicators include tumor markers and inflammatory markers; the formula for calculating the coefficient of variation is: standard deviation / mean × 100%;

[0219] The maximum number of iterations is set to 3 to 5. If the above combination of judgments is not satisfied even after reaching the maximum number of iterations, the latest thinking result generated in the current iteration is output.

[0220] The working principle of the above technical solution is as follows: There are 5 conditions in the diagnostic termination conditions:

[0221] First, the completeness of the diagnostic evidence chain is measured by calculating the ratio of the number of valid evidence to the total number of required evidence. When the completeness is ≥90% and the exclusion rate of key differential diagnoses reaches 100%, it indicates that the diagnostic evidence is relatively comprehensive and the key diagnoses have accurately excluded other possibilities, which helps to judge the reliability of the diagnosis. This judgment condition ensures the sufficiency of the evidence and the accuracy of the diagnostic direction.

[0222] Second, the stability of the diagnosis is judged by calculating the fluctuation range of the diagnostic uncertainty index for three consecutive iterations. The fluctuation range is based on the ratio of the difference between two adjacent indices to the previous index. If the fluctuation range for three consecutive iterations is less than 5%, it indicates that the diagnostic uncertainty fluctuates within a small range, the diagnostic results tend to be stable, and the uncertainty risk caused by large fluctuations in the diagnostic results is reduced.

[0223] Third, the matching degree between the latest evidence and the current diagnostic conclusion is calculated based on cosine similarity. When the matching degree is >95% and there are ≥3 high-impact factor literatures in the past three years, it indicates that the current diagnostic conclusion is supported from the perspective of knowledge base and high-impact literature. This makes the diagnostic conclusion not only highly consistent with the internal evidence matching, but also corroborated by external authoritative literature, thus enhancing the credibility of the diagnosis.

[0224] Fourth, the dynamic monitoring tool outputs a lesion stability score. When the score is >85 and there are no new high-risk features, it reflects that the lesion status is relatively stable from a pathological perspective, suggesting that the diagnosis result may tend to be stable. The lesion stability score and high-risk features provide key pathological information for diagnosis, which helps to determine whether the condition is stable and whether the diagnosis can be terminated.

[0225] Fifth, for key clinical indicators (such as tumor markers and inflammatory markers), the change of the indicators is measured by calculating the coefficient of variation (the ratio of the standard deviation to the mean) of two consecutive test values. When the coefficient of variation is <15%, it indicates that the key clinical indicators are relatively stable, reflecting the stability of the patient's condition at the level of clinical indicators, and providing clinical basis for terminating the diagnosis.

[0226] In addition, the maximum number of loops is set to 3 to 5 to prevent the diagnostic process from looping indefinitely. If the above combination of judgments is not satisfied even after reaching the maximum number of loops, the latest thinking result generated in the current loop is output to avoid the diagnostic process from continuing indefinitely due to the inability to meet the diagnostic termination conditions, and to ensure that the latest diagnostic thinking results are provided within certain time and resource constraints.

[0227] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the scientific quantification and dynamic judgment of the diagnostic termination conditions can be achieved through the collaborative verification mechanism of multi-dimensional intelligent agents; the setting of the maximum number of iterations further balances the diagnostic accuracy and efficiency, avoiding the predicament of infinite iteration; the multi-condition combination judgment mode integrates objective data indicators and subjective experience rules, and improves the decision reliability and clinical applicability of the multi-agent diagnostic system through clear calculation methods and threshold settings.

[0228] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-agent diagnostic planning device based on a consultation thinking process, characterized in that, include: Diagnostic planning agent, pathology slide analysis agent, in-hospital information aggregation agent, information search agent, knowledge base search agent, critical agent, arbitration agent, and risk assessment agent; Each agent interacts with data through a dynamic priority message bus; the dynamic priority message bus is configured to adjust the agent calling priority in real time based on the urgency of the case and the completeness of the information. The diagnostic planning agent is configured as follows: driven by a deep learning model pre-trained on a pathological image dataset based on a visual Transformer architecture, it receives the case ID, formulates a diagnostic plan through the diagnostic stage-agent mapping table, and calls the analysis information of the pathological slide analysis agent, the hospital information aggregation agent, the information search agent, and the knowledge base search agent, and generates an initial diagnosis by combining the case information of the case ID; The pathological slide analysis agent is configured to: receive the slide ID, call preset tools to analyze the pathological slide, and output structured results; The intelligent agent for aggregating hospital information is configured to: receive patient IDs and obtain patient information within the hospital through database queries and system interface calls; The information search agent is configured to receive search commands and access certified medical professional databases and academic search engines to obtain external information. The knowledge base search agent is configured to receive search instructions and generate supporting documents by semantically retrieving the built-in knowledge base. The critical agent is configured to receive initial diagnostic and analytical information and generate a critical report; the analytical information includes structured results, patient in-hospital information, external information, and supporting literature. The arbitration agent is configured to receive initial diagnosis, analysis information and critique report, as well as the diagnostic uncertainty index generated by the risk assessment agent. Based on the arbitration rules based on pathological diagnostic criteria, evidence completeness and literature support, it judges the validity of the diagnosis, outputs the diagnosis result that is judged to be valid, and triggers the diagnosis correction of the diagnosis result that is judged to be invalid when there are more than 3 objections in the critique report. The risk assessment agent is configured to receive initial diagnostic and analysis information, utilize a machine learning risk prediction model trained based on the patient's time-series clinical data to assess the risk of disease progression and treatment, simultaneously generate a diagnostic uncertainty index, and feed back the risk assessment results in real time to the diagnostic planning agent and arbitration agent in JSON format via a standardized API interface to assist in further diagnosis and decision-making.

2. The multi-agent diagnostic planning device based on consultation thinking process according to claim 1, characterized in that, The diagnostic planning agent includes a persistent memory module configured to record information throughout the entire case analysis process, and selectively performs at least one of the following actions based on the case type and the current information completeness: The system calls upon the hospital's internal information aggregation intelligence until it determines that diagnostic information is sufficient. The criteria for determining sufficient information are: information covering medical history, imaging reports, and pathological slide analysis results, and a missing information rate of <5%. The missing information rate is calculated as: number of missing key fields / total number of preset required fields × 100%. A dynamic priority invocation order is adopted to invoke the pathology slide analysis intelligent agent to perform slide analysis. The dynamic priority invocation order is as follows: tumor cases are invoked first, and non-tumor cases first invoke the hospital information aggregation intelligent agent, and then invoke the pathology slide analysis intelligent agent. Invoke information search agents and knowledge base search agents to obtain external information and supporting documents; The initial diagnosis is generated by comprehensively calling the analysis information.

3. The multi-agent diagnostic planning device based on consultation thinking process according to claim 1, characterized in that, In the configuration of the pathological slide analysis agent, the preset tools include a database query tool, a whole slide classification and recognition tool, a tissue segmentation tool, a target detection tool, a region description tool, and a dynamic monitoring tool. The logic for calling the preset tools is as follows: the tool is called through the StdIO transmission mechanism. The trigger condition is that the pathological slide analysis agent will automatically retry if there is no data feedback within 10 seconds after receiving the slide ID. The transmission format of the tool call result is JSON. The database query tool is configured to: locate and read slide data; trigger a scan notification when the slide data is not yet digitized; the slide data format conforms to the DICOM standard; the scan notification is sent via text pop-up and voice prompt; and the triggering delay is less than 5 minutes. The whole-slice classification and recognition tool is configured as follows: a multi-instance learning (MIL) model based on a multi-stage attention mechanism outputs disease probability and coordinates of suspicious areas; the coordinates of suspicious areas are physical coordinates in mm. The tissue segmentation tool is configured to: be based on the U-Net++ deep learning model, and output the tissue type and distribution parameters within the segmented region; the distribution parameters include area proportion and morphological irregularity. The target detection tool is configured to: identify and count specific cell types within the detection area based on the YOLOv8 model; The region description tool is configured to generate textual descriptions of slice regions based on the PathVLM visual language model; the textual descriptions of slice regions include tissue morphology, cell characteristics, and diagnostic tendencies. The dynamic monitoring tool is configured as follows: based on time series analysis and LSTM machine learning algorithm, it dynamically monitors and analyzes the changes of pathological sections at different time points and outputs a lesion stability score; the changes include changes in cell proliferation rate and lesion area; the lesion stability score is derived by weighting the cell morphology change rate and the lesion area variation coefficient, with the cell morphology change rate and the lesion area variation coefficient accounting for 60% and 40% of the weights, respectively; the lesion stability score ranges from 0 to 100.

4. The multi-agent diagnostic planning device based on consultation thinking process according to claim 1, characterized in that, The institute's information aggregation intelligent body includes database query tools, system interface call tools, and information structuring modules; The database query tool is configured to automatically generate SQL statements to query patient data; the SQL statements comply with medical data privacy standards and undergo field anonymization. The system interface call tool is configured to: obtain standardized medical information through the PACS / HIS / LIS system interface; after receiving the request, the system interface shall complete the data query, processing and return the result within 15 seconds; The information structuring module is configured to integrate multi-source data into a unified format diagnostic support document; the fields of the diagnostic support document include anonymized patient ID, basic information, medical history, imaging report, and laboratory results.

5. The multi-agent diagnostic planning device based on consultation thinking process according to claim 1, characterized in that, Upon receiving a search command, it generates supporting documents using the built-in knowledge base for semantic retrieval, including: The search instructions are transformed into feature vectors using a PubMedBERT medical terminology embedding model fine-tuned based on the UMLS terminology database. Semantic retrieval is performed in the built-in knowledge base by calculating the cosine similarity between feature vectors; the cosine similarity threshold is ≥0.

85. Extract the built-in knowledge base texts corresponding to the Top-N similarity feature vectors as supporting evidence; the built-in knowledge base texts include high-impact factor journal articles from the past three years; N≥5 in Top-N; high impact factor≥5; The system uses a fine-tuned large model based on GPT-4 in the medical field to generate structured answers based on supporting evidence, and then generates supporting literature based on the structured answers. The structured answers include a summary of supporting evidence and an analysis of their relevance to the search instructions.

6. The multi-agent diagnostic planning device based on consultation thinking process according to claim 1, characterized in that, The critical agent is configured to perform the following operations: Based on the diagnostic criteria, the sufficiency of the initial diagnosis in supporting the diagnostic conclusion was verified to obtain the verification conclusion. The diagnostic criteria were: among the supporting evidence for the initial diagnosis, the confidence level of the pathological slide analysis results was <70%, or key differential disease types were omitted. Based on the verification conclusions and analysis information, differential diagnosis suggestions are proposed; Generate a critical report that includes challenges to the chain of evidence and recommendations for further investigation.

7. The multi-agent diagnostic planning device based on consultation thinking process according to claim 1, characterized in that, The arbitration agent is configured to execute the following judgment logic using a finely tuned LLaMA-2 model based on pathology expertise to determine the validity of a diagnosis: The completeness of the examination items required for diagnosis should be checked; the standard for completeness is: the missing rate of core examination items <10%; Verify the exclusion criteria for differential diagnosis; the exclusion criteria are: excluding diseases with a high probability of ≥90% in the differential diagnosis. Assess the degree of support from literature evidence for the diagnostic conclusion; the minimum standard for support is: ≥2 articles from high-impact factor literature in the past three years. Verify whether the number of logical chain nodes in the diagnostic reasoning logic chain is greater than the preset number, which is 3 to 5. Assess whether the diagnostic uncertainty index is below a preset threshold, which is 30.

8. A multi-agent diagnostic planning method based on a consultation thinking process, characterized in that, include: S1. The diagnostic planning agent receives the case ID and initializes the diagnostic process; The diagnostic process includes case type labeling and agent call priority settings; S2. Perform multi-agent cooperative analysis: S21. Call the hospital information aggregation intelligent agent to obtain the patient's hospital information; S22. Call the pathology slide analysis agent to perform multi-dimensional analysis of digital slides; S23. Call the knowledge base / information search agent to obtain supporting documents and external information; S24. Generate an initial diagnosis including a chain of evidence; the chain of evidence is ordered according to clinical decision weights, in the following order: pathological evidence with a weight of 40%, clinical evidence with a weight of 30%, and literature evidence with a weight of 30%. S3. The critical agent submits a critical report on the initial diagnosis; S4. The arbitration agent receives and, based on the initial diagnosis, the critical report, and the diagnostic uncertainty index generated by the risk assessment agent, judges the validity of the diagnosis according to the judgment logic described in claim 7. S5. If the arbitration agent determines that the diagnosis is valid, then output the final diagnosis; If the arbitration agent determines that the diagnosis is invalid, it feeds back the arbitration judgment result and related information to step S1, triggering iterative optimization of the diagnosis process; after updating the diagnosis plan based on the arbitration judgment result and related information, step S1 executes steps S2, S3 and S4 again. S6. Repeat steps S2 to S5 until the diagnostic termination condition is met.

9. The multi-agent diagnostic planning method based on consultation thinking process according to claim 8, characterized in that, The pathology slide analysis agent is invoked to perform multi-dimensional analysis of digital slides, including: After receiving the slice ID, verify the availability of the digital slice. The verification criteria for digital slice availability are: the digital slice is in DICOM format, the horizontal pixel count is not less than 2000, and there is no data corruption. The following steps are performed sequentially on the available slices: full slice classification using a multi-instance learning (MIL) model based on a multi-stage attention mechanism; tissue segmentation using a U-Net++ deep learning model; object detection using a YOLOv8 model; and region description using a PathVLM visual language model. This generates a multi-dimensional analysis report that includes disease probability distribution, coordinates of suspicious regions, tissue composition ratio, and morphological description.

10. The multi-agent diagnostic planning method based on consultation thinking process according to claim 8, characterized in that, The diagnostic termination criteria include a combination of at least three of the following: (1) The completeness of the evidence chain for the arbitration intelligent agent is ≥90% and the exclusion rate of key identification diagnosis reaches 100%; the calculation method for completeness is: number of valid evidence / total number of required evidence × 100%; (2) The fluctuation range of the diagnostic uncertainty index output by the risk assessment agent is less than 5% for three consecutive iterations; the fluctuation range is calculated as: |current index - previous index| / previous index × 100%; (3) The latest evidence returned by the knowledge base search agent matches the current diagnostic conclusion with a degree of greater than 95%, and the number of high-impact factor literatures in the past three years is greater than or equal to 3; the degree of matching is calculated based on cosine similarity. (4) The lesion stability score output by the dynamic monitoring tool of the pathological slide analysis intelligent agent is >85 points and there are no new high-risk characteristics; (5) The coefficient of variation of two consecutive test values ​​of key clinical indicators obtained by the in-hospital information aggregation intelligent agent is <15%; key clinical indicators include tumor markers and inflammatory markers; the formula for calculating the coefficient of variation is: standard deviation / mean × 100%; The maximum number of iterations is set to 3 to 5. If the above combination of judgments is not satisfied even after reaching the maximum number of iterations, the latest thinking result generated in the current iteration is output.

Images