Heart failure treatment aid decision generation system based on multi-modal data fusion

Abstract

The invention discloses a heart failure treatment aid decision generation system based on multi-modal data fusion, and the system comprises a data collection module which is used for collecting the multi-modal data of a patient, and the multi-modal data comprises structured data, unstructured data and medical image data; the data processing module is used for carrying out standardization, quantization and vectorization processing on the multi-modal data; the knowledge graph construction module is used for constructing a knowledge graph of heart failure treatment, and the knowledge graph comprises a disease entity, a pathological feature, a treatment scheme and an association relationship thereof; the reasoning module is used for generating a personalized treatment decision based on the knowledge graph and the patient data; and the treatment scheme generation module is used for dynamically adjusting and outputting a personalized treatment scheme in combination with the real-time state data of the patient. The problems that multi-modal data are difficult to fuse and real-time disease change is difficult to dynamically adjust in heart failure diagnosis and treatment are solved, accurate diagnosis and personalized treatment are realized through knowledge graph reasoning and a dynamic correction mechanism, and the diagnosis and treatment efficiency and accuracy are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer, and particularly relates to a heart failure treatment auxiliary decision generation system based on multi-modal data fusion. BACKGROUND

[0002] Heart failure (HF) is a highly complex chronic disease characterized by multiple etiologies and pathological mechanisms, which poses great challenges to the diagnosis and treatment of patients. With the increasing prevalence of heart failure worldwide, heart failure has become one of the main reasons for significantly increasing hospitalization and mortality. Traditional heart failure diagnosis and treatment methods usually rely on the experience of doctors and single modal data sources, such as laboratory indicators or imaging results. However, the data of heart failure patients is highly heterogeneous, including laboratory indicators, imaging data, hemodynamic parameters, and unstructured descriptions in electronic medical records, and the complex interaction between these data is often difficult to fully analyze and integrate through traditional methods.

[0003] In recent years, multi-modal data fusion technology based on artificial intelligence has provided a new solution for the diagnosis and treatment of complex diseases. However, existing technologies still face many challenges in the context of heart failure diagnosis and treatment: first, the heterogeneity and dynamics of multi-modal data are difficult to effectively integrate, and the correlation analysis between different data sources is limited; second, the condition of heart failure patients changes rapidly, and traditional static rule-based diagnosis and treatment assistance systems often lack dynamic adjustment capabilities, making it difficult to adapt to the real-time condition of patients; third, existing knowledge graph technology has not been fully applied in the field of heart failure, and has not been able to deeply combine rich clinical knowledge with individualized patient data, resulting in insufficient precision of treatment recommendations.

[0004] Based on the above problems, the present application proposes a heart failure treatment auxiliary decision generation system based on multi-modal data fusion. The system provides accurate, scientific, and personalized treatment recommendations through comprehensive collection and fusion of multi-modal data, dynamic reasoning based on knowledge graph, and dynamic correction mechanism combined with real-time patient condition, aiming to significantly improve the efficiency and precision of heart failure diagnosis and treatment, and provide high-quality decision support for doctors. SUMMARY

[0005] In view of the defects in the prior art, the present application provides a heart failure treatment auxiliary decision generation system based on multi-modal data fusion, characterized in that the system comprises:

[0006] a data acquisition module for acquiring multi-modal data of a patient, including:

[0007] a) Structured data: laboratory indicators (NT-proBNP, BNP, electrolytes, eGFR, etc.) and hemodynamic parameters (central venous pressure, pulmonary artery systolic pressure, etc.);

[0008] b) Unstructured data: clinical features such as lung rales distribution, lower extremity edema, etc. extracted from electronic medical records through natural language processing techniques;

[0009] c) Medical image data: ultrasound B-line count, LVEF, and valve regurgitation parameters extracted from medical imaging systems;

[0010] a data processing module for standardizing, quantifying, and vectorizing the multi-modal data;

[0011] a knowledge graph construction module for constructing a knowledge graph related to heart failure treatment, including disease entities, pathological features, treatment plans, and their associated relationships;

[0012] a reasoning module for generating personalized treatment decisions based on the knowledge graph and patient data;

[0013] a treatment plan generation module for dynamically adjusting and outputting personalized treatment plans in combination with real-time patient state data.

[0014] Optionally, the data acquisition module further includes:

[0015] a laboratory data acquisition unit for acquiring patient biomarkers (such as NT-proBNP, BNP), kidney function indicators (eGFR), and electrolyte levels;

[0016] an unstructured data acquisition unit for extracting clinical description data from electronic medical record texts, including lung rales distribution, lower extremity edema, etc.;

[0017] a medical image acquisition unit for extracting feature data from echocardiograms and chest images from medical imaging systems and converting them into structured data.

[0018] Optionally, the data processing module includes:

[0019] an unstructured data analysis unit for converting descriptive text data in electronic medical records into structured data through natural language processing techniques;

[0020] an image data quantification unit for quantifying ultrasound B-line count, left ventricular ejection fraction (LVEF), and other image features;

[0021] a data standardization unit for uniformly converting data from different modalities into embedded vector representations to support subsequent reasoning processing.

[0022] Optionally, the knowledge graph construction module includes:

[0023] a node construction unit for constructing heart failure-related knowledge nodes, including disease entities (ischemic cardiomyopathy, diabetic nephropathy, etc.), pathological mechanisms (such as circulatory congestion), drugs (such as diuretics, ARNI), and treatment plans;

[0024] a relationship definition unit for defining relationships between nodes, including:

[0025] a) the relationship between causes and pathological mechanisms;

[0026] b) the relationship between comorbidities and drug contraindications;

[0027] c) the relationship between drugs and metabolic pathways;

[0028] a dynamic updating unit for extracting the latest clinical guidelines in real time through Retrieval Augmentation Generation (RAG) and updating the knowledge graph.

[0029] Optionally, the reasoning module includes:

[0030] a reasoning engine for generating cause analysis and treatment decisions based on the knowledge graph and patient data;

[0031] a drug dosage adjustment unit for dynamically adjusting drug dosages according to patient individualization parameters, including:

[0032] a) adjusting diuretic dosage according to changes in renal function (such as decreased eGFR);

[0033] b) triggering potassium supplementation or adjusting drug combinations according to electrolyte levels (such as serum potassium);

[0034] a multi-path reasoning unit for reasoning and optimizing different treatment paths to generate the optimal treatment plan.

[0035] Optionally, the treatment plan generation module includes:

[0036] a volume status assessment unit for generating assessment results of circulatory congestion and tissue edema based on patient biomarkers (such as BNP), hemodynamic parameters (such as central venous pressure), and imaging data;

[0037] a drug response prediction unit for predicting patient responses to different drugs based on knowledge graph rules;

[0038] a dynamic adjustment unit for automatically optimizing treatment plans in combination with real-time monitoring data (such as urine output, body weight, electrolyte levels, etc.) of patients.

[0039] Optionally, the system further includes:

[0040] a real-time monitoring module for collecting dynamic data of patients, including urine output, electrolyte levels, body weight changes, etc.

[0041] Early warning mechanism for triggering early warning and suggesting adjustment of treatment plan when key indicators are abnormal;

[0042] Feedback optimization module for dynamically optimizing treatment path according to real-time monitoring data.

[0043] Optionally, the system adopts cross-modal data fusion technology, including:

[0044] Hierarchical processing architecture for multi-level processing of structured data, unstructured data and image data;

[0045] Tensor packaging technology for embedding multi-modal data into a unified tensor representation;

[0046] Dynamic weighting mechanism for dynamically adjusting the weight of different modal data according to the patient's condition to optimize the inference result.

[0047] Optionally, the system further includes a retrieval enhancement generation module, including:

[0048] Similar case retrieval unit for retrieving cases similar to the current patient's condition from the historical case database;

[0049] Case reference module for combining similar case data with knowledge graph inference results to optimize treatment plans.

[0050] Optionally, the system further includes:

[0051] Model training module for fine-tuning the heart failure large model through transfer learning and LoRA technology;

[0052] Time series processing module for optimizing the prediction ability of the dynamic treatment adjustment based on the time-tagged patient data optimization model;

[0053] Verification module for evaluating the consistency and reliability of system decision-making through perfect medical records and common clinical error medical records.

[0054] The heart failure treatment auxiliary decision-making generation system based on multi-modal data fusion of the present application, through the close cooperation of each module, constructs a complete process from data acquisition, processing, inference to treatment plan generation. The system makes full use of the rich information of multi-modal data, combines professional knowledge graph and advanced algorithms, realizes accurate diagnosis and individualized treatment of heart failure patients, and has important clinical application value and broad popularization prospect. BRIEF DESCRIPTION OF DRAWINGS

[0055] The above and other objects, features and advantages of the present disclosure exemplary embodiments will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example, and like or corresponding reference numerals are used to refer to like or corresponding elements throughout. In the drawings:

[0056] Figure 1 is a diagram illustrating a heart failure treatment auxiliary decision generation system based on multi-modal data fusion according to an embodiment of the present application;

[0057] Figure 2 is a diagram illustrating a heart failure diagnosis and treatment process framework based on agent cooperation according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0059] The terms used in the embodiments of the present application are only for the purpose of describing particular embodiments and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two.

[0060] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe …, these … should not be limited to these terms. These terms are only used to distinguish … from one another. For example, without departing from the scope of the embodiments of the present application, the first … can also be referred to as the second …, and similarly, the second … can also be referred to as the first ….

[0061] It should be understood that the term "and / or" used herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0062] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "when it is determined" or "in response to determining" or "when [a stated condition or event] is detected" or "in response to detecting [a stated condition or event]."

[0063] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a product or process that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such product or process. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the product or process that includes the stated element.

[0064] As shown in FIG. 1, Figure 1 The present application provides a heart failure treatment aided decision-making generation system based on multi-modal data fusion. By collecting, processing and fusing the multi-modal clinical data of patients, and combining the constructed heart failure treatment knowledge graph, the system realizes the accurate diagnosis of heart failure patients and the dynamic generation of personalized treatment plans. The system is composed of multiple modules, including data acquisition module, data processing module, knowledge graph construction module, reasoning module, treatment plan generation module and auxiliary monitoring and optimization module. Each module closely cooperates to form a complete heart failure treatment aided decision-making process.

[0065] Firstly, the data acquisition module is responsible for acquiring multi-modal data from the clinical data of patients, including structured data, unstructured data and medical image data, etc. Among them, the structured data mainly includes laboratory indicators such as NT-proBNP, the ratio of BNP and NT-proBNP, serum creatinine, liver function indicators (ALT / AST), serum albumin, electrolyte levels, etc. These indicators can reflect the heart load status, kidney function status, liver function status and electrolyte balance of patients, providing important basis for evaluating heart failure degree and developing treatment plan. The unstructured data is mainly derived from electronic medical record text, and the clinical description data of patients such as the distribution of lung rales and the degree of lower extremity edema are extracted through natural language processing technology. These clinical features are of great significance to evaluate the degree of pulmonary circulation congestion and tissue edema of patients. Medical image data is directly acquired through the interface with PACS system, and ultrasonic cardiogram and chest image data of patients are obtained. By using advanced image processing algorithm, image features such as ultrasonic B-line count and left ventricular ejection fraction (LVEF) are extracted, and they are converted into structured data to provide accurate quantitative indicators for evaluating heart function and volume status.

[0066] Secondly, the data processing module standardizes, quantifies, and vectorizes the collected multi-modal data. The unstructured data parsing unit converts descriptive text data in electronic medical records into structured data through natural language processing technology. For example, descriptive words such as "mild concave edema" or "severe concave edema in both lower extremities" are converted into standardized edema grading, providing quantifiable data support for volume status assessment. The image data quantization unit quantizes ultrasound B-line count, left ventricular ejection fraction (LVEF), and other image features, extracts key parameters, and converts them into numerical form using deep learning algorithms and image segmentation techniques. The data standardization unit is responsible for converting data from different modalities into embedded vector representations, embedding all data into a unified data space through unit conversion, normalization processing, and tensor packaging techniques, supporting subsequent reasoning and decision-making processes.

[0067] Next, the knowledge graph construction module establishes a professional knowledge graph for heart failure treatment, including disease entities, pathological mechanisms, drugs, treatment plans, and their associated relationships. The node construction unit constructs knowledge nodes related to heart failure, covering various heart failure causes (such as ischemic cardiomyopathy, valvular heart disease), pathological mechanisms (such as myocardial ischemia, myocardial remodeling), clinical features (such as elevated BNP, hyponatremia), and detailed information on treatment plans (such as diuretics, ARNI drugs). The relationship definition unit defines the relationships between nodes, including the relationships between causes and pathological mechanisms, comorbidities and drug contraindications, and drug and metabolic pathways. For example, it defines the causal relationship between "coronary heart disease" and "myocardial ischemia" or the association between "renal insufficiency" and "diuretic dose adjustment." The dynamic updating unit uses retrieval augmentation generation technology to extract the latest clinical guideline content in real time and update it to the knowledge graph, ensuring the timeliness and accuracy of the knowledge base.

[0068] In the reasoning module, the reasoning engine generates reasoning for cause analysis and treatment decision-making based on the constructed knowledge graph and the patient's individualized data. The drug dose adjustment unit dynamically adjusts drug doses based on the patient's individualized parameters, such as renal function, electrolyte levels, blood pressure, etc. For example, it adjusts the dose of diuretics based on changes in eGFR and decides whether to supplement potassium or adjust drug combinations based on blood potassium levels. The multi-path reasoning unit reasons and optimizes different treatment paths, combining the rules in the knowledge graph and the patient's real-time state to generate the optimal treatment plan. This process fully considers the patient's complex clinical situation, ensuring the safety and effectiveness of the treatment recommendations.

[0069] The treatment plan generation module outputs personalized treatment plans based on the results of the inference module. The volume status assessment unit integrates the patient's biomarkers, hemodynamic parameters, and imaging data to assess the degree of circulatory congestion and tissue edema, providing a scientific basis for volume management. The drug response prediction unit predicts the patient's response to different drugs based on the rules of the knowledge graph, helping doctors choose the most appropriate drugs and dosages. The dynamic adjustment unit combines real-time monitoring data such as urine output, electrolyte levels, and weight changes to automatically optimize treatment plans, ensuring the accuracy and timeliness of treatment.

[0070] In addition, the system also includes real-time monitoring modules and feedback optimization modules. The real-time monitoring module continuously collects dynamic data of patients and monitors key indicators. When an abnormality is detected, the warning mechanism will trigger in time to prompt the doctor and suggest adjusting the treatment plan. The feedback optimization module dynamically optimizes the treatment path and the rule weight of the knowledge graph according to the patient's treatment feedback and monitoring data, improving the intelligent level and adaptability of the system.

[0071] In order to realize the efficient fusion of cross-modal data, the system adopts a hierarchical processing architecture, tensor packaging technology and dynamic weighting mechanism. The hierarchical processing architecture processes structured data, unstructured data and image data in multiple layers, extracts features and performs preliminary analysis. The tensor packaging technology embeds multi-modal data into a unified tensor representation, eliminating the problem of data heterogeneity. The dynamic weighting mechanism dynamically adjusts the weight of different modal data according to the patient's condition and real-time state, optimizing the accuracy of the inference result.

[0072] The system also integrates a retrieval enhancement generation module, which retrieves similar cases from the historical case database through the similar case retrieval unit. The case reference module combines the data of these similar cases with the inference results of the knowledge graph to further optimize the treatment plan. This function takes advantage of big data to provide valuable references for the treatment of complex cases.

[0073] In terms of model training, the model training module of the system fine-tunes the pre-trained heart failure large model through transfer learning and LoRA technology, improving the model's performance on specific tasks. The time series processing module optimizes the model's prediction ability for dynamic treatment adjustment based on time-tagged patient data, better capturing the trend of patient condition changes. The verification module uses perfect medical records and common clinical error cases to evaluate the consistency and reliability of the system's decisions, ensuring the stability and accuracy of the system in actual application.

[0074] The heart failure treatment auxiliary decision generation system based on multi-modal data fusion of the application constructs a complete process from data acquisition, processing, reasoning to treatment scheme generation through the close cooperation of each module. The system makes full use of the rich information of multi-modal data, combines professional knowledge graph and advanced algorithms, realizes the accurate diagnosis and individualized treatment of heart failure patients, and has important clinical application value and broad popularization prospect.

[0075] The application provides a heart failure treatment auxiliary decision generation system based on multi-modal data fusion. By collecting, processing and fusing the multi-modal clinical data of patients, and combining the constructed heart failure treatment knowledge graph, the accurate diagnosis and dynamic generation of individualized treatment scheme for heart failure patients are realized. The system is composed of multiple modules, including a data acquisition module, a data processing module, a knowledge graph construction module, a reasoning module, a treatment scheme generation module, and an auxiliary monitoring and optimization module. Each module closely cooperates to form a complete heart failure treatment auxiliary decision process.

[0076] Firstly, the data acquisition module is responsible for acquiring multi-modal data from the clinical data of patients, including structured data, unstructured data and medical image data, etc. Among them, the structured data mainly includes laboratory indexes such as NT-proBNP, the ratio of BNP to NT-proBNP, serum creatinine, liver function indexes (ALT / AST), serum albumin, electrolyte level, etc. These indexes can reflect the heart load state, kidney function state, liver function state and electrolyte balance of the patient, and provide important basis for evaluating the degree of heart failure and formulating treatment scheme. The unstructured data is mainly derived from the electronic medical record text, and the clinical description data of the patient is extracted through natural language processing technology, such as the distribution of lung rales and the degree of lower extremity edema. These clinical features are of great significance to evaluate the degree of pulmonary circulation congestion and tissue edema of the patient. The medical image data is directly acquired through the interface with the PACS system, and the echocardiogram and chest image data of the patient are obtained. By using advanced image processing algorithms, image features such as ultrasonic B-line count and left ventricular ejection fraction (LVEF) are extracted, and they are converted into structured data to provide accurate quantitative indexes for evaluating the heart function and volume state.

[0077] The data processing module standardizes, quantizes and vectorizes the collected multi-modal data. Among them, the hierarchical fusion formula (CFLF) and the modal interaction fusion formula (MIF) are realized in this module for multi-modal data fusion calculation. The hierarchical fusion formula is used to combine the patient's etiology type, image features and biomarkers to calculate the comprehensive results of volume load in layers. Specifically:

[0078] ;

[0079] Among them, represents the first layer etiology weight, is the first layer modal feature value, is the first layer dynamic weight. The dynamic weight is calculated by the following formula:

[0080] ;

[0081] wherein, is the basic weight, is the real-time correlation parameter. The data processing module can dynamically adjust the weight of each modal data according to the specific etiology and condition of the patient, to ensure the accuracy of the fusion result. The modal interaction fusion formula is used to quantify the synergistic effect between different modal data:

[0082] ;

[0083] wherein, the modal interaction effect coefficient is calculated by the following formula:

[0084] ;

[0085] and are modal feature vectors, is a normalization parameter, is the semantic correlation between the modalities. Through these calculations, the data processing module fully excavates the complementary information of multi-modal data, improving the effect of data fusion.

[0086] The unstructured data analysis unit converts descriptive text data in electronic medical records into structured data through natural language processing technology. For example, descriptive words such as "mild concave edema" or "severe concave edema in both lower extremities" are converted into standardized edema grading, providing quantifiable data support for volume status assessment. The image data quantification unit quantifies ultrasound B-line count, left ventricular ejection fraction LVEF and other image features, extracts key parameters and converts them into numerical form using deep learning algorithms and image segmentation techniques. The data standardization unit is responsible for converting data from different modalities into embedded vector representation, through unit conversion, normalization processing and tensor packaging technology, all data are embedded into a unified data space, supporting subsequent reasoning and decision-making processing.

[0087] The knowledge graph construction module establishes a professional knowledge graph for heart failure treatment, including disease entities, pathological mechanisms, drugs, treatment plans, and their associated relationships. The node construction unit constructs knowledge nodes related to heart failure, covering detailed information on various heart failure causes (such as ischemic cardiomyopathy, valvular heart disease), pathological mechanisms (such as myocardial ischemia, myocardial remodeling), clinical features (such as elevated BNP, hyponatremia), and treatment plans (such as diuretics, ARNI drugs). The relationship definition unit defines the relationships between nodes, including the relationships between causes and pathological mechanisms, comorbidities and drug contraindications, drugs and metabolic pathways, etc. For example, the causal relationship between "coronary heart disease" and "myocardial ischemia" or the association between "renal insufficiency" and "diuretic dose adjustment" is defined. The dynamic updating unit extracts the latest clinical guideline content in real time through retrieval enhancement generation technology and updates it to the knowledge graph, ensuring the timeliness and accuracy of the knowledge base. In this module, the dynamic revision algorithm is implemented, which dynamically optimizes the knowledge graph by calculating the dynamic influence factor of the pathological state and adjusting the weight of the entity relationships in the knowledge graph.

[0088] In the reasoning module, the reasoning engine generates reasoning for cause analysis and treatment decision based on the constructed knowledge graph and the patient's individualized data. The drug dose adjustment unit dynamically adjusts the drug dose according to the patient's individualized parameters, such as kidney function, electrolyte level, blood pressure, etc. For example, according to the change of eGFR, the dose of diuretics is adjusted; according to the blood potassium level, it is decided whether to need potassium supplementation or adjust the drug combination. In this process, the conflict detection algorithm and the treatment path optimization algorithm are implemented in this module.

[0089] By calculating the conflict weight, the system can select the optimal solution among multiple possible treatment paths. The multi-path reasoning unit reasons and optimizes different treatment paths, combining the rules in the knowledge graph and the real-time state of the patient, to generate the optimal treatment plan. This process fully considers the patient's complex clinical situation, ensuring the safety and effectiveness of the treatment recommendations.

[0090] The treatment plan generation module outputs personalized treatment plans based on the results of the inference module. The volume status assessment unit assesses the degree of circulatory congestion and tissue edema by integrating the patient's biomarkers, hemodynamic parameters, and imaging data, providing a scientific basis for volume management. In this process, the hierarchical fusion formula (CFLF) is used to calculate the comprehensive results of volume load. The drug response prediction unit predicts the patient's response to different drugs based on the rules of the knowledge graph, helping doctors choose the most appropriate drugs and dosages. The dynamic adjustment unit combines real-time monitoring data such as urine output, electrolyte levels, and body weight changes to automatically optimize treatment plans, ensuring the accuracy and timeliness of treatment. During treatment, the system also uses the feedback update formula to optimize the weights of relevant rules in the knowledge graph based on the patient's actual treatment response, improving the system's adaptive ability.

[0091] In addition, the system also includes real-time monitoring modules and feedback optimization modules. The real-time monitoring module continuously collects dynamic data of patients and monitors key indicators. When an abnormality is detected, the warning mechanism will trigger in time to prompt the doctor and suggest adjusting the treatment plan. The feedback optimization module dynamically optimizes the treatment path and the rule weight of the knowledge graph according to the patient's treatment feedback and monitoring data, improving the intelligent level and adaptability of the system. Through the above dynamic adjustment mechanism, the system can timely reflect the changes of the patient's condition, ensuring the effectiveness of the treatment plan.

[0092] In order to realize the efficient fusion of cross-modal data, the system adopts hierarchical processing architecture, tensor packaging technology and dynamic weighting mechanism. The hierarchical processing architecture processes structured data, unstructured data and image data in multiple layers, extracts features and performs preliminary analysis. In this process, the modal interaction fusion formula (MIF) is used to strengthen the synergy between multi-modal data. The tensor packaging technology embeds multi-modal data into a unified tensor representation, eliminating the problem of data heterogeneity. The dynamic weighting mechanism dynamically adjusts the weights of different modal data according to the patient's condition and real-time state, optimizing the accuracy of the inference result.

[0093] The system also integrates a retrieval enhancement generation module that retrieves similar cases from the historical case database through a similar case retrieval unit. The case reference module combines the data of these similar cases with the reasoning results of the knowledge graph to further optimize the treatment plan. This function takes advantage of big data to provide valuable references for the processing of complex cases. In terms of model training, the model training module of the system fine-tunes the pre-trained heart failure large model through transfer learning and LoRA technology, improving the model's performance on specific tasks. The time series processing module optimizes the model's prediction ability for dynamic treatment adjustment based on time-labeled patient data, better capturing the changing trends of patient conditions. The verification module uses perfect medical records and common clinical error cases to evaluate the consistency and reliability of the system's decisions, ensuring the stability and accuracy of the system in actual application.

[0094] Specifically, to achieve the generation of personalized treatment recommendations, the present application constructs a system for the collection, integration, and analysis of multi-modal data, which comprehensively obtains the clinical characteristics and pathological information of patients from structured data, unstructured data, and image data. The data collection and processing process first starts with structured data, including three categories of laboratory indicators, hemodynamic parameters, and metabolic parameters. Among the laboratory indicators, NT-proBNP, as an important biomarker for heart failure diagnosis, can reflect the cardiac load status of patients. To improve its applicability in special populations (such as patients with impaired liver and kidney function), the present application corrects the liver and kidney metabolic effects of the original NT-proBNP data. At the same time, the ratio of BNP to NT-proBNP can further evaluate the cardiac pressure load and functional status of patients. The present application also collects the serum creatinine level of patients for evaluating kidney function, and analyzes whether the patient has electrolyte disorder by combining with the electrolyte (such as serum sodium and serum potassium) level, which plays a key role in optimizing diuretic use and potassium supplementation plan.

[0095] In terms of hemodynamic parameters, the application focuses on collecting central venous pressure (CVP), pulmonary artery systolic pressure (PASP), and cardiac output (CO). CVP data can accurately reflect the patient's volume status and degree of circulatory congestion, providing a scientific basis for diuretic therapy and volume adjustment. Changes in PASP are used to assess right ventricular load and pulmonary circulation pressure, providing important data support for adjusting treatment plans for pulmonary hypertension. Cardiac output, as an important parameter reflecting the pumping function of the heart and the perfusion capacity of the whole body, directly affects the selection of drug dosage and treatment strategy. In addition, the application also collects metabolic parameters of the patient, including estimated glomerular filtration rate (eGFR) calculated based on serum creatinine level, which is used to evaluate renal function and guide the use of ARNI drugs; serum albumin reflects the patient's nutritional status and metabolic reserve level, providing a basis for optimizing comprehensive treatment; changes in liver function indicators (ALT / AST) are used to determine whether the liver function is impaired due to heart failure and provide a reference for drug selection.

[0096] In addition to structured data, the application also uses natural language processing (NLP) technology and automated image analysis algorithms to extract clinical features of patients from unstructured data. The distribution of lung rales in electronic medical records is one of the important diagnostic criteria, and the application analyzes the physical examination description in the medical record text through NLP algorithm to extract the spatial distribution of rales (such as double lung bottom rales, right lung middle lobe rales) and its severity, and quantize it as structured data for evaluating the degree of pulmonary congestion. The degree of lower extremity edema is also important, and the application analyzes the descriptive text in the electronic medical record (such as "right lower extremity mild pitting edema" or "both lower extremities severe pitting edema") through NLP, and combines the grading standard of pitting edema to convert it into standardized data for volume status analysis.

[0097] Automatic extraction of image feature data is one of the important technical links of the application. Through the interface with PACS (Picture Archiving and Communication System), the application can directly obtain the chest ultrasound image data of the patient, and use the ultrasound image automatic segmentation algorithm to extract the B-line count. B-line count is an important indicator for evaluating the degree of pulmonary interstitial edema, which can reflect the volume load status of the patient. In addition, the application also uses the edge segmentation algorithm to calculate the left ventricular ejection fraction (LVEF) through the cardiac ultrasound image automatic analysis module to evaluate the left ventricular systolic function of the patient, providing key support for treatment plans such as the use of diuretics or the selection of positive inotropic drugs.

[0098] Therefore, the multi-modal data types include the following data:

[0099] 1. Clinical sign data:

[0100] Clinical signs data is an important information for patients to represent heart failure status, including jugular venous pressure, pulmonary rales, and lower extremity edema, etc. The present application parses these signs description from electronic medical records through natural language processing (NLP) technology, and converts unstructured text data into structured numerical data. For example, "mild pitting edema of lower extremity" can be parsed as numerical value 1 (mild), while "severe pitting edema of lower extremity" is parsed as numerical value 3 (severe).

[0101] In addition, the present application defines a specific volume load analysis model for quantifying the volume status of patients. For example, for coronary heart disease patients, the volume load status is calculated by the following formula:

[0102] Stasis index = 0.7 x BNP + 0.3 x central venous pressure;

[0103] This formula is designed based on clinical research data of coronary heart disease patients, which can more accurately reflect the volume load difference of such patients. For other causes (such as patients with cardiomyopathy), specific formulas will be used for analysis, such as volume load = 0.5 x albumin + 0.5 x tissue Doppler E / e' ratio, to ensure the individualization and accuracy of the analysis method.

[0104] 2. Laboratory indicators:

[0105] Laboratory test data is an important source of reflecting the patient's internal metabolic state, mainly including the following contents:

[0106] NT-proBNP dynamic trend: Monitor the change trend of heart failure related biomarkers in patients, and correct false positive values caused by liver dysfunction. For example, in patients with high bilirubin levels, NT-proBNP values will be overestimated, and the present application dynamically corrects NT-proBNP values by combining liver enzyme indicators (such as ALT, AST).

[0107] Electrolyte balance data: such as dynamic trend analysis of urinary sodium and potassium, for evaluating the sodium excretion capacity and electrolyte disorder status of patients, which has important reference value for the evaluation of diuretic sensitivity.

[0108] Kidney function indicators: including creatinine, eGFR (glomerular filtration rate) and serum albumin level, for evaluating the kidney function status and its influence on drug metabolism and treatment plan.

[0109] 3. Image data:

[0110] Image data is automatically extracted through the interface with PACS (Picture Archiving and Communication System) system, and the heart failure related image parameters are extracted, including:

[0111] Lung ultrasound B-line quantification: Using a deep learning segmentation algorithm, the number of B-lines in each frame of ultrasound images is quantified and the B-line density is calculated to assess the degree of pulmonary interstitial edema.

[0112] Cardiac chamber size and valvular regurgitation parameters: Cardiac chamber diameter, left ventricular ejection fraction (LVEF), and valvular regurgitation degree are extracted from cardiac ultrasound images to assess ventricular systolic function and hemodynamic status.

[0113] Other imaging features, such as ultrasound ascites volume (ml) and pulmonary artery systolic pressure (PASP), are used to assess volume load and right ventricular function.

[0114] 4. Hemodynamic parameters:

[0115] Hemodynamic parameters are the core data reflecting the state of the circulatory system, mainly including:

[0116] Central venous pressure (CVP): used to assess the volume status of the venous system and is an important indicator for volume load analysis.

[0117] Pulmonary artery systolic pressure (PASP): reflects the pressure load state of the right ventricle and is an important parameter for evaluating right heart function.

[0118] Tissue Doppler E / e' ratio: calculated by cardiac ultrasound tissue Doppler imaging and used to assess left ventricular diastolic function, especially in patients with cardiomyopathy.

[0119] 5. Drug metabolism data:

[0120] Drug metabolism data is used to assess a patient's ability to metabolize therapeutic drugs and to tailor their individualized dosages, primarily including:

[0121] Diuretic sensitivity index: Based on the sodium excretion fraction and renal function curve, it dynamically quantifies the patient's response to diuretics and assesses whether they have diuretic resistance.

[0122] CYP450 enzyme system and glomerular filtration rate synergistic effect model: By combining CYP450 enzyme system activity and eGFR value, the drug metabolic pathway is predicted and a basis for dose adjustment is provided.

[0123] After the above multi-modal data acquisition is completed, the application realizes the integration and analysis of data through cross-modal data fusion technology. First, the data from different sources are standardized, including unit conversion and distribution normalization, and unstructured data (such as lung rales and edema features extracted by NLP) and image data (such as ultrasound B-line count and LVEF) are converted into unified structured vector representation. Second, according to the specific condition of the patient (such as acute heart failure or stable period of chronic heart failure), the weight distribution of different types of data is dynamically adjusted, for example, the weight of CVP and B-line count is increased in volume adjustment, and the weight of eGFR and electrolyte level is increased in drug dosage optimization. Finally, combined with knowledge graph technology, the multi-modal data is analyzed in association with the etiology, pathological mechanism and treatment scheme of heart failure, and a personalized pathological feature model of the patient is constructed to provide high-quality input data for subsequent reasoning.

[0124] The application realizes all-round analysis of patient etiology, pathological mechanism, comorbidity and treatment selection through knowledge graph technology, and provides support for personalized treatment decision. As the core technology of the application, the construction process of the knowledge graph includes node definition, relationship construction and dynamic updating mechanism. The setting of node type covers three categories of disease entities, clinical features and treatment schemes, among which the disease entity nodes include common causes and comorbidities such as ischemic cardiomyopathy, valvular heart disease and diabetic nephropathy. The clinical feature nodes cover the main pathological manifestations of heart failure such as BNP elevation, hyponatremia and pulmonary congestion, and the treatment scheme nodes include standard drugs and treatment paths such as diuretics, ARNI (angiotensin receptor neprilysin inhibitor), beta blockers, etc. This node definition lays the foundation for the reasoning function of the knowledge graph.

[0125] In the knowledge graph, the relationship between nodes is defined by logical rules, which describes the association between etiology and pathological mechanism, comorbidity and drug contraindication, drug and metabolic pathway. For example, the etiology node "coronary heart disease" is associated with the pathological mechanism "myocardial ischemia" through logical reasoning, which further leads to the clinical manifestation of "BNP elevation"; the relationship between the comorbidity node "renal insufficiency" and the drug node "spironolactone" reflects the drug contraindication, which suggests that the drug should be used with caution; the association between the drug node "furosemide" and the metabolic pathway "renal excretion" indicates that the drug dosage needs to be adjusted in the case of reduced eGFR (glomerular filtration rate). The definition of these relationships enables the knowledge graph to dynamically adapt to the individual characteristics of the patient, and realizes accurate matching in the reasoning process.

[0126] The knowledge graph of the present application also has dynamic updating capability. The latest medical guidelines and drug databases are integrated in real time through the retrieval enhancement generation (RAG) technology to ensure that the content is always consistent with the current medical standards. For example, the system can extract treatment recommendations from the latest ESC (European Society of Cardiology) or ACCF / AHA (American College of Cardiology Foundation / American Heart Association) heart failure guidelines, and dynamically update the drug selection and dosage adjustment rule base in the knowledge graph. In addition, the knowledge graph can also supplement regional diagnosis and treatment experience by retrieving historical case data in the local diagnosis and treatment database, further improving its applicability and accuracy.

[0127] Based on the knowledge graph constructed as described above, the present application realizes dynamic reasoning logic for associating individualized clinical data of patients with the etiology, pathological mechanism and treatment plan of diseases. The reasoning process takes the etiology and comorbidities of the patient as input and gradually expands through the logical rules of the knowledge graph. For example, for a patient with an etiology of "ischemic cardiomyopathy" and a comorbidity of "diabetic nephropathy", the system first associates to the pathological mechanism "coronary ischemia" through the knowledge graph, and further infers the pathological results of "myocardial remodeling" and "BNP level increase". Subsequently, the system combines the patient's renal function status (eGFR reduction) to suggest adjusting the diuretic dose to avoid kidney damage, and recommends the use of tolvaptan to preferentially treat hyponatremia. This reasoning method can dynamically consider the multi-dimensional characteristics of the patient and generate personalized plans in combination with the treatment rules in the graph.

[0128] In terms of treatment decision generation, the present application further combines the reasoning results of the knowledge graph and the clinical guideline rules to form a closed-loop mechanism for volume status assessment and drug response prediction. Volume status assessment is a key link in treatment decision-making. The system infers that the patient has circulatory stasis by associating BNP elevation and CVP elevation and other indicators, and recommends using loop diuretics for volume management. At the same time, if the patient also shows B-line density elevation and hypoproteinemia, it is suggested that there may be tissue edema, and the use of albumin and diuretics is recommended for treatment. In terms of drug response prediction, the system analyzes the patient's drug tolerance based on the drug metabolism path rules of the knowledge graph. For example, when the patient's eGFR is <30, the system will automatically reduce the recommended dose of ARNI to 50% of the standard dose; when the blood potassium is <3.5 mmol / L, the system suggests the use of potassium supplement and potassium-sparing diuretics. Through this rule-based and logical reasoning method, the system can quickly generate a treatment strategy that meets the patient's condition.

[0129] The output of the personalized treatment plan is one of the ultimate goals of the present application. The system generates structured treatment recommendations, including the preferred plan and alternative plan, based on the knowledge graph reasoning results and clinical guideline rules. For example, for patients with circulatory congestion (BNP elevation and CVP elevation) and hyponatremia (serum sodium ≤135 mmol / L), the system recommends the use of furosemide (intravenous injection, 0.5 mg / kg) combined with tolvaptan (7.5 mg, once a day) as the preferred plan, and provides scientific basis according to the recommended path for HFrEF (heart failure with reduced ejection fraction) patients in the 2023 ESC heart failure guideline. If the patient has diuretic resistance (urinary sodium <20 mmol / L and eGFR decreases), the system suggests using ultrafiltration treatment (200 ml / h) as an alternative plan. In addition, the system also supports a dynamic adjustment mechanism, which updates the patient's state data every 4 hours by monitoring the patient's urine output, body weight and electrolyte levels in real time. For example, when the system monitors that the serum potassium level is lower than 3.5 mmol / L, it will automatically trigger a potassium supplement plan; when the eGFR decreases to below 30, the system will dynamically adjust the dose of ARNI. This dynamic adjustment mechanism ensures the personalization and real-time nature of the treatment plan.

[0130] The present application realizes the overall function of the system through the knowledge graph engine and the retrieval augmented generation (RAG) module. The knowledge graph engine integrates the contents of international authoritative guidelines such as ESC and ACCF / AHA, and combines with the DrugBank drug database and local historical diagnosis and treatment data to form a high-quality reasoning rule library. The RAG module dynamically generates personalized treatment paths by real-time retrieval of similar cases in the 100,000 heart failure database, combined with the patient's current data and knowledge graph reasoning results. Finally, the system can output individualized treatment plans (suitable for Stage C / D patients) that meet the ACCF / AHA staging standards, providing decision support for clinicians and realizing the closed-loop optimization from data collection, reasoning analysis to treatment plan output.

[0131] Through the above-mentioned technologies, the knowledge graph of the present application not only can accurately analyze the patient's etiology- comorbidity association, but also can dynamically adapt to the real-time changes of the patient's condition, providing scientific and comprehensive support for personalized treatment decisions, and greatly improving the efficiency and accuracy of primary hospitals in heart failure diagnosis and treatment.

[0132] The present application realizes the generation of heart failure diagnosis and personalized treatment recommendations by constructing a collection, integration and analysis system based on multi-modal data. This system comprehensively integrates structured data, unstructured data and image data, fully excavates the clinical characteristics and pathological information of patients, and combines with knowledge graph technology for dynamic reasoning and treatment decision-making, aiming to provide accurate, real-time and personalized treatment plans, and significantly improving the efficiency and accuracy of heart failure diagnosis and treatment.

[0133] The collected multi-modal data of the present application covers five categories of structured data, unstructured data, image data, hemodynamic parameters and drug metabolism data. Among them, the structured data includes laboratory indicators such as NT-proBNP, BNP and NT-proBNP ratio, blood creatinine, liver function indicators (ALT / AST), serum albumin, and glomerular filtration rate (eGFR) calculated based on serum creatinine level. As an important biomarker for heart failure diagnosis, NT-proBNP can reflect the cardiac load status of the patient. The present application improves its applicability in patients with impaired liver and kidney function by correcting its liver and kidney metabolism effect. The ratio of BNP to NT-proBNP further enhances the evaluation ability of cardiac pressure load and functional status. The level of blood creatinine is used to evaluate the renal function of the patient, and combined with the level of electrolyte (such as serum sodium and serum potassium), it is analyzed whether the patient has electrolyte disorder, which provides the basis for optimizing the use of diuretics and potassium supplement scheme. Changes in liver function indicators are used to determine whether the patient's liver function is impaired due to heart failure, providing a reference for drug selection; serum albumin reflects the nutritional status and metabolic reserve capacity of the patient, providing important support for the optimization of comprehensive treatment; eGFR is used to evaluate the renal function of the patient, providing guidance for the use of ARNI drugs.

[0134] In the collection of hemodynamic parameters, the present application focuses on obtaining central venous pressure (CVP), pulmonary artery systolic pressure (PASP) and cardiac output (CO). CVP data can accurately reflect the patient's volume status and degree of circulatory congestion, providing a scientific basis for diuretic therapy and volume adjustment; changes in PASP reflect right ventricular load and pulmonary circulation pressure, providing support for adjusting the treatment plan of pulmonary hypertension; cardiac output, as a core parameter reflecting the pumping function of the heart and the perfusion capacity of the whole body, directly affects the selection of drug dosage and treatment strategy. These hemodynamic parameters are standardized and participate in the reasoning of treatment decision-making together with other clinical data.

[0135] In addition, the present application extracts the clinical characteristics of the patient from the electronic medical record through natural language processing (NLP) technology. The distribution of lung rales in the electronic medical record is one of the important diagnostic bases, the present application extracts the spatial distribution (such as double lung bottom rales, right lung middle lobe rales) and severity of rales from the physical examination description in the medical record text, and quantifies them as structured data for evaluating the degree of pulmonary congestion. The degree of lower extremity edema is also important, the present application analyzes the descriptive text (such as "mild pitting edema" or "double lower extremity severe pitting edema") in the electronic medical record, combines the grading standard of pitting edema, and converts it into standardized data for volume status analysis.

[0136] Automatic extraction and quantification of image data is one of the core technologies of the present application. Through the interface with PACS (Picture Archiving and Communication System), the present application can directly obtain patient chest ultrasound image data, and use ultrasound image automatic segmentation algorithm to extract B-line count. B-line count is an important indicator for evaluating the degree of pulmonary interstitial edema, which can reflect the volume load state of the patient. In addition, the present application also uses an automatic analysis module for cardiac ultrasound images to calculate the left ventricular ejection fraction (LVEF) using edge segmentation algorithm to evaluate the left ventricular systolic function of the patient, and provide important support for treatment plan (such as the use of diuretics or the selection of positive inotropic drugs). The image data also includes heart cavity size, valve regurgitation degree, pulmonary artery systolic pressure, and ultrasound ascites volume, etc. These parameters are quantitatively processed by deep learning algorithm and converted into structured data that can be directly used by the model.

[0137] In the process of data integration and analysis, the present application uses multiple technologies to realize efficient fusion of multi-modal data. First, standardization processing is performed on data from different sources, including unit conversion and distribution normalization, to convert unstructured data (such as lung rales and edema features extracted by NLP) and image data (such as ultrasound B-line count and LVEF) into a unified structured vector representation. Second, the present application uses a hierarchical fusion architecture method to dynamically adjust the weight distribution of different types of data according to the specific condition of the patient (such as acute heart failure or chronic heart failure in stable period). For example, the weights of CVP and B-line count are increased in volume adjustment, while the weights of eGFR and electrolyte level are increased in drug dosage optimization. In addition, through heterogeneous data packaging technology, all multi-modal data are uniformly encoded as tensor representation, for example, text data is converted into a 256-dimensional embedding vector by BERT model, image data is encoded into an embedding vector of length 768 by ViT (Visual Transformer), and electrocardiogram waveform data is converted into a 1024-dimensional signal vector. This unified tensor representation ensures the processing and reasoning of different modal data in the same data space.

[0138] In order to further improve the semantic understanding ability of data fusion, the present application combines knowledge graph enhanced fusion technology. The knowledge graph contains rich medical concepts and their relationships, including disease entities (such as ischemic cardiomyopathy, valvular heart disease, etc.), clinical features (such as BNP elevation, hyponatremia, etc.), and treatment plans (such as diuretics, ARNI drugs, etc.). In the knowledge graph, the relationship between nodes is defined by logical rules, for example, the cause node "coronary heart disease" is associated to the pathological mechanism "myocardial ischemia" through logical reasoning, which further leads to the clinical manifestation of "BNP elevation"; the association between the drug node "furosemide" and the metabolic pathway "renal excretion" suggests that the drug dosage needs to be adjusted in the case of reduced eGFR. By mapping multi-modal data to knowledge graph nodes and performing association analysis, the system can achieve higher level semantic understanding based on data fusion.

[0139] Based on the knowledge graph, the application realizes dynamic reasoning logic for associating individualized clinical data of patients with etiology, pathological mechanism and treatment plan of diseases. For example, for a patient with a disease of "ischemic cardiomyopathy" and a complication of "diabetic nephropathy", the system firstly associates to the pathological mechanism of "coronary ischemia" through the knowledge graph, and further infers the pathological results of "myocardial remodeling" and "BNP level increase". Subsequently, the system combines the renal function status (eGFR decrease) of the patient, and prompts to adjust the diuretic dose to avoid kidney damage, and recommends the use of tolvaptan to preferentially treat hyponatremia. This reasoning method dynamically considers the multi-dimensional characteristics of the patient, and generates an individualized plan in combination with the treatment rules in the graph.

[0140] In terms of treatment decision generation, the application combines the reasoning results of the knowledge graph with the clinical guideline rules to form a closed-loop mechanism for volume status assessment and drug response prediction. Volume status assessment infers that the patient has circulatory stasis by associating indicators such as BNP elevation and CVP elevation, and suggests using loop diuretics for volume management. At the same time, if the patient also shows B-line density increase and hypoproteinemia, it is suggested that there may be tissue edema, and it is recommended to use albumin and diuretics for treatment. In terms of drug response prediction, the system analyzes the patient's drug tolerance based on the drug metabolism path rules of the knowledge graph, for example, when the patient's eGFR < 30, the system automatically reduces the recommended dose of ARNI to 50% of the standard dose; when the blood potassium < 3.5 mmol / L, the system prompts to use potassium supplement and potassium-sparing diuretics.

[0141] Through the knowledge graph engine and the retrieval augmentation generation (RAG) module, the application realizes the closed-loop optimization from data collection, reasoning analysis to treatment plan output. The knowledge graph engine integrates the contents of international authoritative guidelines such as ESC, ACCF / AHA, and combines DrugBank drug database and local historical diagnosis and treatment data to form a high-quality reasoning rule library. The RAG module dynamically generates individualized treatment paths by real-time retrieval of similar cases in the 100,000 heart failure database, in combination with the patient's current data and the reasoning results of the knowledge graph. Finally, the system can output an individualized treatment plan (suitable for Stage C / D patients) that meets the ACCF / AHA staging standard, providing precise decision support for clinicians.

[0142] The application realizes comprehensive analysis of the etiology, pathological mechanism and treatment needs of patients through multi-modal data fusion, knowledge graph reasoning and dynamic adjustment technology, provides scientific support for precise diagnosis and individualized treatment of heart failure, and provides technical support for improving the efficiency of heart failure diagnosis and treatment in primary hospitals.

[0143] The present application successfully realizes the precise diagnosis of heart failure patients and the dynamic generation of personalized treatment recommendations through the comprehensive integration and fusion technology of multi-modal data. In the fusion process of multi-modal data, the combined application of hierarchical fusion architecture method, heterogeneous data packaging technology and knowledge graph enhanced fusion effectively solves the problem of heterogeneity between data and improves the comprehensive processing ability of the model to multi-dimensional information. Through the hierarchical fusion architecture method, the system can perform differential volume load analysis on patients of different etiological types according to the empirical formula in the clinical guidelines. For example, for patients with coronary heart disease, the system focuses on combining BNP and central venous pressure data for volume load calculation, while for patients with cardiomyopathy, more attention is paid to the comprehensive evaluation of serum albumin level and tissue Doppler E / e' ratio. This etiology-specific calculation model ensures the accuracy of volume load analysis and provides a scientific basis for the development of subsequent treatment strategies.

[0144] In terms of data packaging, the heterogeneous data packaging technology realizes the unified expression of different modal data through standardization processing and tensorization representation. After the text data is segmented by the BERT model, it is converted into an embedded vector, the image data is encoded into a visual feature vector by ViT, and the electrocardiogram waveform data is represented as a high-dimensional signal vector. All modal data are packaged in the form of a unified tensor, establishing a unified input framework for cross-modal data. This packaging method not only solves the problem of data heterogeneity, but also provides a good foundation for efficient calculation and reasoning of the model. For example, the lung ultrasound B-line count in the image data and the clinical description in the text data can realize feature fusion in the same tensor space, further improving the synergy of multi-modal data.

[0145] Knowledge graph enhanced fusion is another technical core of the present application. By constructing a medical knowledge graph in the field of heart failure, the system performs deep correlation analysis of multi-modal data with the etiology, pathological mechanism and treatment of the disease. The nodes in the knowledge graph include three categories of disease entities, clinical features and treatment options, and the relationships between nodes are defined by logical rules. For example, the "coronary heart disease" node is associated with the "BNP elevation" clinical manifestation through the "myocardial ischemia" mechanism, and the "hypoproteinemia" node is further associated with the "furosemide resistance" treatment result through "interstitial edema". Combined with the Retrieval Augmented Generation (RAG) technology, the system can dynamically retrieve the latest content in medical guidelines and optimize the reasoning process of treatment options in real time. For example, when a patient is diagnosed with hypoproteinemia and has diuretic resistance, the system will automatically retrieve the relevant fragments in the ESC or ACCF / AHA guidelines and recommend a treatment strategy of combining albumin and tolvaptan. The combination of dynamic retrieval and knowledge graph reasoning enables the system to quickly generate scientific and precise treatment recommendations based on real-time changes in the patient's condition.

[0146] Through the above process, the present application not only realizes the efficient fusion of multi-modal data, but also shows significant advantages in the diagnosis and treatment of heart failure. First, the system can comprehensively cover clinical signs, laboratory indicators, image data, hemodynamic parameters and drug metabolism data, ensuring the comprehensiveness and multidimensionality of the data. Secondly, combined with the dynamic adjustment mechanism, the system can dynamically adjust the data weight and reasoning logic according to the real-time condition of the patient (such as acute heart failure attack or chronic heart failure stable period), ensuring the individualization and real-time of the treatment plan. In addition, based on the deep semantic understanding ability of the knowledge graph and the dynamic retrieval function of the RAG technology, the system can provide accurate decision support in complex cases, significantly improving the efficiency and scientificity of heart failure diagnosis and treatment.

[0147] The technical framework of the present application also pays special attention to the coverage and optimization of the whole process of doctor's work. Through the fusion of multi-modal data and knowledge graph reasoning, the system can form a complete closed-loop process from patient data acquisition, condition analysis to treatment suggestion generation. For example, in the evaluation of patient volume state, the system can combine the increase of BNP, the increase of central venous pressure and the change of lung B-line count to infer that the patient has circulatory stasis, and suggest to use loop diuretics for volume management first. In terms of drug dosage optimization, the system can dynamically adjust the drug dosage according to the patient's electrolyte level and renal function status (such as eGFR value), for example, when eGFR falls below 30, automatically prompt to reduce the ARNI drug dosage to 50% of the standard dosage. This closed-loop workflow greatly reduces the workload of doctors in complex cases, and also ensures the scientificity and individualization of the treatment plan.

[0148] The present application provides strong technical support for the accurate diagnosis and individualized treatment of heart failure through the combination of multi-modal data fusion, dynamic reasoning and knowledge graph enhancement technology. In terms of technical implementation, the system constructs a unified data input framework through hierarchical fusion architecture method, heterogeneous data packaging technology and knowledge graph enhancement fusion technology, and realizes dynamic optimization of decision support combined with RAG technology. In terms of clinical application, the system is widely applicable to the diagnosis of heart failure patients, volume state evaluation, drug dosage optimization and treatment plan recommendation, etc., providing an efficient and accurate decision support tool for clinicians, significantly improving the efficiency of heart failure diagnosis and treatment and the prognosis level of patients. In the future, the present application can be further extended to the diagnosis and treatment of other complex chronic diseases, providing intelligent support for a wider range of clinical scenarios.

[0149] The present application realizes the comprehensive integration and dynamic optimization of multi-modal data through the progressive layering of fusion, modal interaction effect calculation and knowledge enhancement analysis. The present application takes the hierarchical fusion architecture method as the basic step of the entire data fusion framework. The hierarchical fusion formula (CFLF) calculates the comprehensive results of volume load by combining the multi-modal data of the patient's etiology type, image features and biomarkers. The modal data is divided into different levels according to the etiology type, for example, the BNP and central venous pressure are mainly concerned for coronary heart disease patients, the serum albumin and E / e' ratio are the core indicators for cardiomyopathy patients, and the PASP and B-line count are more dependent on the evaluation of pulmonary arterial hypertension patients. The CFLF formula dynamically adjusts the weight of each layer of modal data to ensure that the fusion results can accurately reflect the volume load state of patients with different etiologies.

[0150] Firstly, the present application takes the hierarchical fusion architecture method as the basic step of the entire data fusion framework. The hierarchical fusion formula (CFLF) calculates the comprehensive results of volume load by combining the multi-modal data of the patient's etiology type, image features and biomarkers. The formula is as follows:

[0151] ;

[0152] Among them, represents the etiology weight of the first layer, reflecting the influence of the etiology on the volume load; is the modal feature value (such as BNP, central venous pressure, etc.) of the first layer; is the dynamic weight of the first layer, and the calculation formula is: ;

[0153] ;

[0154] In the formula, is the basic weight of the first layer, which is determined by clinical guidelines and cohort studies, is the real-time correlation parameter of the modal feature of the first layer and the patient's condition. The modal data is divided into different levels according to the etiology type, for example, the BNP and central venous pressure are mainly concerned for coronary heart disease patients, the serum albumin and E / e' ratio are the core indicators for cardiomyopathy patients, and the PASP and B-line count are more dependent on the evaluation of pulmonary arterial hypertension patients. For coronary heart disease patients, the calculation formula of volume load is: ;

[0155] ;

[0156] And for cardiomyopathy patients, the calculation formula of volume load is adjusted to:​​​​

[0157] ;

[0158] This hierarchical processing approach closely combines the patient's data characteristics with their etiology type, providing a solid foundation for subsequent modal interaction and knowledge association.

[0159] Secondly, based on hierarchical fusion, the present invention further enhances the synergy between multimodal data through the modality interaction fusion formula (MIF). The calculation of modality interaction effect aims to quantify the complementary relationship between different modality data. For example, the interaction between BNP and B-line count may better reflect the volume load, while The interaction with central venous pressure is more suitable for evaluating left ventricular diastolic function. The modality interaction fusion formula is:

[0160] ;

[0161] Among them, the modal interaction effect coefficient The calculation formula is:

[0162] ;

[0163] In the above formula, and For modal and modal characteristic values ​​(such as BNP, B-line count), and For modal and modal The eigenvector of is the normalization parameter, For modal and modal For example, when BNP and When the interaction effect of line counts is strong, the interactive fusion results can more accurately reflect the patient's volume load status.

[0164] Based on modal interaction calculations, this paper further combines the knowledge graph enhancement fusion formula (KEF) to deeply associate multimodal data with medical knowledge. The core of the knowledge enhancement fusion formula is to map modal features to node weights in the knowledge graph and calculate the correlation weights between nodes and data features:

[0165] ;

[0166] in, The first The weight of the node reflects the impact of the node on the current condition; For the relevance weight of the node and the modal feature, the calculation formula is:

[0167] ;

[0168] In the above formula, is the vector representation of the modal feature, is the embedding vector of the knowledge graph node, is the modal feature value (such as BNP or serum albumin) associated with the node . Through this formula, the invention can dynamically adjust the weight of each node in the knowledge graph, for example, when evaluating "hypoproteinemia", the system will preferentially combine serum albumin and tissue edema related data to generate a more accurate evaluation result.

[0169] Finally, the results of hierarchical fusion, modal interaction and knowledge enhancement are integrated into the overall dynamic fusion formula (TDF) to form a complete dynamic fusion framework:

[0170] ;

[0171] This formula integrates the characteristics of hierarchical data, the synergistic effect between modalities and the semantic enhancement of the knowledge graph in the final fusion, providing comprehensive support for personalized diagnosis and treatment of heart failure patients. For example, when the patient's BNP is , the central venous pressure is 12 mmHg, the serum albumin is , the E / e' ratio is 15, and the B-line count is 28, the system will first calculate the hierarchical volume load through CFLF, then evaluate the modal interaction effect through MIF, and finally map these data to the knowledge graph nodes through KEF to generate the overall dynamic fusion result, which is used to guide the development of treatment plans.

[0172] In the knowledge graph construction module, the dynamic updating unit uses the KEF formula to realize the dynamic adjustment of the weight of the knowledge graph node. Through the KEF formula, the dynamic updating unit can adjust the weight of each node in the knowledge graph according to the real-time data of the patient. For example, when evaluating "hypoproteinemia", the system will preferentially combine serum albumin and tissue edema related data to dynamically increase the weight of the relevant nodes, generating a more accurate evaluation result.

[0173] And the TDF formula in the data processing module is realized by the data standardization unit and the fusion calculation unit, which is responsible for integrating the results of CFLF, MIF and KEF to form a complete dynamic fusion framework. This process integrates the characteristics of hierarchical data, the synergistic effect between modalities and the semantic enhancement of the knowledge graph, providing comprehensive support for personalized diagnosis and treatment of heart failure patients.

[0174] The application provides a training method of a heart failure intelligent agent cooperation system, which realizes comprehensive integration and dynamic optimization of a heart failure diagnosis and treatment process through a time label-based data system and an intelligent agent cooperation training process.

[0175] In the design of the data system, the application adopts a time label-based organization method, all clinical data are attached with accurate time stamp information, and the intelligent agent can capture the time sequence dynamic change in the diagnosis and treatment process of the patient. The input data includes structured data and unstructured data, the structured data is, for example, laboratory indicators (BNP, urine sodium concentration), image quantitative results (B-line count) and dynamic monitoring data (such as 6-hour urine volume), and the unstructured data includes electronic medical record text, ultrasound description and doctor's advice and the like. The key information is extracted through natural language processing (NLP) technology, and is fused with the structured data, and finally forms a standardized time sequence input for the intelligent agent to generate dynamic reasoning and decision.

[0176] In the data system, the application takes a “perfect medical record” as a training benchmark. The perfect medical record refers to a high-quality case containing a complete diagnosis and treatment process, covering complete multi-modal data from admission to discharge, such as patient clinical observation, treatment response and next action suggestion. The typical data format is JSON, which records event type, time stamp, key parameters and next action. For example, in the diuretic response evaluation, the data may include urine sodium concentration, 6-hour urine volume and eGFR change rate, and the next action is suggested as “furosemide dose increase by 50%”. Another typical data format records a plurality of clinical observation events and their treatment responses, for example, “pulmonary rales range expands to the lower angle of scapula” and “24-hour body weight decreases by 1.2 kg after the use of tolvaptan”. These data are organized by time sequence, allowing the intelligent agent to dynamically analyze the patient state change and predict the next best action. In addition, the system also designs a data gap detection mechanism to find missing items through time sequence comparison, for example, not recording lung signs within the specified time or not performing the planned inspection operation.

[0177] The training process of the agent is divided into two stages: independent training and collaborative training. In the independent training stage, each agent focuses on learning professional judgment at a specific time point. For example, the diuretic response evaluation agent learns how to determine the next dose adjustment based on the changes in urine sodium concentration, urine volume, and eGFR. In the collaborative training stage, communication protocols such as HL7 FHIR are used to establish a collaboration mechanism between agents, enabling them to share information and work together to complete diagnosis and treatment tasks. The core objectives of training include time-sensitive next action prediction and clinical process integrity check. In time-sensitive prediction, the agent needs to generate a next action suggestion in real time based on the input data, such as recommending a cardiac ultrasound examination within 2 hours when the BNP level exceeds 500 pg / ml. In the clinical process integrity check, the agent can automatically find missing diagnosis and treatment links to ensure that each important clinical operation is performed, such as issuing a reminder when a lung sign is not recorded or central venous pressure is not reviewed.

[0178] To verify the performance of the agent, the present application designs a training evaluation mechanism based on perfect medical records. The decision-making ability of the model is evaluated by comparing it with perfect medical records, mainly investigating the time deviation between the agent's suggestions and the actual diagnosis and treatment plan, as well as the consistency of key decision nodes. For example, at the key node of diuretic adjustment, whether the agent's suggestion is consistent with the doctor's actual operation. At the same time, by intentionally introducing missing data (such as missing urine sodium concentration or cardiac ultrasound results), the agent's ability to complete data gaps is tested, and "imperfect medical records" are used to simulate common clinical errors (such as not reviewing electrolytes in time), to investigate the agent's error correction ability. This verification mechanism ensures that the agent not only performs well under ideal conditions, but also adapts to complex clinical scenarios.

[0179] In the processing of input and output data, the present application adopts standardized methods to ensure the uniformity of data and the universality of the model. Structured data is normalized to ensure consistency of different units and dimensions; unstructured data is extracted by NLP model to obtain key parameters and time labels, and then fused with structured data to form a unified time series input. The output data includes the next action suggestion generated by the agent and the integrity check result of the diagnosis and treatment process. For example, the system may output specific suggestions such as "furosemide dose increased by 50%" or "found that central venous pressure was not reviewed", for the doctor to refer to.

[0180] As shown in Figure 2 The present application adopts a multi-level processing flow based on agent collaboration to gradually realize the whole process management from data input to decision output. The whole process starts with time-sliced data, and through layer-by-layer processing and collaborative reasoning of the agent, finally outputs accurate diagnosis and treatment decisions. Specifically, the following processes are included:

[0181] First, the system performs time slicing on the input multimodal data, organizing structured and unstructured clinical data into time series. Time-sliced ​​data is the core input for the entire process, ensuring that all data accurately reflects the dynamic changes in the patient's condition. This process effectively integrates timestamp information with multimodal data, providing a foundation for subsequent agent reasoning.

[0182] Next, the time-sliced ​​data is passed in parallel to two core agent modules: the Clinical Information Agent and the Laboratory Interpretation Agent. The Clinical Information Agent is responsible for processing unstructured data related to patient vital signs, medical history, and electronic medical record text. It extracts key features using natural language processing techniques and labels key clinical events. The Laboratory Interpretation Agent focuses on analyzing structured data, including laboratory indicators, imaging features, and dynamic monitoring data. For example, the Laboratory Interpretation Agent can infer a patient's volume load status based on information such as BNP and urine volume, and evaluate treatment efficacy based on time series changes. These two agents analyze data from different dimensions and aggregate the results into the Diagnostic Test Agent.

[0183] The Diagnostic Test Agent is the core node of the entire process, responsible for synthesizing the analysis results from the first two layers of agents to generate preliminary diagnostic conclusions and test recommendations. For example, if the Clinical Information Agent identifies a patient's description of increased pulmonary crackles, and the Laboratory Interpretation Agent detects a decrease in urine sodium concentration and fluctuations in eGFR, the Diagnostic Test Agent can combine this information to determine that the patient may be volume overloaded and recommend adjusting the diuretic dose or performing an ultrasound examination.

[0184] The initial diagnostic results are then passed to the Clinical Management Agent, which is responsible for optimizing treatment plans and providing dynamic management recommendations. For example, if the Diagnostic Testing Agent concludes volume overload, the Clinical Management Agent dynamically adjusts medication dosage based on the patient's real-time data and treatment response and generates a follow-up plan. The Follow-up Agent then develops a detailed follow-up schedule based on the treatment plan, including review schedules, examination items, and methods for monitoring response indicators.

[0185] Throughout the entire diagnosis and treatment process, the process monitoring agent monitors all steps in real time to ensure the integrity and accuracy of the treatment process. For example, the agent can identify if key procedures are omitted, such as not checking central venous pressure as planned or not recording weight changes. Furthermore, the process monitoring agent verifies the quality and consistency of data, ensuring the scientific and reliable nature of subsequent decisions.

[0186] Finally, after the collaborative processing of all agents, the system will finally deliver the final diagnosis and treatment plan to the clinician through the decision output module. The output decision content includes clear diagnostic conclusions, treatment recommendations based on real-time data, follow-up plans, and potential risk prompts.

[0187] Through this multi-level agent collaboration framework, the system can achieve accurate and efficient diagnosis and treatment support in complex clinical scenarios, significantly improving the level and efficiency of intelligent diagnosis and treatment of heart failure.

[0188] The present application proposes a system method for constructing a heart failure knowledge graph, which forms a comprehensive knowledge network that can support diagnosis and treatment decisions by comprehensively integrating clinical guidelines, multi-modal data, and dynamic relationship modeling. The heart failure knowledge graph is the core knowledge base of the system, and its construction process covers data source integration, knowledge representation and relationship modeling, technical implementation path, and verification and iterative optimization. The construction method is described in detail below.

[0189] First, the construction of the heart failure knowledge graph is based on the integration of multi-source data. The present application extracts key knowledge points from clinical guidelines and multi-modal data to form a complete knowledge system framework. In the aspect of clinical guideline analysis, the system analyzes the heart failure diagnosis and treatment guidelines (such as ACC / AHA and ESC guidelines) and converts them into a machine-readable rule framework. The analysis content includes the pathological mechanism differences of etiological classification (such as coronary heart disease, cardiomyopathy, valvular disease, etc.), drug use rules (such as ARNI dosage adjustment, diuretic sensitivity threshold), and the disposal logic of comorbidities (such as drug contraindications in the presence of renal dysfunction). In addition, the system also integrates multi-modal data, including laboratory indicators (such as BNP, electrolyte concentration), imaging features (such as ultrasound B-line count), and unstructured clinical features extracted from electronic medical records (such as NLP extracted jugular venous distension description). The integration of these data sources provides rich basic information for the construction of the knowledge graph.

[0190] In terms of knowledge representation and relationship modeling, the present application constructs the knowledge graph in the form of medical concept triples (entity-relation-entity). For example, for the association between heart failure etiology and treatment, the following knowledge triples can be established: [coronary heart disease] → [causes] → [circulatory congestion]; [furosemide] → [contraindication] → [creatinine clearance rate <30 ml / min]. Through this explicit relationship modeling, not only can the associations between heart failure etiology, treatment, and comorbidities be clearly represented, but also the automatic reasoning of clinical diagnosis and treatment pathways can be supported. In addition, the present application also develops a dynamic correction algorithm to calculate the impact of specific conditions (such as liver and kidney dysfunction) on biomarkers and treatment plans. For example, when a patient has liver dysfunction, the system can dynamically adjust the threshold of BNP to optimize the diagnostic accuracy.

[0191] In terms of technical implementation path, the application combines large model technology and tool chain integration to realize automatic construction and updating of knowledge graph. In large model assisted construction, RAG (retrieval augmented generation) technology is used to extract question and answer pairs from guidelines, generate structured knowledge nodes, and fine-tune the model through LoRA (low rank adaptation) technology to make the knowledge graph closely aligned with clinical practice. For example, when updating new diagnosis and treatment guidelines, the system can analyze the text content through NLP technology and automatically add new rules to the knowledge graph.

[0192] In addition, the system also integrates tool chain for rule conflict detection and compatibility verification of comorbidities of new knowledge points, to ensure the logical consistency and clinical applicability of the knowledge graph.

[0193] After the construction of the knowledge graph, the application further improves its accuracy and practicality through verification and iterative optimization. First, clinical experts cooperate to correct the key decision nodes of the knowledge graph. For example, for the special rules of volume management of patients with valvular disease, heart disease experts annotate and correct the knowledge nodes involved. Second, through the reverse verification of actual cases, the differences between the treatment plan recommended by the knowledge graph and the actual case disposal are compared, and the weights of the key relationships in the graph are dynamically optimized. For example, when the diuretic response curve in the knowledge graph is inconsistent with the actual case performance, the system will automatically adjust the threshold or weight to better adapt to the real clinical scene.

[0194] The final heart failure knowledge graph can be presented in the form of a visual network, clearly showing the relationships between etiology, treatment and comorbidities. For example, in the visualization interface, users can query the "coronary heart disease" node and intuitively see information such as associated pathological mechanisms, recommended treatment plans and contraindicated drugs. At the same time, each knowledge node is labeled with an evidence level (such as guideline recommendation level) to support traceability queries for clinical decisions, ensuring the scientificity and reliability of diagnosis and treatment recommendations.

[0195] As can be seen, the application integrates clinical guidelines, multi-modal data and large model technology to construct a dynamic and accurate heart failure knowledge graph. The graph not only supports precise diagnosis and personalized treatment of heart failure, but also has high scalability and can be dynamically optimized as medical knowledge updates.

[0196] The dynamic correction algorithm of the present application provides a complete technical implementation process for the dynamic optimization of the heart failure knowledge graph. Through multi-step calculation and adjustment, it ensures that the knowledge graph can adapt to the pathological state of the patient in real time and generate accurate personalized diagnosis and treatment recommendations. The first step of the dynamic correction algorithm is the calculation of the dynamic influence factor of the pathological state. The core of this step is to quantify the dynamic influence degree of the patient's current pathological state on the relevant rules in the knowledge graph. The system calculates the dynamic influence factor by the formula , where represents the key parameter value of the pathological state, such as ALT (glutamic-pyruvic transaminase) value, serum sodium concentration, etc.; is the normal reference value midpoint of the parameter, used for standardizing the abnormality degree; represents the change of the pathological state to the initial rule weight; is a normalization parameter used to control the smoothness of adjustment. For example, when the patient's ALT value is (normal range is ), the system calculates the dynamic influence factor , indicating that the abnormal liver function has a greater impact on the relevant rules.

[0197] After the dynamic influence factor is calculated, the system will dynamically adjust the entity relationship weight in the knowledge graph to reflect the real-time state of the patient. The adjustment formula is:

[0198] , which is used to recalculate the relationship weight between entity and entity in the knowledge graph. Here, represents the initial weight, is a dynamic adjustment coefficient used to control the amplification effect of the dynamic influence factor on the weight adjustment. For example, if the initial weight of "[Furosemide] [Contraindication] [Creatinine clearance rate] = 0.8 in the knowledge graph, the patient's renal dysfunction influence factor , and the adjustment coefficient , then the adjusted weight . The dynamic adjustment of the weight reflects the change in the importance of the current pathological state to the rule, helping the system to generate more accurate diagnosis and treatment recommendations.

[0199] After adjusting the weight, the system optimizes the possible treatment paths through the conflict detection algorithm and identifies potential contradictions in the paths. The calculation formula of the conflict weight is , which is used to quantify the conflict degree of each treatment path, where is the weight of the entity relationship in the path, ​is an indicator function of whether there is a contradictory relationship in the path (a value of 1 indicates a conflict, and 0 indicates no conflict). For example, in a certain treatment path, if the weights of [ACEI] [contraindication] and [diuretics] [may cause] the weights of [low blood potassium] are 0.6 and 0.7 respectively, and both are related to potential conflicts, then the conflict weight is 0.6 + 0.7 = 1.3. The system preferentially selects the treatment path with the smallest conflict according to the conflict weight, ensuring the safety and effectiveness of the treatment plan.

[0200] After determining the treatment path, the system supports a dynamic feedback and knowledge graph updating mechanism. By monitoring the patient's response to the treatment plan in real time (such as the magnitude of weight loss, increase in urine output, etc.), the system dynamically optimizes the rules and weights in the knowledge graph. The update formula is:

[0201] is used to adjust the weight in real time, where is the current weight, is the evaluation coefficient (such as the percentage of patient weight change), is the feedback adjustment coefficient. For example, when the patient's weight decreases by 1.5 kg after adjusting the diuretic dose, and the urine output increases significantly, the system calculates the weight of [furosemide] [dose adjustment] to be raised from 0.7 to 0.756 after feedback adjustment. Through this dynamic feedback mechanism, the system can continuously optimize the rules and weights of the knowledge graph, making it consistent with the patient's actual condition.

[0202] The dynamic correction algorithm of the present invention has shown significant innovation and practical value in clinical application. For example, in the case of abnormal liver function, the system can dynamically increase the diagnostic threshold of BNP to avoid misjudgment due to abnormal liver function; in the case of low blood sodium, the system can reduce the priority of ACEI use to reduce drug-induced risks; in the case of good treatment response, the system can optimize the priority of the current path through dynamic feedback. These functions not only significantly improve the dynamic adaptability of the knowledge graph, but also provide strong technical support for personalized decision-making in heart failure diagnosis and treatment.

[0203] Through a series of dynamic influence factor calculations, weight adjustments, conflict detection, and feedback optimization, the present invention constructs a closed-loop knowledge graph optimization framework. This framework can dynamically adjust the knowledge graph according to the patient's real-time pathological state, transforming from static rules to dynamic decision-making, and continuously optimizing the accuracy of treatment recommendations. This design provides an innovative solution for intelligent diagnosis and treatment of heart failure and other complex chronic diseases, and has broad application prospects.

[0204] For example, in the clinical scenario of liver dysfunction, when the patient's ALT and AST levels are significantly elevated, the system first evaluates the impact of liver dysfunction on the diagnostic rules through the dynamic impact factor calculation formula. The abnormality of ALT and AST will cause the diagnostic threshold of BNP to be appropriately increased to avoid misdiagnosis caused by false elevation of BNP level due to liver dysfunction. For example, when the patient's ALT value is 80 U / L (normal range is 0-40 U / L), the system calculates the dynamic impact factor Di=2.42D_i = 2.42Di=2.42 through the formula. Subsequently, the system dynamically adjusts the weight of the diagnostic rules related to BNP in the knowledge graph according to this factor, so that the diagnostic threshold of BNP is increased to a range suitable for the current state of the patient, thereby improving the accuracy of diagnosis.

[0205] For example, in the scenario of impaired renal function, the knowledge graph strengthens the adaptability of drug contraindications by dynamically adjusting the rule weights. For example, when the patient's creatinine clearance rate decreases to close to 30 ml / min, the system significantly increases the weight of "[furosemide] → [contraindication] → [creatinine clearance rate < 30 ml / min]" to remind the doctor to pay attention to the risk of drug use. If the patient's dynamic impact factor Di=1.5D_i = 1.5Di=1.5, the original weight of the rule is 0.8, and the adjustment coefficient is 0.3, then the adjusted weight increases to 1.16, indicating that the importance of this contraindication rule in the current pathological state is significantly increased.

[0206] The scenario of electrolyte disturbance further demonstrates the application of the system in the optimization of treatment paths. For example, when the patient has low blood sodium (serum sodium concentration < 135 mmol / L) or low blood potassium, the system will quantify potential conflicts in the treatment path through the conflict detection formula. For example, in a certain treatment path, "[ACEI] → [contraindication] → [low blood sodium]" and "[diuretics] → [may cause] → [low blood potassium]" are both relevant rules, and the weights are 0.6 and 0.7, respectively. The system calculates the conflict weight Ck=0.65C_k = 0.65Ck=0.65 through the formula, and preferentially selects the path with lower conflict weight, thereby avoiding potential drug interactions or adverse reactions.

[0207] In the feedback update scenario of treatment response, the system adjusts the weights of the knowledge graph by monitoring the patient's response to the treatment plan in real time. For example, after adjusting the diuretic dose, the patient's weight decreases by 1.5 kg and urine output increases by 500 ml, indicating that the current treatment plan is effective. The system raises the weight of "[furosemide] -> [dose adjustment]" from 0.7 to 0.756 through the feedback optimization formula to strengthen the priority of the current path. At the same time, the weights of other treatment paths are reduced accordingly to ensure that the optimized knowledge graph is more suitable for the patient's actual situation.

[0208] This complete dynamic correction mechanism embodies the closed-loop optimization capability of the knowledge graph, i.e., from the detection of pathological states to the calculation of dynamic impact factors, then to the adjustment of knowledge graph rule weights and optimization of treatment paths, and finally through real-time feedback to continuously update the rule weights, forming a dynamically adaptive diagnosis and treatment decision system. Through this mechanism, the system can capture subtle changes in the patient's condition in real time and map them to the corresponding rules in the knowledge graph, generating precise and personalized diagnosis and treatment recommendations.

[0209] The dynamic correction algorithm of the present application constructs an intelligent knowledge graph system through the quantification of dynamic impact factors of pathological states, real-time adjustment of knowledge graph weights, optimization of treatment paths, and closed-loop mechanism of feedback update. The system not only can dynamically adjust the weights of diagnosis and treatment rules, but also can effectively identify potential risks in complex clinical scenarios, optimize treatment paths, and continuously improve itself according to the patient's treatment response.

[0210] Note that the computer readable medium described above can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the disclosure, the computer readable signal medium can include a computer readable program code propagated on or through a computer readable medium, in baseband or as part of a carrier wave. The computer readable signal medium can take a variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0211] The computer readable medium described above can be included in the electronic device described above; alternatively, the computer readable medium can exist as a separate entity in which the electronic device is incorporated.

[0212] Computer program code for carrying out operations of the disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0213] The computer program product of the present disclosure can be a computer program embodied on a non-transitory computer readable medium. When the program runs on a computer, the flowchart and / or block diagram in the flowchart and / or block diagram can be implemented.

[0214] The units described in the embodiments of the present disclosure can be implemented by software or hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0215] The above describes the preferred embodiments of the present disclosure, which aims to make the spirit of the present disclosure clearer and easier to understand, and is not intended to limit the present disclosure. Any modification, replacement, improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the appended claims of the present disclosure.

Claims

1. A heart failure treatment auxiliary decision-making system based on multimodal data fusion, characterized in that: The system comprises: Data acquisition module, used to collect multimodal data of patients, including structured data, unstructured data and medical imaging data; A data processing module, configured to perform standardization, quantization, and vectorization on the multimodal data; A knowledge graph construction module is used to construct a knowledge graph for heart failure treatment, including disease entities, pathological characteristics, treatment plans, and their relationships; The reasoning module is used to generate personalized treatment decisions based on the knowledge graph and patient data; The treatment plan generation module is used to dynamically adjust and output personalized treatment plans based on the patient's real-time status data.

2. The heart failure treatment auxiliary decision-making system based on multimodal data fusion according to claim 1, characterized in that: The data acquisition module further comprises: Laboratory data collection unit, used to collect patients' biomarkers, renal function indicators, and electrolyte levels; Unstructured data collection unit, used to extract clinical description data from electronic medical record text, including the distribution of lung rales and the degree of lower limb edema; The medical image acquisition unit is used to extract characteristic data of echocardiogram and chest image from the medical imaging system and convert it into structured data.

3. The heart failure treatment auxiliary decision-making system based on multimodal data fusion according to claim 1, characterized in that: The data processing module includes: Unstructured data parsing unit, used to convert descriptive text data in electronic medical records into structured data through natural language processing technology; An image data quantification unit is used to quantify ultrasound B-line counts, left ventricular ejection fraction (LVEF) and their image characteristics; The data normalization unit is used to uniformly convert data from different modalities into embedded vector representations to support subsequent inference processing.

4. The system according to claim 1, wherein: The knowledge graph construction module includes: Node construction unit, used to construct knowledge nodes related to heart failure, including disease entities, pathological mechanisms, drugs and treatment plans; The relationship definition unit is used to define the relationship between nodes, including at least the following relationships: the relationship between etiology and pathological mechanisms; the relationship between comorbidities and drug contraindications; and, the relationship between drugs and metabolic pathways; Dynamic update unit, used to extract the latest clinical guidelines and update the knowledge graph in real time through retrieval enhancement generation technology.

5. The system according to claim 1, wherein: The reasoning module includes: Reasoning engine, used to generate etiology analysis and treatment decisions based on knowledge graphs and patient data; A drug dosage adjustment unit, used to dynamically adjust drug dosages based on individual patient parameters, including diuretic dosages based on changes in renal function, and triggering potassium supplementation or adjusting drug combinations based on electrolyte levels; The multi-path reasoning unit is used to reason and optimize different treatment paths to generate the optimal treatment plan.

6. The system according to claim 1, wherein: The treatment plan generation module includes: a volume status assessment unit for generating an assessment of circulatory congestion and tissue edema based on the patient's biomarkers, hemodynamic parameters, and imaging data; Drug response prediction unit, used to predict patients' responses to different drugs based on knowledge graph rules; Dynamic adjustment unit, used to automatically optimize treatment plans based on real-time patient monitoring data.

7. The system according to claim 1, wherein: The system further comprises: A real-time monitoring module is used to collect patients' dynamic data, including urine volume, electrolyte levels, and weight changes; An early warning mechanism is used to trigger warnings and recommend adjustments to treatment plans when key indicators are abnormal; Feedback optimization module, used to dynamically optimize the treatment pathway based on real-time monitoring data.

8. The system according to claim 1, wherein: The system adopts cross-modal data fusion technology, including: Layered processing architecture for multi-level processing of structured data, unstructured data, and image data; Tensor encapsulation technology, used to embed multimodal data into a unified tensor representation; Dynamic weighting mechanism dynamically adjusts the weights of different modal data according to the patient's condition to optimize the inference results.

9. The system according to claim 1, wherein: The system further includes a search enhancement generation module, comprising: A similar case retrieval unit is used to retrieve cases similar to the current patient's condition from the historical case database; The case reference module is used to combine similar case data with knowledge graph reasoning results to optimize treatment plans.

10. The system according to claim 1, wherein: The system further comprises: Model training module, used to fine-tune the large heart failure model through transfer learning and LoRA technology; A time series processing module is used to optimize the model's predictive ability for dynamic treatment adjustments based on time-labeled patient data; The validation module is used to evaluate the consistency and reliability of system decisions using perfect case records and common clinical error case records.

Images