Method and apparatus for cardiac function assessment and prognosis prediction
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING INST OF TECH
- Filing Date
- 2026-02-09
- Publication Date
- 2026-06-19
Smart Images
Figure CN122245742A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical artificial intelligence, medical image analysis and cardiovascular disease prognosis prediction, and in particular to an artificial intelligence-driven non-invasive method and device for assessing cardiac function and predicting prognosis in pulmonary hypertension. Background Technology
[0002] Pulmonary arterial hypertension (PAH), a high-risk cardiopulmonary vascular disease, has a core pathological mechanism of increased pulmonary vascular resistance leading to increased right ventricular load, resulting in right ventricular dilation, decreased compliance, and reduced pumping function, ultimately progressing to right heart failure. Clinically, quantitative assessment of the degree of cardiac function impairment and prognostic risk stratification in PAH patients are crucial.
[0003] The severity assessment of PAH-related RVD is highly dependent on hemodynamic parameters such as right atrial pressure, pulmonary artery pressure, and cardiac output. Currently, due to the limited predictive accuracy of non-invasive imaging methods, cardiac hemodynamic evaluation mainly relies on right heart catheterization (RHC). However, this invasive procedure carries risks such as puncture injury, infection, and bleeding, and requires repeated procedures, which limits patient safety and medical efficiency while significantly increasing medical costs.
[0004] In recent years, the application of artificial intelligence in the quantitative assessment and risk prediction of cardiovascular diseases has been expanding. However, the current field lacks a dedicated intelligent non-invasive quantitative system for PAH-RVD, especially a full-link AI model for cardiac structure-function-prognosis.
[0005] Therefore, how to provide an intelligent system that can perform non-invasive quantitative assessment of the cardiac structure and hemodynamic characteristics of patients with pulmonary hypertension, and further realize prognostic risk prediction, in order to replace some RHC functions and improve the accuracy of diagnosis and prognostic prediction capabilities, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] In view of the above problems, the present invention provides a method and apparatus for cardiac function assessment and prognosis prediction to overcome or at least partially solve the above problems. It solves the problems of existing cardiac function assessments, such as reliance on RHC, high invasiveness, and inaccurate ultrasound imaging. It achieves non-invasive, automated, precise, and interpretable cardiac function analysis.
[0007] This invention provides the following solution:
[0008] A method for assessing cardiac function and predicting prognosis includes: Acquire the patient's multimodal medical data and preprocess the multimodal medical data, which includes at least multi-sectional echocardiographic images, Doppler blood flow spectrum images, and clinical data; Based on the preprocessed multi-section echocardiogram image data, the key cardiac structures are automatically segmented using a deep learning segmentation model to obtain cardiac structural features. Based on the cardiac structural features, cardiac morphological parameters are extracted and / or motion tracking is performed to extract cardiac temporal functional parameters. Based on the preprocessed Doppler blood flow spectrum image, hemodynamic spectrum features are extracted; By integrating the cardiac structural features, the hemodynamic spectral features, and the preprocessed clinical data, at least one cardiac hemodynamic parameter is non-invasively predicted using a multi-task regression deep learning model. Based on the cardiac structural features, cardiac hemodynamic parameters, and clinical data, a prognostic prediction deep learning model is used to quantitatively assess the risk of future disease progression in patients and predict the risk of patients experiencing preset endpoint events, including the risk of death, readmission, and clinical deterioration in patients with pulmonary hypertension.
[0009] Preferably, the multi-plane echocardiographic image data includes at least two of the following: apical two-chamber view, apical four-chamber view, right ventricular outflow tract view, parasternal left ventricular long-axis view, parasternal short-axis view, and inferior vena cava view; the Doppler blood flow spectrum image includes at least two of the following: tricuspid regurgitation spectrum, pulmonary artery blood flow spectrum, mitral valve orifice anterior blood flow spectrum, tricuspid valve orifice anterior blood flow spectrum, and aortic valve anterior blood flow spectrum.
[0010] Preferably, the deep learning segmentation model is a 2D nnU-Net network with an attention mechanism, used to segment at least one of the following structures: left ventricle, left atrium, right atrium, right ventricle, and inferior vena cava; the cardiac morphological parameters include at least one of the following: left / right ventricular end-diastolic area, left / right ventricular end-systolic area, tricuspid annulus systolic displacement, mitral annulus systolic displacement, inferior vena cava diameter, and right myocardial thickness.
[0011] Preferably, the method further includes motion tracking of the cardiac structure using optical flow estimation based on the segmentation results, in order to extract temporal functional parameters of the heart.
[0012] Preferably, the hemodynamic spectral characteristics include at least one of peak flow velocity, acceleration time, deceleration time, and velocity-time integral.
[0013] Preferably, the multi-task regression deep learning model employs a cross-view attention mechanism and a temporal attention mechanism to fuse features from multiple views of echocardiography; the cross-view attention mechanism is used to align and fuse features from different ultrasound sections under the same cardiac phase; and the temporal attention mechanism is used to model the dynamic changes of the complete cardiac cycle within the same ultrasound section.
[0014] Preferably, the fused ultrasound structural features, the hemodynamic spectral features, and the clinical data are aligned to form a multimodal fused feature; The multimodal fusion features are input into a shared fully connected network to learn common latent representations; Based on the aforementioned common potential characterization, different cardiac hemodynamic parameters are predicted through multiple independent regression branches; the cardiac hemodynamic parameters include at least two of the following: mean pulmonary artery pressure, pulmonary artery systolic pressure, right atrial pressure, right ventricular pressure, cardiac output, pulmonary vascular resistance, and pulmonary capillary wedge pressure.
[0015] Preferably, the prognostic prediction deep learning model is a time-to-event prediction model built on a deep neural network, used to output an individualized risk score and / or survival probability curve for the patient to experience at least one endpoint event, such as death, readmission, or deterioration of cardiac function, within a preset follow-up period.
[0016] Preferably, the follow-up time axis is discretized into several adjacent time intervals, and multiple output nodes corresponding one-to-one with the time intervals are set at the network output end to predict the risk of the end event occurring in each time interval. By multiplying the conditional risks of each time interval, an individualized survival function and the probability of occurrence of the corresponding death or composite endpoint event at any preset follow-up time point are obtained, thereby achieving joint prediction of patient survival and short-, medium- and long-term prognostic risks.
[0017] A cardiac function assessment and prognostic prediction device for performing the above-described cardiac function assessment and prognostic prediction method, the device comprising: The data acquisition and preprocessing unit is used to acquire the patient's multimodal medical data and preprocess the multimodal medical data, which includes at least multi-sectional echocardiographic images, Doppler blood flow spectrum images, and clinical data. The ultrasound structure segmentation and motion tracking unit is used to automatically segment key cardiac structures based on the preprocessed multi-section echocardiogram image data using a deep learning segmentation model to obtain cardiac structural features, and extract cardiac morphological parameters based on the cardiac structural features and / or perform motion tracking to extract cardiac temporal functional parameters. The Doppler spectral feature extraction unit is used to extract hemodynamic spectral features based on the preprocessed Doppler blood flow spectral image. The non-invasive cardiac function index assessment unit is used to fuse the cardiac structural features, the hemodynamic spectral features and the preprocessed clinical data, and non-invasively predict at least one cardiac hemodynamic parameter through a multi-task regression deep learning model. The patient prognosis prediction unit is used to quantitatively assess the risk of future disease progression of patients based on the cardiac structural features, cardiac hemodynamic parameters and clinical data, using a prognosis prediction deep learning model, and to predict the risk of patients experiencing preset endpoint events, including the risk of death, rehospitalization and clinical deterioration in patients with pulmonary hypertension.
[0018] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This application provides a method and device for cardiac function assessment and prognosis prediction, which realizes non-invasive cardiac function assessment. By using a deep learning model to perform structural analysis and hemodynamic parameter prediction on ultrasound images, it can partially replace right heart catheterization, enabling the acquisition of hemodynamic information that can only be measured by traditional RHC without invasive means, thereby reducing patient risk.
[0019] This invention improves the accuracy and repeatability of cardiac structural parameter measurements. By using an automatic segmentation and motion tracking model based on echocardiography, it avoids subjective errors caused by manual measurement by doctors and improves the measurement stability and consistency of key indicators such as inferior vena cava diameter, tricuspid regurgitation velocity, and TAPSE.
[0020] By integrating multimodal deep learning, high-performance prognostic prediction was achieved. By combining structural features, non-invasive hemodynamic parameters, and clinical data, a deep learning-based prognostic prediction model was constructed, which has higher prediction accuracy for endpoint events such as death and readmission, and can realize early identification of high-risk patients.
[0021] It has good clinical applicability and deployment capabilities, is compatible with ultrasound equipment from different brands, and can be embedded in PACS systems to achieve real-time clinical analysis. The device hardware is simple and can be deployed on servers, workstations, or in the cloud, facilitating widespread application.
[0022] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0024] Figure 1 This is a flowchart of the cardiac function assessment and prognosis prediction method provided in the embodiments of the present invention; Figure 2 This is an overall system framework diagram provided in the embodiments of the present invention; Figure 3 This is a flowchart of the segmentation of key structures in echocardiography provided in an embodiment of the present invention; Figure 4 This is a structural diagram of the non-invasive regression model for cardiac hemodynamics provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the cardiac function assessment and prognosis prediction device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a cardiac function assessment and prognosis prediction device provided in an embodiment of the present invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.
[0026] See Figure 1 This invention provides a method for assessing cardiac function and predicting prognosis, such as... Figure 1 As shown, the method may include: S101: Acquire the patient's multimodal medical data and preprocess the multimodal medical data. The multimodal medical data includes at least multi-sectional echocardiographic images, Doppler flow spectrum images, and clinical data. Specifically, in this application embodiment, the multi-sectional echocardiographic images may include at least two of the following: apical two-chamber view, apical four-chamber view, right ventricular outflow tract view, parasternal left ventricular long-axis view, parasternal short-axis view, and inferior vena cava view. The Doppler flow spectrum images include at least two of the following: tricuspid regurgitation spectrum, pulmonary artery flow spectrum, mitral valve orifice anterior flow spectrum, tricuspid valve orifice anterior flow spectrum, and aortic valve anterior flow spectrum.
[0027] S102: Based on the preprocessed multi-section echocardiogram image data, a deep learning segmentation model is used to automatically segment key cardiac structures to obtain cardiac structural features, and cardiac morphological parameters are extracted based on the cardiac structural features and / or motion tracking is performed to extract cardiac temporal functional parameters; in specific implementation, the deep learning segmentation model in this application embodiment can be a 2D nnU-Net network with an attention mechanism, used to segment at least one of the right atrium, right ventricle, and inferior vena cava; the cardiac morphological parameters include at least one of the right ventricular end-diastolic area, right ventricular end-systolic area, tricuspid annulus systolic displacement, and inferior vena cava diameter.
[0028] Furthermore, it also includes using optical flow estimation methods based on the segmentation results to perform motion tracking of the cardiac structure in order to extract temporal functional parameters of the heart.
[0029] S103: Based on the preprocessed Doppler blood flow spectrum image, hemodynamic spectrum features are extracted; in specific implementation, the hemodynamic spectrum features in this application embodiment may include at least one of peak flow velocity, acceleration time, deceleration time, and velocity-time integral.
[0030] S104: By fusing the cardiac structural features, the hemodynamic spectral features, and the preprocessed clinical data, at least one cardiac hemodynamic parameter is non-invasively predicted using a multi-task regression deep learning model. Specifically, in this application embodiment, the multi-task regression deep learning model may employ a cross-view attention mechanism and a temporal attention mechanism to fuse multi-view echocardiogram features. The cross-view attention mechanism is used to align and fuse features from different ultrasound sections within the same cardiac phase. The temporal attention mechanism is used to model the dynamic changes of a complete cardiac cycle within the same ultrasound section.
[0031] Furthermore, the fused ultrasound structural features, the hemodynamic spectral features, and the clinical data are aligned to form a multimodal fusion feature; The multimodal fusion features are input into a shared fully connected network to learn common latent representations; Based on the aforementioned common potential characterization, different cardiac hemodynamic parameters are predicted through multiple independent regression branches; the cardiac hemodynamic parameters include at least two of the following: mean pulmonary artery pressure, pulmonary artery systolic pressure, right atrial pressure, right ventricular pressure, cardiac output, pulmonary vascular resistance, and pulmonary capillary wedge pressure.
[0032] S105: Based on the cardiac structural features, the cardiac hemodynamic parameters, and the clinical data, a prognostic prediction deep learning model is used to quantitatively assess the risk of future disease progression in patients and predict the risk of patients experiencing a preset endpoint event, which includes the risk of death, rehospitalization, and clinical deterioration in patients with pulmonary hypertension.
[0033] In a specific implementation, the embodiments of this application may provide that the prognostic prediction deep learning model is a time-to-event prediction model built on a deep neural network, which is used to output an individualized risk score and / or survival probability curve for patients who experience at least one endpoint event, such as death, rehospitalization, or deterioration of cardiac function, within a preset follow-up period.
[0034] Furthermore, the follow-up time axis is discretized into several adjacent time intervals, and multiple output nodes corresponding one-to-one with the time intervals are set at the network output end to predict the risk of the end event occurring in each time interval. By multiplying the conditional risks of each time interval, an individualized survival function and the probability of occurrence of the corresponding death or composite endpoint event at any preset follow-up time point are obtained, thereby achieving joint prediction of patient survival and short-, medium- and long-term prognostic risks.
[0035] The cardiac function assessment and prognostic prediction method provided in this application realizes automatic segmentation and motion tracking of cardiac structures by intelligently analyzing multimodal echocardiography and clinical data; extraction and non-invasive regression prediction of hemodynamic parameters (such as mPAP, CO, RAP, PVR, PAWP); cardiac function classification (mild, moderate, and severe RVD); and prognostic risk prediction based on multi-source data fusion (death, readmission, and worsening of cardiac function, etc.).
[0036] The cardiac function assessment and prognosis prediction methods provided in the embodiments of this application will be described in detail below.
[0037] To implement the method provided in this application, such as Figure 2 As shown, the overall network includes: a data acquisition and preprocessing module, an ultrasound structure segmentation and motion tracking module, a Doppler spectrum feature extraction module, a non-invasive assessment module for cardiac function indicators, and a prognostic prediction module for PAH-RVD patients.
[0038] Step 1: Data Acquisition and Preprocessing Module.
[0039] This module is primarily used to acquire multi-source patient data. During system operation, the data acquisition module first acquires multi-sectional echocardiographic images of the patient, including standard cardiac function-related views such as A2C, A4C, RVOT, PLAX, PSAX, and IVC. Simultaneously, it acquires blood flow spectrograms such as tricuspid regurgitation spectrum and pulmonary artery blood flow spectrum, M-mode images, demographic information, and other clinical data. The acquired data includes: (1) Ultrasound dynamic image sequence: A4C, RVOT, PLAX, PSAX, IVC, right ventricular inflow tract and other ultrasound sections, each with no less than 3 complete cardiac cycles; (2) Blood flow spectrum images: tricuspid regurgitation spectrum, pulmonary artery blood flow spectrum, mitral valve orifice anterior blood flow spectrum, tricuspid valve orifice anterior blood flow spectrum and aortic valve anterior blood flow spectrum, etc., including peak flow velocity and waveform full cycle; (3) Demographic information and other clinical data.
[0040] Secondly, this module performs the following preprocessing on the echocardiogram: (1) Scale / resolution standardization: In order to improve the applicability of the model under different brands and different probe settings, the present invention further standardizes the resolution and physical scale of all input images (e.g., uniformly sampled to a preset pixel spacing) to reduce the impact of device differences.
[0041] (2) ROI extraction: By detecting the ultrasonic fan-shaped boundary, non-imaging areas such as black borders, scales, text annotations, and electrocardiogram curves outside the image are cropped and masked, leaving only the fan-shaped imaging area where the real myocardium and blood pool are located; (3) Adaptive contrast enhancement: The gray values of the pixels in the ROI are normalized and mapped to the standard range of [0,255]. Then, histogram equalization or adaptive histogram equalization is used to enhance the contrast of the image so that the echoes of the heart chamber, myocardium and blood flow are more evenly distributed in the overall gray range. (4) Noise reduction filtering: The present invention employs noise reduction strategies such as median filtering, bilateral filtering or anisotropic diffusion filtering in pixels within the ROI to suppress local small-scale noise; (5) Cardiac cycle standardization: Based on the real-time acquisition of electrocardiograms by echocardiography, equidistant frame sequences covering at least one complete cardiac cycle are extracted from the standardized sequence to construct video segment samples of uniform length.
[0042] Step 2: Ultrasonic structural segmentation and motion tracking module.
[0043] This module is primarily responsible for automatically acquiring cardiac structural regions and automatically measuring cardiac morphological parameters. The cardiac region is segmented using ultrasound structural segmentation and motion tracking modules, extracting cardiac structural and temporal motion features, and automatically measuring cardiac morphological parameters such as left / right ventricular end-diastolic area, left / right ventricular end-systolic area, tricuspid annulus systolic displacement, mitral annulus systolic displacement, inferior vena cava diameter, and right myocardial thickness. Figure 3 As shown, the implementation of this step specifically includes: (1) Segmentation of Key Structures in Echocardiography: This invention uses 2D nnU-Net as the basic framework for structural segmentation, and introduces an attention mechanism on the basis of the 2D-Unet network. Additive attention is used to incorporate information from adjacent ultrasound images, thereby improving segmentation accuracy. For each view of each patient, a preprocessed image is input into the model. Segmentation targets include: left atrium, left ventricle, right atrium, right ventricle, pulmonary artery, inferior vena cava, and other structures. A schematic diagram of the algorithm is shown below. Figure 2 As shown.
[0044] (2) Automatic measurement of cardiac morphological parameters: Morphological parameters are obtained based on the structural segmentation results. The main cardiac morphological parameters include: ascending aortic diameter, aortic root diameter, left atrial anteroposterior diameter, right ventricular anteroposterior diameter, interventricular septum thickness, left ventricular posterior wall thickness, left ventricular end-diastolic diameter, left ventricular end-systolic diameter, main pulmonary artery diameter, inferior vena cava diameter, right atrial superior-inferior diameter, right atrial left-right diameter, right ventricular transverse diameter, right ventricular wall thickness, right ventricular end-diastolic area, and right ventricular end-systolic area.
[0045] (3) Cardiac motion tracking: Based on the segmented myocardial and valve annulus regions, area and contour integrals are calculated on the segmentation results of each frame to obtain curves showing the changes in the area and estimated volume of the left and right ventricles over time. This is used to characterize the ventricular systolic-diastolic dynamics within one or more cardiac cycles. Furthermore, temporal functional parameters such as end-diastolic area / volume, end-systolic area / volume, and ejection fraction can be extracted. On the other hand, optical flow estimation modeling is used to estimate the pixel-level displacement field between adjacent frames.
[0046] Step 3: Doppler spectral feature extraction module.
[0047] The Doppler feature extraction module performs envelope detection from the blood flow spectrum and automatically extracts key hemodynamic features such as TR, Vmax, AT, DT, and VTI. This module is used to automatically detect the envelope and identify feature points in Doppler spectra, generating key hemodynamic features including Vmax, acceleration time (AT), deceleration time (DT), and velocity-time integral (VTI). The Doppler spectra processed by this module mainly include: tricuspid regurgitation spectrum, pulmonary artery blood flow spectrum, mitral valve anterior blood flow spectrum, tricuspid valve orifice anterior blood flow spectrum, and aortic valve anterior blood flow spectrum.
[0048] Step 4: Non-invasive assessment module of cardiac function indicators This module is the core of the invention. It uses the fused image features, Doppler features and clinical data features to construct a multi-output regression model to quantify hemodynamic parameters such as mPAP, RAP, CO, PVR, and PAWP.
[0049] The following is combined Figure 4 This document provides a detailed description of the non-invasive cardiac function assessment module of the present invention. Based on the aforementioned ultrasound structural segmentation and temporal modeling, and spectral envelope extraction, this module uniformly encodes and fuses multi-view echocardiographic features, clinical data, and Doppler spectral features to establish a deep neural network for regression prediction of cardiac function indicators. The main cardiac function indicators include pulmonary artery systolic pressure (PASP), mean pulmonary artery pressure (mPAP), pulmonary capillary wedge pressure (PAWP), pulmonary vascular resistance (PVR), right ventricular output (CO), and right atrial pressure (RAP).
[0050] A complete cardiac structure-dynamics representation was established using cross-view attention and temporal self-attention mechanisms, and a multi-task regression network was employed to non-invasively estimate cardiac function indices. The model utilized supervised learning, constructing supervised learning samples by pairing non-invasive ultrasound features with invasive measurements such as mPAP, RAP, CO, PVR, and PAWP obtained from right heart catheterization (RHC), which were then used to train the multi-task regression network.
[0051] To ensure that different view planes align and complement each other within the same cardiac phase, this invention constructs a multi-head self-attention mechanism with the view as the sequence dimension at each cardiac phase. Specifically, for a fixed phase... Collect the features of all views at that moment: Using views as the sequence dimension, construct cross-view multi-head self-attention:
[0052] The temporal attention mechanism unit, to model the dynamic changes in cardiac structure and motion within a complete cardiac cycle, constructs a self-attention mechanism along the time dimension within each view. Specifically, for a fixed view... Collect cross-view fusion features of all phases within a canonical cardiac cycle: In terms of time phase For the sequence dimension, construct a temporal multi-head self-attention mechanism:
[0053] The multimodal feature fusion unit integrates echocardiographic features, Doppler spectral features, and clinical data to construct a unified cardiac function feature vector. Specifically, firstly, it fuses echocardiographic features, Doppler spectral features, and clinical data to construct a unified cardiac function feature vector. and Echocardiographic features are obtained by splicing them along the feature dimension. Secondly, in the view dimension, The ultrasonic structure-dynamic eigenvectors are obtained by splicing them together. Ultimately, Compared with the blood flow Doppler feature vector obtained in step three Structured vector of clinical data Concatenate along the feature dimension:
[0054] The multi-task regression unit employs a multi-task regression framework to perform joint non-invasive prediction of multiple cardiac function and hemodynamic parameters. Specifically, firstly, the aforementioned fused features are input into a 2-3 layer shared fully connected network to learn common latent representations related to all tasks; secondly, for different prediction targets (e.g., mPAP, PASP, RAP, CO, PVR, etc.), respective regression branches are set, and physical consistency constraints between indicators are introduced to ensure consistency between prediction results and invasive measurement results.
[0055] Step 5: PAH-RVD patient prognosis prediction module.
[0056] The PAH-RVD patient prognosis prediction module integrates all the above features and uses a multimodal deep learning network to predict time-varying endpoint events (death, readmission, and deterioration of cardiac function) and output prognostic stratification results.
[0057] Based on the aforementioned ultrasound structure-dynamics modeling, Doppler spectral feature extraction, and hemodynamic multi-task regression, this module constructs a multimodal deep learning model to quantitatively assess the risk of future disease progression in patients, enabling non-invasive prediction of mortality risk, readmission risk, and clinical deterioration risk in patients with pulmonary arterial hypertension (PAH).
[0058] The prognostic prediction module is based on a deep neural network to build a time-to-event prediction model. It takes cardiac structural-dynamic characteristics, non-invasively estimated hemodynamic parameters and clinical data as inputs, and outputs a risk score and individualized survival probability of death, readmission or deterioration of cardiac function within a preset follow-up period.
[0059] Specifically, this invention first combines the cardiac morphological parameters (such as left / right ventricular end-diastolic area, left / right ventricular end-systolic area, tricuspid annulus systolic displacement, mitral annulus systolic displacement, inferior vena cava diameter, right myocardial thickness, etc.) extracted by the aforementioned ultrasound structural segmentation and motion tracking module, and the hemodynamic parameters such as mPAP, PASP, RAP, PAWP, PVR, and CO output by the non-invasive assessment module of cardiac function indicators, with the patient's basic clinical data, laboratory test results, medication information, etc., to form a multimodal feature vector; then, this multimodal feature is input into the deep neural network in the prognostic prediction module for time-to-event modeling.
[0060] To address the prevalent censoring in follow-up data and the non-equidistant intervals between different follow-up time points, this invention discretizes the follow-up time axis into several adjacent time intervals (e.g., divided into 6-month intervals). Multiple output nodes, each corresponding to a time interval, are set at the network output end to predict the risk (discrete hazard function) of the conditional endpoint event occurring within each time interval. By multiplying the conditional risks of each time interval, an individualized survival function at any preset follow-up time point can be obtained. By combining the probability of occurrence of corresponding death or composite endpoint events, a combined prediction of patient survival and short-, medium-, and long-term prognostic risks can be achieved.
[0061] Furthermore, based on the above time-to-event prediction results, the system can generate individualized risk scores, survival probability curves during the follow-up period, and event occurrence probabilities at key time points (such as 1 year and 3 years) for each PAH-RVD patient. This can be used to assist clinicians in risk stratification, optimize medication intensity and follow-up strategies, and improve the ability to assess the medium- and long-term prognosis of patients with pulmonary hypertension-related heart failure.
[0062] In summary, the cardiac function assessment and prognosis prediction method provided in this application achieves non-invasive cardiac function assessment. By using a deep learning model to perform structural analysis and hemodynamic parameter prediction on ultrasound images, it can partially replace right heart catheterization, enabling the acquisition of hemodynamic information that can only be measured by traditional RHC without invasive means, thus reducing patient risk.
[0063] This invention improves the accuracy and repeatability of cardiac structural parameter measurements. By using an automatic segmentation and motion tracking model based on echocardiography, it avoids subjective errors caused by manual measurement by doctors and improves the measurement stability and consistency of key indicators such as inferior vena cava diameter, tricuspid regurgitation velocity, and TAPSE.
[0064] By integrating multimodal deep learning, high-performance prognostic prediction was achieved. By combining structural features, non-invasive hemodynamic parameters, and clinical data, a deep learning-based prognostic prediction model was constructed, which has higher prediction accuracy for endpoint events such as death and readmission, and can realize early identification of high-risk patients.
[0065] It has good clinical applicability and deployment capabilities, is compatible with ultrasound equipment from different brands, and can be embedded in PACS systems to achieve real-time clinical analysis. The device hardware is simple and can be deployed on servers, workstations, or in the cloud, facilitating widespread application.
[0066] See Figure 5 This application embodiment can also provide a cardiac function assessment and prognosis prediction device, such as... Figure 5 As shown, the apparatus for performing the above-described cardiac function assessment and prognostic prediction methods may include: The data acquisition and preprocessing unit 501 is used to acquire the patient's multimodal medical data and preprocess the multimodal medical data, which includes at least multi-section echocardiographic image data, Doppler blood flow spectrum images and clinical data. The ultrasound structure segmentation and motion tracking unit 502 is used to automatically segment key cardiac structures based on the preprocessed multi-section echocardiogram image data using a deep learning segmentation model to obtain cardiac structural features, and extract cardiac morphological parameters based on the cardiac structural features and / or perform motion tracking to extract cardiac temporal functional parameters. The Doppler spectral feature extraction unit 503 is used to extract hemodynamic spectral features based on the preprocessed Doppler blood flow spectral image. The non-invasive cardiac function index assessment unit 504 is used to fuse the cardiac structural features, the hemodynamic spectral features and the preprocessed clinical data, and non-invasively predict at least one cardiac hemodynamic parameter through a multi-task regression deep learning model. The patient prognosis prediction unit 505 is used to quantitatively assess the risk of future disease progression of patients based on the cardiac structural features, cardiac hemodynamic parameters and clinical data through a prognosis prediction deep learning model, and to predict the risk of patients experiencing preset endpoint events, including the risk of death, rehospitalization and clinical deterioration in patients with pulmonary hypertension.
[0067] This application embodiment can also provide a cardiac function assessment and prognosis prediction device, the device including a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the above-described cardiac function assessment and prognostic prediction methods according to the instructions in the program code.
[0068] like Figure 6 As shown in the illustration, a cardiac function assessment and prognosis prediction device provided in this application embodiment may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.
[0069] In this embodiment, the processor 10 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.
[0070] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can perform operations in the embodiments of the cardiac function assessment and prognosis prediction method.
[0071] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions: Acquire the patient's multimodal medical data and preprocess the multimodal medical data, which includes at least multi-sectional echocardiographic images, Doppler blood flow spectrum images, and clinical data; Based on the preprocessed multi-section echocardiogram image data, the key cardiac structures are automatically segmented using a deep learning segmentation model to obtain cardiac structural features. Based on the cardiac structural features, cardiac morphological parameters are extracted and / or motion tracking is performed to extract cardiac temporal functional parameters. Based on the preprocessed Doppler blood flow spectrum image, hemodynamic spectrum features are extracted; By integrating the cardiac structural features, the hemodynamic spectral features, and the preprocessed clinical data, at least one cardiac hemodynamic parameter is non-invasively predicted using a multi-task regression deep learning model. Based on the cardiac structural features, cardiac hemodynamic parameters, and clinical data, a prognostic prediction deep learning model is used to quantitatively assess the risk of future disease progression in patients and predict the risk of patients experiencing preset endpoint events, including the risk of death, readmission, and clinical deterioration in patients with pulmonary hypertension.
[0072] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.
[0073] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.
[0074] Communication interface 12 can be an interface for a communication model, used to connect with other devices or systems.
[0075] Of course, it should be noted that, Figure 6 The structure shown does not constitute a limitation on the cardiac function assessment and prognosis prediction device in the embodiments of this application. In actual applications, cardiac function assessment and prognosis prediction devices may include more than Figure 6 More or fewer components as shown, or combinations of certain components.
[0076] This application embodiment may also provide a computer-readable storage medium for storing program code for performing the steps of the above-described cardiac function assessment and prognostic prediction methods.
[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0078] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the cardiac function assessment and prognostic prediction methods described in various embodiments or certain parts of the embodiments of this application.
[0079] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A method of cardiac function assessment and prognosis prediction, characterized by, The method includes: Acquire the patient's multimodal medical data and preprocess the multimodal medical data, which includes at least multi-sectional echocardiographic images, Doppler blood flow spectrum images, and clinical data; Based on the preprocessed multi-section echocardiogram image data, the key cardiac structures are automatically segmented using a deep learning segmentation model to obtain cardiac structural features. Based on the cardiac structural features, cardiac morphological parameters are extracted and / or motion tracking is performed to extract cardiac temporal functional parameters. Based on the preprocessed Doppler blood flow spectrum image, hemodynamic spectrum features are extracted. By integrating the cardiac structural features, the hemodynamic spectral features, and the preprocessed clinical data, at least one cardiac hemodynamic parameter is non-invasively predicted using a multi-task regression deep learning model. Based on the cardiac structural features, cardiac hemodynamic parameters, and clinical data, a prognostic prediction deep learning model is used to quantitatively assess the risk of future disease progression in patients and predict the risk of patients experiencing preset endpoint events, including the risk of death, readmission, and clinical deterioration in patients with pulmonary hypertension.
2. The cardiac function evaluation and prognosis prediction method according to claim 1, characterized in that, The multi-plane echocardiographic image data includes at least two of the following: apical two-chamber view, apical four-chamber view, right ventricular outflow tract view, parasternal left ventricular long-axis view, parasternal short-axis view, and inferior vena cava view; the Doppler blood flow spectrum image includes at least two of the following: tricuspid regurgitation spectrum, pulmonary artery blood flow spectrum, mitral valve orifice anterior blood flow spectrum, tricuspid valve orifice anterior blood flow spectrum, and aortic valve anterior blood flow spectrum.
3. The cardiac function evaluation and prognosis prediction method according to claim 1, characterized by, The deep learning segmentation model is a 2D nnU-Net network with an attention mechanism, used to segment at least one of the following structures: left ventricle, left atrium, right atrium, right ventricle, and inferior vena cava; the cardiac morphological parameters include at least one of the following: left / right ventricular end-diastolic area, left / right ventricular end-systolic area, tricuspid annulus systolic displacement, mitral annulus systolic displacement, inferior vena cava diameter, and right myocardial thickness.
4. The cardiac function evaluation and prognosis prediction method according to claim 1, characterized by, It also includes using optical flow estimation based on the segmentation results to perform motion tracking of the cardiac structure in order to extract cardiac temporal functional parameters.
5. The cardiac function evaluation and prognosis prediction method according to claim 1, characterized by, The hemodynamic spectral characteristics include at least one of peak flow velocity, acceleration time, deceleration time, and velocity-time integral.
6. The cardiac function evaluation and prognosis prediction method according to claim 1, characterized by, The multi-task regression deep learning model employs cross-view attention and temporal attention mechanisms to fuse features from multiple views of echocardiography. The cross-view attention mechanism is used to align and fuse features from different ultrasound sections within the same cardiac phase. The temporal attention mechanism is used to model the dynamic changes of the complete cardiac cycle within the same ultrasound section.
7. The cardiac function evaluation and prognosis prediction method according to claim 6, characterized in that, The fused ultrasound structural features, the hemodynamic spectral features, and the clinical data are aligned to form a multimodal fused feature; The multimodal fusion features are input into a shared fully connected network to learn common latent representations; Based on the aforementioned common potential characterization, different cardiac hemodynamic parameters are predicted through multiple independent regression branches; the cardiac hemodynamic parameters include at least two of the following: mean pulmonary artery pressure, pulmonary artery systolic pressure, right atrial pressure, right ventricular pressure, cardiac output, pulmonary vascular resistance, and pulmonary capillary wedge pressure.
8. The method for assessing cardiac function and predicting prognosis according to claim 1, characterized in that, The prognostic prediction deep learning model is a time-to-event prediction model built on a deep neural network, used to output an individualized risk score and / or survival probability curve for patients who experience at least one endpoint event, such as death, rehospitalization, or deterioration of cardiac function, within a preset follow-up period.
9. The method for assessing cardiac function and predicting prognosis according to claim 8, characterized in that, The follow-up time axis is discretized into several adjacent time intervals. Multiple output nodes corresponding to each time interval are set at the network output end to predict the risk of the terminal event occurring in each time interval. By multiplying the conditional risks of each time interval, an individualized survival function and the probability of occurrence of the corresponding death or composite endpoint event at any preset follow-up time point are obtained, thereby achieving joint prediction of patient survival and short-, medium- and long-term prognostic risks.
10. A device for assessing cardiac function and predicting prognosis, characterized in that, The apparatus for performing the cardiac function assessment and prognostic prediction method according to any one of claims 1-9, the apparatus comprising: The data acquisition and preprocessing unit is used to acquire the patient's multimodal medical data and preprocess the multimodal medical data, which includes at least multi-sectional echocardiographic images, Doppler blood flow spectrum images, and clinical data. The ultrasound structure segmentation and motion tracking unit is used to automatically segment key cardiac structures based on the preprocessed multi-section echocardiogram image data using a deep learning segmentation model to obtain cardiac structural features, and extract cardiac morphological parameters based on the cardiac structural features and / or perform motion tracking to extract cardiac temporal functional parameters. The Doppler spectral feature extraction unit is used to extract hemodynamic spectral features based on the preprocessed Doppler blood flow spectral image. The non-invasive cardiac function index assessment unit is used to fuse the cardiac structural features, the hemodynamic spectral features and the preprocessed clinical data, and non-invasively predict at least one cardiac hemodynamic parameter through a multi-task regression deep learning model. The patient prognosis prediction unit is used to quantitatively assess the risk of future disease progression of patients based on the cardiac structural features, cardiac hemodynamic parameters and clinical data, using a prognosis prediction deep learning model, and to predict the risk of patients experiencing preset endpoint events, including the risk of death, rehospitalization and clinical deterioration in patients with pulmonary hypertension.