Cardio-cerebrovascular disease early warning system based on multi-modal information

Abstract

The application discloses a kind of cardiovascular and cerebrovascular disease early warning system based on multi-modal information, including data acquisition layer, intelligent analysis layer and intervention response layer.Data acquisition layer collects image data, physiological signal data and structured clinical data.Intelligent analysis layer includes modal exclusive feature extraction module, cross-modal alignment fusion module and risk stratification classification module;Modal exclusive feature extraction module carries out feature extraction to each modal data respectively;Cross-modal alignment fusion module projects each modal feature to the same latent space, calculates cross-modal attention weight by multi-head attention mechanism and carries out weighted fusion;Risk stratification classification module carries out risk stratification based on fusion feature, and outputs risk grade and confidence score.Intervention response layer generates and pushes early warning information and individualized intervention scheme according to risk grade.The application can improve the accuracy and timeliness of cardiovascular and cerebrovascular disease risk prediction.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical technology, specifically to an early warning system for cardiovascular and cerebrovascular diseases based on multimodal information. Background Technology

[0002] Traditional methods for assessing and warning of cardiovascular and cerebrovascular disease risk primarily rely on single-modality medical data, such as electrocardiograms, echocardiograms, blood pressure measurements, or CT / MRI images. However, the occurrence and development of cardiovascular and cerebrovascular diseases involve the interaction of multiple factors, including genetics, environment, and lifestyle. Single-modality data struggles to comprehensively capture the complexity and dynamic changes in disease progression, often resulting in insufficient accuracy and timeliness in early warnings. Studies have shown that conventional risk scoring models (such as the Framingham Risk Score and ASCVD risk assessment models) do not fully integrate multimodal cardiovascular data and have limited generalization ability across different populations.

[0003] In recent years, with the development of artificial intelligence and deep learning technologies, early warning methods for cardiovascular and cerebrovascular diseases based on multimodal data fusion have received widespread attention. Fundus retinal images serve as a window for non-invasive observation of vascular networks, and multiple studies have confirmed a significant association between retinal vascular parameters (such as arteriovenous ratio and vascular tortuosity) and cardiovascular disease risk. Cardiac CT / MRI can assess the nature of coronary plaques, myocardial function, and the degree of fibrosis. Physiological signals (dynamic electrocardiogram, blood oxygenation, and sleep monitoring) can capture transient arrhythmias and hidden nocturnal risks. Structured clinical data provides demographic, biochemical, and medical history information.

[0004] However, existing technologies still have the following shortcomings: First, most multimodal early warning systems lack targeted solutions for the differences in sampling rates and data formats among various modalities, and simple splicing or averaging easily introduces noise; Second, the interpretability of existing models is insufficient, and the "black box" nature of deep learning models limits the trust and acceptance of clinicians; Third, most studies are based on retrospective data validation and lack a closed-loop design for the entire process from pre-hospital monitoring and in-hospital diagnosis to post-hospital management, and the maturity of clinical translation needs to be improved; Fourth, key challenges such as cross-modal data heterogeneity, modality missing, and cross-center generalization ability have not yet been systematically resolved. Summary of the Invention

[0005] The purpose of this invention is to provide a cardiovascular and cerebrovascular disease early warning system based on multimodal information to solve the problems in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a cardiovascular and cerebrovascular disease early warning system based on multimodal information, comprising: The data acquisition layer is configured to acquire image data, physiological signal data, and structured clinical data of the target object. The image data includes fundus images, cardiac CT and / or cardiac MRI. The physiological signal data includes at least one of dynamic electrocardiogram signals, pulse wave signals, blood oxygen saturation signals, and polysomnography signals. The image data, physiological signal data, and structured clinical data acquired by the data acquisition layer are collectively referred to as modal data. The intelligent analysis layer, communicatively connected to the data acquisition layer, includes a modality-specific feature extraction module, a cross-modal alignment and fusion module, and a risk stratification and classification module. The modality-specific feature extraction module extracts features from each modality's data and outputs a feature vector corresponding to each modality's data. The cross-modal alignment and fusion module projects the feature vectors of each modality onto the same latent space, calculates cross-modal attention weights using a multi-head attention mechanism, and performs weighted fusion of the feature vectors based on these cross-modal attention weights to output a fused feature vector. The risk stratification and classification module performs risk stratification based on the fused feature vector and outputs the risk level and confidence score. The intervention response layer is communicatively connected to the intelligent analysis layer and is configured to generate and push early warning information and personalized intervention plans based on the risk level.

[0007] Furthermore, the modality-specific feature extraction module includes: The fundus image processing submodule is configured to segment retinal vessels based on the U-Net++ model, transform the vascular network into a graph structure, update features through a graph neural network, and output a graph-level feature vector of the fundus image. The cardiac CT image processing submodule is configured to use the kernel density fuzzy C-means algorithm for myocardial segmentation, extract the three-dimensional structure of the coronary artery tree through the region growing method and active contour model, extract multi-scale texture and frequency domain features using the squeeze-excitation network and linear frequency modulation transform, identify high-risk plaque features and calculate the pericoronary fat attenuation index, and output cardiac CT feature vectors. The cardiac MRI processing submodule is configured to segment the left ventricular boundary based on deep learning, calculate the ejection fraction and ventricular volume, calculate myocardial strain parameters through feature tracking technology, evaluate the delayed gadolinium enhancement region, and output cardiac MRI feature vectors. The physiological signal processing submodule is configured to preprocess and extract specific features from the electrocardiogram (ECG), airflow, and blood oxygenation signals in polysomnography signals, and output cardiopulmonary coupling feature vectors; perform denoising and QRS detection on dynamic ECG signals, identify abnormal ECG types based on deep convolutional neural networks, and output ECG feature vectors; and extract time-domain and frequency-domain features from pulse wave signals and output hemodynamic parameter vectors. The clinical feature encoding module is configured to encode categorical variables, standardize numerical variables, and handle missing values ​​in structured clinical data, and output clinical feature vectors.

[0008] Furthermore, the cross-modal alignment and fusion module includes a multi-level fusion unit, which includes: The first-level fusion unit is configured to perform feature concatenation or 3D convolution fusion on multiple sequence data within the same modality, and output comprehensive features of the same modality. The second-level fusion unit is configured to perform attention-weighted concatenation of feature vectors from different modalities through a multi-head attention mechanism. The cross-modal attention weights reflect the contribution of each modality to the current prediction task, are dynamically adjusted, and achieve interpretability. The three-level fusion unit is configured to use soft voting or stacking ensemble learning methods to perform decision-level fusion of the outputs of each modality-independent classifier, and output the final comprehensive risk score.

[0009] Furthermore, the risk stratification and classification module includes a contrastive learning unit, a stratified decision-making unit, and an output unit. The contrastive learning unit is configured to perform risk feature comparison based on the feature space obtained through contrastive learning training. The feature space is constructed during the model training phase by bringing the feature vectors of samples with the same risk level closer together and pushing the feature vectors of samples with different risk levels further apart, thereby enhancing the inter-class discrimination in the feature space. The stratified decision-making unit is configured to first perform binary classification to determine whether it is high-risk. If it is determined to be high-risk, it is further classified into extremely high-risk or high-risk through fine-grained classification. If it is determined to be non-high-risk, it is further classified into medium-risk or low-risk through fine-grained classification. The output unit is configured to output the risk level, confidence score, and ranking of the contribution of each modality.

[0010] Furthermore, the risk levels include extremely high risk, high risk, intermediate risk, and low risk. The criteria for extremely high risk are: the fused feature vector is determined to be high risk by the risk stratification and classification module, and the confidence score is greater than a preset confidence threshold. Simultaneously, the system detects preset critical signs based on image data or physiological signal data and triggers an immediate emergency response. The criteria for high risk are: the fused feature vector is determined to be high risk by the risk stratification and classification module, and the confidence score is greater than a preset confidence threshold. Enhanced monitoring and a specialist consultation recommendation within one week are triggered. The criteria for intermediate risk are: at least two modalities of the feature vectors, after independent evaluation, indicate risk, and further examination recommendations are triggered. The criteria for low risk are: none of the features of any modality indicate significant risk, and routine health management recommendations are triggered.

[0011] Furthermore, the data acquisition layer includes: The in-hospital data acquisition unit is configured to import DICOM standard image data through the PACS system interface, import structured medical records and laboratory data through the HIS / EMR system interface, and acquire physiological signal data through bedside monitors and dynamic electrocardiogram recorders. The pre-hospital data acquisition unit is configured to monitor heart rate, dynamic electrocardiogram signals, blood pressure, blood oxygen saturation and polysomnography signals in real time through wearable devices, collect home physiological signal data through portable monitoring devices, and transmit the data to the cloud analysis platform via 5G / Wi-Fi / Bluetooth. The symptom-triggered acquisition unit is configured to automatically trigger a high-frequency data acquisition mode when abnormal physiological indicators are detected, thereby shortening the sampling interval.

[0012] Furthermore, the intervention response layer includes: The cloud-based decision support platform is configured to receive multimodal analysis results in real time via a message queue, perform cross-validation with reference to clinical guidelines, and automatically generate personalized intervention plans based on risk levels and specific risk factors. The proactive warning and response module is configured to call the on-duty physician for manual review when a high-risk signal is detected and the confidence level exceeds a preset threshold. After confirmation, the patient and emergency contact will be notified simultaneously, and the warning information will be pushed to the community health service center. In critical situations, the regional emergency command center will be automatically notified. The resident-side service module is configured to provide access to cardiac health records, personalized health reminders, home data uploads, and online consultation channels, and supports family member binding and early warning information sharing. The primary care empowerment module is configured to automatically identify and record individuals with hypertension, hyperglycemia, hyperlipidemia, and overweight / obesity, recommend primary care observation or referral to higher-level hospitals based on the warning level, provide a real-time overview of the status of high-risk patients within its jurisdiction, automatically generate follow-up plans and provide reminders when they are due.

[0013] Furthermore, it also includes an interpretability module and a closed-loop management module; the interpretability module is configured as follows: By analyzing the SHAP values, the overall contribution of each mode and feature to the model prediction is analyzed, and a global interpretation is generated. The LIME algorithm is used to generate local interpretations for a single prediction, and key imaging regions, key signal segments and key clinical indicators are marked with heat maps. It provides counterfactual reasoning to simulate the change in risk level after improvement of target clinical indicators, which include at least one of low-density lipoprotein cholesterol, systolic blood pressure, glycated hemoglobin, body mass index, and pericoronary fat attenuation index. The closed-loop management module is configured as follows: After the intelligent analysis layer issues an AI warning, it records the results of the clinician's manual review and confirmation within a preset time. Record the treatment outcomes, final diagnosis, and result data of the clinical team in implementing the intervention plan; The resulting data is automatically fed back to the model in the intelligent analysis layer for incremental learning, continuously optimizing model performance.

[0014] Furthermore, the data acquisition layer also includes a data standardization and quality control unit, configured as follows: Image data is uniformly converted to DICOM or NIfTI format, physiological signal data is uniformly converted to EDF or CSV format, and structured clinical data follows the HL7 FHIR standard; Perform deduplication, missing value imputation using multiple imputation methods, and outlier detection and labeling; The verification results are: image signal-to-noise ratio greater than or equal to 20dB, effective signal duration greater than or equal to 6 hours, and data integrity rate greater than or equal to 95%. The data is de-identified and anonymized, and the transmission uses TLS 1.3 encryption, while the storage uses AES-256 encryption.

[0015] A method for early warning of cardiovascular and cerebrovascular diseases based on multimodal information, wherein the early warning system for cardiovascular and cerebrovascular diseases based on multimodal information as described above includes: Collect imaging data, physiological signal data, and structured clinical data of the target subjects; Feature extraction is performed on each modal data separately, and the corresponding modal feature vectors are output. The modal feature vectors are projected onto the same latent space, cross-modal attention weights are calculated using a multi-head attention mechanism, and the modal feature vectors are weighted and fused based on the cross-modal attention weights to output a fused feature vector. Risk stratification is performed based on the fused feature vector, and the risk level and confidence score are output. Early warning information is generated and pushed out based on the risk level.

[0016] After adopting the above technical solution, the present invention has at least the following beneficial effects: 1. Enhanced Early Warning Performance: This invention integrates four modalities—fundus images, cardiac CT / MRI, physiological signals (ECG, pulse wave, blood oxygenation, sleep monitoring), and structured clinical data—to construct a modality-specific feature extraction module. This module provides customized processing for the data characteristics of each modality, effectively addressing the issue of cross-modal data heterogeneity. Multimodal features are projected onto the same latent space via a multi-head attention mechanism for dynamic weighted fusion. The cross-modal attention weights reflect the contribution of each modality to the current prediction task, avoiding the noise introduced by simple splicing or averaging fusion. This improves the sensitivity and specificity indicators in cardiovascular and cerebrovascular event prediction, enhancing the accuracy of early warning for cardiovascular and cerebrovascular diseases.

[0017] 2. Closed-Loop Management Throughout the Entire Process: This invention achieves closed-loop management throughout the entire disease process, from pre-hospital intelligent monitoring terminal and in-hospital data interface collection, multimodal AI fusion analysis, proactive early warning push, to clinical intervention response and feedback optimization. In the pre-hospital stage, wearable devices and portable monitoring terminals collect home physiological signals in real time, filling the gap in pre-hospital early warning in traditional treatment models. In the in-hospital stage, PACS / HIS system integration automatically imports images and structured data, achieving multidisciplinary information integration. In the post-hospital stage, a resident-facing mini-program provides health record inquiries, medication reminders, and follow-up management. This five-step closed-loop mechanism of early warning, response, treatment, feedback, and optimization automatically feeds clinical outcome data back to the model for incremental learning, enabling continuous system optimization and iteration.

[0018] 3. High level of data security and standardization: This invention establishes unified standards for multimodal data acquisition and quality control. Image data is uniformly converted to DICOM / NIfTI format, signals are uniformly converted to EDF / CSV format, and structured clinical data follows the HL7 FHIR standard, ensuring interoperability and reusability of multi-source data. The entire data chain employs de-identification and anonymization, TLS 1.3 transmission encryption, and AES-256 storage encryption, effectively protecting patient privacy and ensuring medical data compliance.

[0019] 4. High Response Timeliness: Pre-hospital monitoring data is transmitted back to the cloud analysis platform in real time via 5G / Wi-Fi, enabling timely push of early warning information and multi-level linkage response, thus gaining valuable golden time window for the treatment of cardiovascular and cerebrovascular diseases. Attached Figure Description

[0020] Figure 1 This is a flowchart of the overall system architecture of an embodiment of the present invention; Figure 2 This is a detailed flowchart of multimodal data processing and fusion according to an embodiment of the present invention; Figure 3 This is the early warning response and closed-loop management process of this invention embodiment; Figure 4 This is an embodiment of the fundus image processing pipeline of the present invention; Figure 5 This is the cross-modal multi-head attention fusion mechanism in this embodiment of the invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown in the figure, the cardiovascular and cerebrovascular disease early warning system based on multimodal information provided in this embodiment of the invention includes a data acquisition layer, an intelligent analysis layer, and an intervention response layer.

[0023] The data acquisition layer is configured to acquire imaging data, physiological signal data, and structured clinical data of the target object. Imaging data includes fundus images, cardiac CT scans, and cardiac MRI scans. Physiological signal data includes dynamic electrocardiogram signals, pulse wave signals, blood oxygen saturation signals, and polysomnography signals. Structured clinical data includes demographic information, physical examination indicators, laboratory biochemical indicators, past medical history and medication history, and lifestyle information. In this application, imaging data, physiological signal data, and structured clinical data are respectively used as different data modalities, constituting the aforementioned modal data.

[0024] The intelligent analysis layer communicates with the data acquisition layer and includes a modality-specific feature extraction module, a cross-modal alignment and fusion module, and a risk stratification and classification module. The modality-specific feature extraction module extracts features from each modality's data separately, outputting a feature vector corresponding to each modality. The cross-modal alignment and fusion module projects the feature vectors from each modality onto the same latent space, calculates cross-modal attention weights using a multi-head attention mechanism, and performs weighted fusion of the feature vectors based on these weights, outputting a fused feature vector. The risk stratification and classification module performs risk stratification based on the fused feature vector, outputting the risk level and confidence score.

[0025] The intervention response layer and the intelligent analysis layer are communicatively connected and configured to generate and push early warning information and personalized intervention plans based on the risk level. The early warning information and personalized intervention plans serve as risk assessment data output, provided to clinicians for reference when making diagnostic and treatment decisions based on clinical circumstances, and do not constitute a diagnosis or determination of a treatment plan.

[0026] like Figure 2As shown, the data acquisition layer includes an in-hospital data acquisition unit, a pre-hospital data acquisition unit, and a symptom-triggered acquisition unit. The in-hospital data acquisition unit imports DICOM standard image data through the PACS system interface as image data; it imports structured medical records and laboratory data through the HIS and EMR system interfaces as structured clinical data; and it acquires physiological signal data through bedside monitors and Holter monitors. Specifically, the in-hospital data acquisition unit interfaces with fundus cameras, CT equipment, and MRI equipment deployed within the hospital. The image data output by these devices is transmitted to the in-hospital PACS server via the DICOM protocol. This invention retrieves and obtains image sequences of a specified patient from the PACS server through a RESTful API interface based on the DICOM web standard. For cardiac CT data, coronary artery CTA scan sequences are acquired, with scan parameters set to a tube voltage of 120kV, tube current automatically adjusted according to the patient's body mass index, slice thickness of 0.625mm, reconstruction matrix of 512×512, and coverage from the tracheal bifurcation to the diaphragm. For cardiac MRI data, steady-state free precession sequences were acquired, including long-axis two-chamber, three-chamber, and four-chamber sequences, as well as short-axis cine sequences. Scanning parameters were set as follows: repetition time 3.4 ms, echo time 1.2 ms, flip angle 60 degrees, slice thickness 8 mm, slice spacing 2 mm, and temporal resolution 40 ms. For fundus images, non-mydriatic fundus cameras were used to acquire color fundus photographs centered on the optic disc at a 45-degree angle, with an image resolution of at least 2048 × 2048 pixels, output in JPEG or TIFF format. The above structured clinical data were uniformly organized and transmitted according to the HL7 FHIR standard, including patient age, sex, ethnicity, height, weight, body mass index, waist circumference, total cholesterol, triglycerides, LDL cholesterol, HDL cholesterol, fasting blood glucose, glycated hemoglobin, creatinine, uric acid, high-sensitivity C-reactive protein, history of hypertension, diabetes, hyperlipidemia, atrial fibrillation, stroke, smoking history, alcohol consumption history, and exercise habits. Physiological signals are acquired through continuous ECG, blood pressure, and blood oxygen saturation data obtained by bedside monitors, and 24-hour Holter monitoring signals are obtained through a 12-lead Holter monitor. The physiological signals acquired by the in-hospital data acquisition unit constitute the in-hospital physiological signal data, while the physiological signals acquired by the pre-hospital data acquisition unit constitute the pre-hospital physiological signal data.

[0027] The pre-hospital data acquisition unit monitors physiological signal data such as heart rate, dynamic electrocardiogram, blood pressure, blood oxygen saturation, pulse wave and polysomnography signals in real time through wearable devices, collects home physiological signal data through portable monitoring devices, and transmits the data to the cloud analysis platform via 5G network, Wi-Fi or Bluetooth.

[0028] The wearable device is a smartwatch or wristband that integrates photoplethysmography (PPG) and a single-lead ECG sensor. The sampling rates are set to 256Hz for the ECG channel, 125Hz for the pulse wave channel, and 50Hz for the accelerometer channel. The device is worn on the non-dominant wrist. During ECG measurement, the patient places the index finger of their other hand on the side electrode of the device to form a lead circuit. The measurement lasts 30 to 60 seconds. The results are transmitted to the patient's smartphone via the device's built-in Bluetooth module, and then encrypted and uploaded to a cloud platform via a 5G network through a mobile application.

[0029] Portable monitoring devices include a portable sleep monitor and a portable fundus camera. The portable sleep monitor is a simplified version of polysomnography, containing a single-lead ECG electrode, a nasal airflow pressure sensor, and a finger-clip pulse oximetry sensor. When used at home, patients attach the ECG electrode to the corresponding location on their chest, fix the nasal airflow pressure sensor to their upper lip, and wear the finger-clip pulse oximetry sensor on their index finger. The device records overnight data using an SD card or built-in memory, and automatically synchronizes the data to a cloud platform the following morning via USB or Wi-Fi. The portable fundus camera is handheld; patients or primary care medical personnel operate it by pointing it at the pupil to take a picture. The device has built-in autofocus and auto-exposure functions, and the image is uploaded to the cloud via Wi-Fi after shooting.

[0030] The symptom-triggered data acquisition unit automatically triggers a high-frequency data acquisition mode when abnormal physiological indicators are detected, shortening the sampling interval. Specifically, the system monitors the patient's heart rate variability and blood oxygen saturation trends in real time through a wearable device. When preset trigger conditions are detected, such as a sustained decrease in the ratio of low-frequency to high-frequency heart rate variability and a heart rate exceeding 20% ​​above the resting baseline, a drop in blood oxygen saturation exceeding 4 percentage points within 5 minutes, or frequent premature ventricular contractions exceeding 5 times per minute, the system automatically increases the data acquisition frequency to 2 to 4 times the normal level. The normal ECG sampling rate is 256 Hz, which increases to 512 Hz after triggering; the normal blood oxygen sampling rate is 10 Hz, which increases to 25 Hz after triggering. Simultaneously, the system activates the smartphone's microphone and accelerometer to begin recording the patient's voice signals and body movement information, providing supplementary data for subsequent cardiopulmonary function assessment and sleep status analysis. The high-frequency data acquisition mode continues until the abnormal signals subside and stabilize for more than 30 minutes, or until manually confirmed and terminated by a clinician.

[0031] The data acquisition layer also includes a data standardization and quality control unit. This unit uniformly converts image data to DICOM or NIfTI format, physiological signal data to EDF or CSV format, and organizes structured clinical data according to the HL7 FHIR standard. For image data, the data standardization and quality control unit first verifies the completeness and consistency of DICOM labels, checking the presence and reasonable values ​​of key labels such as patient identification, examination date, equipment model, and scanning parameters. For images that do not meet the standard, a format conversion tool is used to convert them to the standard DICOM format and supplement missing labels. For physiological signal data, this unit uniformly converts the original formats output from different devices to EDF format, ensuring that information such as sampling rate, bit width, and channel name of physiological signal data is completely recorded. For structured clinical data, this unit maps patient information to Patient resources, examination results to Observation resources, medical history to Condition resources, and medication to MedicationRequest resources according to the FHIR resource model.

[0032] The data standardization and quality control unit also performs deduplication and outlier detection, identifying and deleting duplicate records caused by multiple imports or interface retransmissions. For missing values ​​in the aforementioned structured clinical or physiological signal data, multiple imputation is used. Specifically, for missing values ​​of continuous variables, a chain equation-based multiple imputation algorithm uses observed values ​​of complete variables to build a predictive model, performs multiple random sampling to impute missing values, generates five complete imputation datasets, and takes the mean as the final imputed value. For missing values ​​of categorical variables, mode imputation or probabilistic imputation using a logistic regression model is employed. Outlier detection uses the interquartile range (ICM). Values ​​exceeding the mean plus or minus three standard deviations or exceeding the clinically reasonable range are marked as outliers and manually reviewed by clinical experts to determine whether to remove or correct them.

[0033] Regarding quality control indicators, data standardization and quality control units verify whether the image signal-to-noise ratio (SNR) is greater than or equal to 20dB. Images with an SNR below the standard are marked as low quality and prompted for re-acquisition. The effective signal duration is verified to be greater than or equal to 6 hours. For polysomnography signals, the remaining effective data duration is calculated after removing the first and last 15 minutes of unstable data. The data integrity rate is verified to be greater than or equal to 95%, i.e., the proportion of data successfully acquired and transmitted to the cloud for each modality out of the total amount that should be acquired. For data that does not meet the quality control standards, the system automatically triggers a re-acquisition notification or marks it as unavailable. Regarding data security, all data undergoes de-identification and anonymization processing at the acquisition end. Direct identifiers such as patient names, ID numbers, and contact information are replaced with unique identifiers generated internally by the system. After anonymization, the data is encrypted using the TLS 1.3 protocol during transmission to ensure confidentiality and integrity during public network transmission. The storage layer uses the AES-256 algorithm to encrypt database files and image files. The key is managed by a hardware security module, and access permissions are managed with fine-grained access control based on a role-based access control model.

[0034] The intelligent analysis layer includes a modality-specific feature extraction module. This module designs dedicated feature extraction pipelines for the characteristics of different modalities, ensuring that effective information from each modality is fully extracted and represented. The modality-specific feature extraction module includes a fundus image processing submodule, a cardiac CT image processing submodule, a cardiac MRI image processing submodule, a physiological signal processing submodule, and a clinical feature encoding module. These are used to extract specific features from fundus images, cardiac CT images, cardiac MRI images, physiological signal data, and structured clinical data, respectively. The feature vectors output by each module are used as feature inputs to the cross-modal alignment and fusion module for the corresponding modality.

[0035] like Figure 4As shown, the fundus image processing submodule adopts an architecture combining vessel segmentation and graph modeling based on U-Net++. First, the original color fundus image is preprocessed. The preprocessing process includes adaptive histogram equalization, denoising, optic disc localization, and cropping. Adaptive histogram equalization divides the image into several equally sized sub-blocks, and performs histogram equalization on each sub-block to improve local contrast and enhance the distinction between vessels and background. Denoising uses a non-local mean filtering algorithm, employing a weighted average of similar pixel blocks in the image to suppress noise while preserving vessel edge details. Optic disc localization uses a method based on morphological operations and Hough circle transform. First, the green channel image is contrast-enhanced, then a top-hat transform is used to highlight bright areas, and Hough circle transform is used to detect candidate optic disc regions, selecting the candidate region with the highest brightness as the optic disc center position. The cropped region is input into U-Net++ while maintaining its original resolution, or it is uniformly scaled according to model input requirements. After preprocessing, the U-Net++ deep learning model is used to perform end-to-end segmentation of the retinal vessels in the fundus image. The U-Net++ deep learning model, based on the standard U-Net structure, introduces nested skip connections and dense convolutional blocks to enhance the reuse and fusion of multi-scale features, thereby improving segmentation accuracy. The model input is a preprocessed color fundus image, and the output is a binary mask of blood vessels of the same size as the input image, where foreground pixels represent blood vessels and background pixels represent non-blood vessels. Preferably, the model is pre-trained using the DRIVE and STARE public datasets, and then fine-tuned using fundus images labeled by partner hospitals. The loss function is a weighted combination of Dice loss and binary cross-entropy loss. After obtaining the binary mask of blood vessels, the fundus image processing submodule transforms the blood vessel network into a graph structure.

[0036] The graph modeling method is as follows: Morphological skeletonization is performed on the binary mask of blood vessels to generate a single-pixel-wide skeleton map of the blood vessel centerline. Bifurcation points and endpoints are detected on the skeleton map. A bifurcation point is defined as a skeleton pixel with three or more blood vessel pixels in its eight-neighborhood, and an endpoint is defined as a skeleton pixel with one or more blood vessel pixels in its eight-neighborhood. Using bifurcation points and endpoints as graph nodes and blood vessel segments between nodes as graph edges, an undirected graph G=(N,E) is constructed, where N is the set of nodes and E is the set of edges. The graph feature parameterization step calculates the node degree for each node, i.e., the number of edges connected to that node. For each edge, the blood vessel segment length is calculated, which is the number of pixels along the skeleton path multiplied by the pixel spacing. The average radius of the blood vessel segment is calculated by sampling the normal direction of the skeleton in multi-scale space to obtain a radius sequence and taking the average value. The tortuosity of the blood vessel segment is calculated, defined as the ratio of the actual length of the blood vessel segment to the Euclidean distance between its two endpoints. At each bifurcation node, the bifurcation angle is calculated, which is the angle between the direction vectors of the edges connected to the node. The feature combination of all nodes and edges forms a topological feature vector. The graph convolution update step employs the GraphSAGE architecture for message passing and node state updates on graph-structured data. GraphSAGE updates the hidden state of nodes by sampling neighboring nodes and aggregating their feature information. The aggregation function uses mean aggregation, averaging the features of the target node's neighboring nodes and concatenating them with the target node's own features. A linear transformation and a non-linear activation function then generate a new hidden state. After two graph convolutional layers, global pooling is performed on all hidden states to generate a graph-level feature vector, i.e., a global representation of the fundus image.

[0037] The cardiac CT image processing submodule processes coronary CTA data. First, adaptive bilateral filtering is used for image enhancement. This filter introduces a spatial adaptive mechanism based on traditional bilateral filtering, automatically adjusting the kernel size according to the gradient information of local image regions. A large kernel is used for efficient noise reduction in smooth areas, while a small kernel is used to preserve edge details in edge regions. The enhanced CT image is then input into the myocardial segmentation module, where the myocardial region is segmented using a kernel density fuzzy C-means algorithm. This algorithm introduces kernel density estimation based on standard fuzzy C-means, mapping the original pixel values ​​to a high-dimensional feature space through a Gaussian kernel function. Clustering segmentation is performed in this high-dimensional space, better handling issues of uneven grayscale and blurred tissue boundaries in CT images. The segmentation output is a binary mask of the left ventricular myocardial region. Coronary artery extraction employs a strategy combining region growing and active contour models. The region growing method uses the coronary artery ostia as seed points and utilizes the contrast agent enhancement characteristics of the coronary vessels to set a growth threshold, obtaining preliminary segmentation results of the coronary artery tree. Based on this, the active contour model uses the preliminary segmentation results as the initial contour. By minimizing the energy function that includes image gradient energy and contour smoothing energy, it drives the contour to evolve towards the real vessel boundary, outputting a refined three-dimensional coronary tree structure. Feature extraction employs a squeeze-excitation network and linear frequency modulation transform to extract multi-scale texture and frequency domain features. This transform, through linear frequency modulation basis function decomposition of local image regions, can simultaneously capture spatial texture information at different scales and frequency domain energy distributions in different directions, demonstrating good distinguishing ability for vessel wall calcification and plaque tissue components. Plaque analysis quantitatively evaluates each plaque in the three-dimensional coronary tree structure. The criteria for identifying low-attenuation plaques are a CT value of less than 30 Henle units in the plaque region; the criteria for identifying positive reconstruction are a ratio of the outer wall area of ​​the vessel segment containing the plaque to the outer wall area of ​​the reference vessel segment greater than 1.1; and the criteria for identifying punctate calcification are a maximum diameter of less than 3 mm for the calcification foci and a calcification area accounting for less than 50% of the total plaque area. The high-risk plaque feature set includes a combination of the above three features, as well as the plaque eccentricity index and plaque burden. Pericoronary fat analysis is assessed based on the pericoronary fat attenuation index. This index is defined as the average CT attenuation value of the peripheral wall of the coronary artery over a specific radial distance, expressed in Henle units. Studies have found that an elevated pericoronary fat attenuation index is associated with vascular inflammation and can independently predict the risk of cardiovascular events. The cardiac CT image processing submodule extracts the fat attenuation index around each main coronary branch and takes the average value of each branch as the final feature value output.

[0038] The cardiac MRI processing submodule analyzes long-axis two-chamber, three-chamber, and four-chamber cardiac sequences, as well as short-axis cine sequences. Long-axis cine sequence analysis uses a deep learning model to segment the boundaries of the left ventricular endocardium and epicardium. The segmentation network employs a U-Net structure with attention gating, taking end-diastolic and end-systolic cardiac MRI image frames as input and outputting the contours of the left ventricular endocardium and epicardium. Based on the segmentation results, the left ventricular ejection fraction is automatically calculated using the formula: end-diastolic volume minus end-systolic volume divided by the end-diastolic volume multiplied by 100%. The absolute values ​​of the end-diastolic and end-systolic volumes are also output in milliliters. Myocardial strain analysis employs feature tracking technology to trace the motion trajectory of myocardial tissue points during the cardiac cycle on short-axis and long-axis image sequences, calculating global longitudinal strain, circumferential strain, and radial strain. Global longitudinal strain is defined as the relative percentage shortening of the myocardium along the long axis from end-diastole to end-systole, with a normal reference value below -20%. Global circumferential strain is defined as the relative percentage shortening of the myocardial circumference in the short-axis direction. Global radial strain is defined as the relative percentage increase in myocardial wall thickness. Delayed gadolinium-enhanced sequences are used to assess the degree of myocardial fibrosis, and the fibrosis volume and the percentage of fibrosis in the left ventricular myocardial volume are calculated by the signal intensity ratio between normal myocardium and fibrotic areas.

[0039] The physiological signal processing submodule includes a nighttime polysomnography signal processing unit, a dynamic electrocardiogram signal processing unit, and a pulse wave and speech signal processing unit.

[0040] The nighttime polysomnography (PSG) signal processing unit is designed based on the SleepCVD-Net architecture, with dedicated feature extraction branches for ECG, airflow, and blood oxygenation signals. The ECG sampling rate is set to 256Hz to 512Hz. Preprocessing steps include bandpass filtering and R-wave detection. The bandpass filter uses a zero-phase digital filter with a passband range of 0.5Hz to 40Hz, effectively filtering out baseline drift and high-frequency electromyographic noise. R-wave detection employs the Pan-Tompkins algorithm, first enhancing the frequency components of the QRS complex through bandpass filtering, then performing differentiation, square sum, and moving window integration, and using an adaptive threshold to detect the R-wave position. After R-wave detection, the jump-by-jump sequence, i.e., the time interval between adjacent R waves, is calculated. Time-frequency domain features of heart rate variability are extracted based on the jump-by-jump sequence. Time-domain features include the standard deviation of the jump-by-jump, the root mean square of the difference between adjacent jump-by-jumps, and the percentage of jump-by-jump intervals greater than 50ms. Frequency domain features were obtained by estimating the power spectral density of the jump-by-jump interval sequence, including low-frequency power, high-frequency power, and the low-frequency to high-frequency power ratio. Heart rate variability entropy was calculated using a sample entropy algorithm, reflecting the complexity of heart rate variability; reduced sample entropy is associated with autonomic nervous system dysfunction. The QT interval was represented by the time interval from the start of the QRS complex to the end of the T wave, corrected using the Bazett formula to achieve a heart rate-independent corrected QT interval. The sampling rate for the airflow signal was set to 50Hz to 100Hz. Preprocessing included low-pass filtering and baseline drift correction. The low-pass filter cutoff frequency was set to 15Hz to preserve the low-frequency components of respiratory motion. Baseline drift correction used polynomial fitting to remove slow trends. Feature extraction included respiratory amplitude (peak value per respiratory cycle), respiratory rate (number of respiratory cycles per unit time), and apnea and hypopnea index (defined as the total number of apnea and hypopnea events per hour during sleep, a core indicator for diagnosing sleep apnea syndrome). The sampling rate for the blood oxygen signal was set to 10Hz to 25Hz. Preprocessing included outlier removal and smoothing filtering. Outlier removal employed median filtering and standard deviation thresholding to remove spikes and sudden drops caused by poor sensor contact. Smoothing filtering used a moving average method with a window width of 3 sampling points. Feature extraction included the oxygen depletion index (the number of times blood oxygen saturation decreased by more than 3 percentage points per hour); minimum blood oxygen (the lowest blood oxygen saturation value recorded overnight); and blood oxygen fluctuation entropy, which quantitatively described the complexity and irregularity of blood oxygen fluctuations by calculating the sample entropy or approximate entropy of the blood oxygen time series. The feature extraction branches for the three signals all employed customized convolutional neural network and recurrent neural network architectures: the ECG branch used a one-dimensional convolutional network based on a residual network, the airflow branch used a long short-term memory network, and the blood oxygen branch used a multi-scale convolutional network. Finally, the output features of the three branches were fused through a cross-modal attention layer. The attention mechanism calculated the contribution weight of each branch's features to the cardiopulmonary coupling state, and a weighted output cardiopulmonary coupling feature vector was generated.

[0041] The dynamic ECG signal processing unit is used to denoise, perform QRS detection, and identify abnormal ECG types from the dynamic ECG signals, outputting ECG feature vectors. The dynamic ECG signals are obtained from 24-hour dynamic ECG signals acquired by the in-hospital data acquisition unit using a 12-lead dynamic ECG recorder. The denoising process is consistent with the preprocessing method of the ECG signals from the nocturnal polysomnography signal processing unit, employing bandpass filtering to eliminate baseline drift and high-frequency electromyographic noise, with the bandpass range set from 0.5Hz to 40Hz. QRS detection uses the Pan-Tompkins algorithm to locate the R-wave and calculates the time-domain and frequency-domain features of heart rate variability based on the jump-by-jump interval sequence. Abnormal ECG type identification uses a deep convolutional neural network to classify the denoised ECG waveform segments, outputting the identification results and ECG feature vectors. When a preset critical ECG type is identified, it is used as a critical sign detected in the physiological signal data for risk assessment by the risk stratification and classification module.

[0042] The pulse wave and speech signal processing unit serve as auxiliary fusion modes. Radial artery pulse waves are acquired from the wearable device using photoplethysmography (PPG), with processing steps including denoising, feature point detection, and parameter calculation. Denoising employs wavelet transform soft thresholding to decompose the signal into multiple scales, reconstructing the details at each level after threshold shrinkage. Feature point detection identifies the main peak, dichroic notch, and dichroic peak of the pulse wave in the time domain. Based on these feature points, time-domain features are calculated, including the main wave amplitude (vertical height from baseline to the main peak), the dichroic notch amplitude (vertical height from baseline to the dichroic notch), the ratio of the main wave amplitude to the dichroic notch amplitude, pulse rate, and pulse rate variability. Frequency-domain features are obtained by performing a Fast Fourier Transform on the pulse wave to obtain the harmonic energy distribution, calculating the proportion of the energy of the first five harmonics to the total energy. Hemodynamic parameters, including cardiac output and peripheral resistance, are estimated using a pulse wave analysis algorithm combined with patient age, gender, and body surface area. Speech signals were acquired via a smartphone microphone, and patients read standard text or produced sustained vowels as prompted. The processing steps included pre-emphasis, framing, windowing, and endpoint detection. Pre-emphasis used a first-order high-pass filter to compensate for high-frequency attenuation of the speech signal. Framing used a frame length of 25ms and a frame shift of 10ms. Windowing employed a Hamming window to reduce spectral leakage. Endpoint detection used a dual-threshold method based on short-time energy and short-time zero-crossing rate to determine the effective speech segments. Feature extraction used Mel-frequency cepstral coefficients; the speech signal was passed through a Mel filter bank to obtain the Mel spectrum, and after taking the logarithm, a discrete cosine transform was performed to obtain 13-dimensional Mel-frequency cepstral coefficients. The fundamental frequency was extracted using the autocorrelation method, reflecting the vocal cord vibration frequency. All acoustic features were combined to assess cardiopulmonary function and the coupling state of neuromodulation.

[0043] The clinical feature encoding module processes structured clinical data. Categorical variables use One-Hot encoding or embedding layers. Variables with fewer value categories, such as gender and ethnicity, use One-Hot encoding; variables with more value categories, such as combinations of past medical history, use embedding layers to map to a low-dimensional continuous vector space. Numerical variables are standardized using Z-Score, ensuring that the mean of each numerical variable is zero and the standard deviation is one, eliminating the influence of dimensional differences on the model. Missing values ​​are handled using multiple imputation, consistent with the aforementioned methods. Feature selection is based on SHAP values ​​to calculate the feature importance ranking of each feature in the gradient boosting tree model, and then combined with clinical expert knowledge to select the core variable subset that contributes most significantly to cardiovascular risk prediction.

[0044] The cross-modal alignment fusion module employs a three-level fusion strategy. Level 1 fusion is a homomodal multi-sequence fusion, using a 3D convolutional neural network to jointly extract features from multiple temporal image sequences for both long and short axis sequences of cardiac MRI, concatenating these multi-sequence image features into a comprehensive cardiac MRI feature vector. For cardiac CT, plain and enhanced sequences are fused into comprehensive image features. Level 2 fusion is a heterogeneous feature-level fusion, employing a multi-head attention mechanism to project imaging features, physiological signal features, and structured clinical features into the same latent space. The fundus image features, cardiac CT features, cardiac MRI features, physiological signal features, and clinical features output from the aforementioned fundus image processing submodule, cardiac CT image processing submodule, cardiac MRI processing submodule, physiological signal processing submodule, and clinical feature encoding module are input into their respective independent classifiers, each outputting a probability vector. The probability vectors of all classifiers are arithmetically averaged by category to obtain a comprehensive probability distribution, and the category with the highest probability is selected as the final comprehensive risk score. For cases where classifier performance varies significantly, a stacking strategy is adopted, where the output probabilities of each classifier are used as meta-features input into a meta-classifier, which then learns the optimal decision-level fusion method.

[0045] The risk stratification and classification module includes a contrastive learning unit, a stratification decision unit, and an output unit. The contrastive learning unit operates during the model training phase. The training set is constructed based on retrospective data, including de-identified medical records from collaborating hospitals and samples from the SHHS Sleep Heart Health Study, UK Biobank, and the MIMIC-IV public database. Each sample is labeled with a risk level by clinical experts according to clinical guidelines. For each sample in the training set, samples belonging to the same risk level are defined as positive sample pairs, and samples belonging to different risk levels are defined as negative sample pairs. The contrastive learning unit constrains the feature representation output by the model through a contrastive loss function, reducing the distance between positive sample pairs in the feature space and increasing the distance between negative sample pairs. The contrastive loss uses a normalized temperature-scaled cross-entropy loss, with the temperature parameter set between 0.1 and 0.5. The stratification decision unit first executes a binary classifier to determine whether the patient is high-risk. Positive samples in the binary classifier are classified as a combined category of extremely high risk and high risk, while negative samples are classified as a combined category of intermediate risk and low risk. If the binary classification determines the patient to be high-risk, a fine-grained classifier further determines whether the patient is extremely high-risk or high-risk. If the binary classification determines the risk as non-high risk, a fine-grained classifier further determines whether it is medium or low risk. Hierarchical decision-making improves overall decision accuracy by decomposing the complex multi-classification problem into a combination of simple classification problems. The final output of the output unit includes three items: risk level, which can be extremely high, high, medium, or low risk; confidence score, a value between 0 and 100%, reflecting the model's certainty about the prediction; and the ranking of modal contributions, arranged from highest to lowest according to the multi-head attention weight, with each modality name and its corresponding contribution percentage.

[0046] like Figure 5 As shown, the specific implementation of the cross-modal multi-head attention fusion unit is as follows: The specific calculation process of the multi-head attention mechanism in the secondary fusion unit is as follows: Suppose that after processing by the modality-specific feature extraction module, a total of M modal feature vectors are obtained. The dimensions of each feature vector are as follows: First, a linear projection layer is used to uniformly map the features of each modality to the same dimension. ,have to ,in Let be the projection matrix for the i-th modality. Then, concatenate all projected eigenvectors along the feature dimension to obtain the global feature matrix. .

[0047] The multi-head attention mechanism computes H attention heads in parallel, with each attention head independently performing a scaled dot product attention operation. For the r-th attention head, the calculation process is as follows:

[0048] in Let d_k = d_model / H be the query, key, and value projection matrix for the r-th attention head, where d_k = d_model / H is the dimension of each head. The attention weight matrix... elements in This represents the attention contribution of the j-th modality to the i-th modality. This weight reflects the strength of the correlation between the features of each modality. It is adaptively learned by the model during training and does not require manual setting.

[0049] The outputs of all attention heads are concatenated along the feature dimension, and then processed by the output projection matrix. Transform to obtain the fused feature vector:

[0050] The This is the comprehensive feature vector after cross-modal alignment and fusion, which is input into the risk stratification and classification module for risk level determination. The global contribution of each modality is calculated by averaging the attention weights corresponding to that modality across all attention heads, specifically:

[0051] The larger the contribution value, the greater the overall impact of the i-th mode on the current fusion result. The output unit sorts the contribution values ​​and outputs them as the contribution ranking of each mode.

[0052] The system also includes an interpretability module, which incorporates a global interpretation unit, a local interpretation unit, and a counterfactual reasoning unit.

[0053] The global interpretation unit analyzes the overall contribution of each modality and feature to the model's predictions using SHAP values. SHAP values, based on the Shapley value in game theory, quantify the marginal contribution of each feature to the prediction result across all feature combinations. The global interpretation unit calculates the SHAP values ​​of each feature for all validation set samples and then averages the absolute values ​​to obtain a global feature importance ranking, displayed in a bar chart for clinical experts to understand the model's main decision-making basis.

[0054] The Local Interpretation Unit (LIU) generates interpretations for single predictions using the LIME algorithm. For a single prediction, the LIME algorithm generates multiple perturbation samples by randomly perturbing the input sample. The original model is then used to predict these perturbation samples, and an interpretable linear model is trained to fit the prediction results of the perturbation samples. The weights of the linear model reflect the direction and magnitude of the influence of each feature on the prediction. For image modalities, Local Interpretation generates heatmaps to annotate key image regions. Red areas in the heatmap represent regions that contribute most to high-risk predictions, while blue areas represent regions that contribute significantly to low-risk predictions. For physiological signals, Local Interpretation annotates key signal segments, such as the key segments that lead to the judgment of ST segment elevation in Holter ECG signals. For structured data, Local Interpretation annotates the clinical indicators and their values ​​that have the greatest impact on the prediction.

[0055] The counterfactual reasoning unit provides simulation suggestions. When the model predicts a patient to be high-risk or very high-risk, the counterfactual reasoning unit iteratively modifies the input feature values ​​to find the minimum squared directional change that can reduce the predicted risk level to low risk. For example, the counterfactual reasoning output might be: if LDL cholesterol decreases from its current value to 2.6 mmol / L and systolic blood pressure decreases to 130 mmHg, the predicted risk level can be reduced from high-risk to intermediate-risk. The counterfactual reasoning results are presented to clinicians in natural language.

[0056] like Figure 3 As shown, the intervention response layer includes a cloud-based decision support platform, an active early warning and response module, a resident-end service module, a grassroots empowerment module, and a closed-loop management module.

[0057] The cloud-based decision support platform receives multimodal analysis results in real time via a message queue. The message queue uses Apache Kafka or RabbitMQ. Each time the analysis layer completes a risk assessment, it generates a JSON message containing the patient's unique identifier, risk level, confidence score, contribution of each modality, and key feature values, and publishes it to the alert message topic. The cloud-based decision support platform subscribes to this topic, consumes and integrates the message content in real time, and generates a comprehensive risk assessment report. The clinical guideline alignment unit performs cross-validation based on the Chinese Cardiovascular Disease Prevention Guidelines and domestic and international ASCVD risk assessment models. Specifically, for the same patient, both the traditional ASCVD risk score and the output risk level of the model of this invention are calculated simultaneously. If there is a significant discrepancy between the two, the system automatically marks it as a case pending review and pushes it to the expert review queue for manual judgment by clinical experts. The personalized treatment plan generation unit automatically generates personalized intervention plans based on the risk level and specific risk factors. For the red (very high risk) category, recommended interventions include immediate initiation of statin therapy. High-intensity statins are recommended if LDL cholesterol is ≥4.9 mmol / L, and ezetimibe may be considered if the patient has diabetes or a history of cardiovascular events. For antiplatelet therapy, low-dose aspirin is recommended if the patient is ≥40 years old and has a low bleeding risk. Emergency room access routes will be sent to the patient's mobile phone, including the address and navigation link of the nearest hospital with emergency intervention capabilities. For the orange (high risk) category, intensive monitoring is recommended, including blood pressure and heart rate measurements at least three times a week, monthly lipid and blood glucose checks, and a visit to a cardiovascular specialist clinic within one week. For the yellow (intermediate risk) category, further investigations are recommended, including coronary CTA or exercise stress ECG, Holter monitoring, and carotid ultrasound. For the green (low risk) category, lifestyle interventions are recommended, including personalized dietary recommendations based on the patient's current diet, a prescription for at least 150 minutes of moderate-intensity aerobic exercise per week, smoking cessation guidance, and a one-year follow-up appointment.

[0058] The active early warning and response module includes a pre-hospital early warning process and an in-hospital coordinated response mechanism. In the pre-hospital early warning process, after the system detects a high-risk signal, it determines whether the AI confidence level exceeds 90%. If it does, the system automatically triggers a voice call to the work mobile phone or landline of the on-duty physician available 24 hours a day. After the on-duty physician answers the call, they listen to the system's voice broadcast of the patient's basic information and early warning summary. The on-duty physician logs into the system to view the complete comprehensive risk assessment report and multimodal analysis details, and completes manual re-judgment within 30 minutes. After the re-judgment confirms that the early warning is valid, the system simultaneously performs three operations: notifies the patient and emergency contacts through dual channels of SMS and voice calls, with the content including the early warning level, recommended measures, and information of nearby medical institutions; pushes the early warning information to the community health service center where the patient belongs through the regional health information platform, and triggers a to-do task reminder for the primary care physician; if the early warning level is red extremely high risk, the system automatically notifies the regional 120 command center through the standard emergency interface protocol, transmits the patient's geographic coordinates, suspected diagnosis and key physiological parameters, and assists the emergency center in dispatching the nearest emergency vehicle for transfer. When the patient arrives at the emergency department, the emergency physician has already received the complete push of preliminary diagnosis information on the workstation, including the early warning level, confidence level, key image markers and recommended disposal path. In terms of in-hospital coordinated response, the emergency department immediately prepares resuscitation beds, monitoring equipment and emergency medicines after the early warning is triggered. The cardiology department and the neurology department initiate an online multi-disciplinary consultation discussion within 15 minutes, and share all the patient's multimodal data and early warning reports through the built-in consultation platform of the system. The interventional catheterization laboratory completes preoperative preparation and personnel arrangement within 30 minutes after receiving the early warning of acute myocardial infarction or stroke. The imaging department opens a green channel for emergency early warning patients, and immediately arranges CTA or MRI examination after the patient arrives. The information department ensures the tracking record and closed-loop confirmation of the early warning information in the whole process.

[0059] The resident-side service module is provided in the form of a WeChat mini-program or a dedicated APP. The heart health record function allows patients to query their personal heart health score, historical trend charts and risk change curves at any time. The health record displays all previous assessment results in the form of a timeline, and the risk change curve displays the score change trend in the last 12 months in the form of a line chart. Personalized health reminders are automatically pushed according to the risk level: extremely high-risk and high-risk patients are pushed medication and monitoring reminders daily, medium-risk patients are pushed once a week, and low-risk patients are pushed a comprehensive health prompt once a month. Home data upload supports patients to independently enter or synchronize home self-measured blood pressure, blood glucose, weight and portable electrocardiogram data via Bluetooth. After the data is uploaded, it is automatically incorporated into the intelligent analysis engine for re-risk assessment. The online consultation channel automatically activates the one-click connection to online physician function after a high-risk early warning, and the patient can initiate a video or text consultation with one click. Family member binding supports the patient to add the mobile phone numbers of up to five emergency contacts to share health early warning information.

[0060] The primary care empowerment module automatically identifies individuals with hypertension, hyperglycemia, hyperlipidemia, and those who are overweight or obese, and establishes electronic health records. Inclusion criteria include: blood pressure consistently above 140 / 90 mmHg or currently taking antihypertensive medication; fasting blood glucose above 7.0 mmol / L or currently taking hypoglycemic medication; total cholesterol above 6.2 mmol / L or currently taking lipid-lowering medication; and a body mass index (BMI) of 28 kg / m² or higher. 2 The tiered medical service recommendations are automatically generated based on the warning level. Extremely high-risk patients are recommended for immediate referral to a regional medical center; high-risk patients are recommended for referral to a specialist outpatient department at a higher-level hospital within 72 hours; medium-risk patients are recommended for further examination or remote consultation at a primary healthcare institution; and low-risk patients are recommended for observation and management at the primary care level. The primary care physician's warning dashboard displays a real-time overview of all high-risk patients within their jurisdiction, including the number of currently monitored patients, the number of newly issued warnings in the last 24 hours, the number of patients awaiting follow-up, and the week-on-week trend of the warnings. Follow-up management automatically generates follow-up plans: monthly for high-risk patients, every three months for medium-risk patients, and annually for low-risk patients. Three days before the scheduled follow-up date, the system automatically sends a reminder notification to both the primary care physician and the patient.

[0061] The closed-loop management module implements a five-step closed loop: early warning, response, handling, feedback, and optimization. In the early warning step, the intelligent analysis layer automatically issues AI warnings, generating warning record numbers and timestamps. In the response step, clinicians manually review the warning within 30 minutes, confirming its validity or determining it as a false alarm; the review result and the reviewing physician's signature are recorded in the system log. In the handling step, the clinical team executes the corresponding level of intervention plan, recording the start time, details of the handling measures, and the completion time. In the feedback step, the clinical team enters the information into the system after the patient receives a final diagnosis, recording the final diagnosis name, diagnostic basis, patient prognosis, discharge status, and the clinician's subjective evaluation of the usefulness of the AI ​​warning. In the optimization step, the AI ​​platform automatically triggers an incremental learning process after collecting sufficient real-world outcome data. Using newly added data from the database over the past three months as training samples, it performs several rounds of parameter updates on the existing model, retaining the version with the best performance on the validation set as the new production version. Before the new version goes live, performance regression testing ensures that all indicators are not lower than the existing version. The incremental learning cycle is set to once a month or triggered immediately when the cumulative number of newly confirmed cases exceeds fifty.

[0062] In a specific implementation of cardiovascular and cerebrovascular disease screening, nighttime physiological signal data were obtained from the Sleep Heart Health Study Database, a multicenter cohort study conducted by the National Heart, Lung, and Blood Institute in the United States. This study aimed to explore the relationship between sleep-disordered breathing and cardiovascular disease and all-cause mortality. The first phase of the study included 6441 participants, and the second phase included 3295 participants. This implementation selected 194 subjects who completed nighttime polysomnography in the second phase, including 80 healthy controls and 114 cardiovascular disease patients, covering four population groups: healthy controls, stroke patients, angina patients, and congestive heart failure patients. Significant differences were found in baseline characteristics among the groups. The cardiovascular disease group had a mean age of 73.7 years, a mean sleep efficiency of 74.9%, and a mean total sleep time of 355.9 minutes, all of which were statistically different from the healthy control group.

[0063] Three core physiological signals were extracted from polysomnography (PSG) recordings as multimodal input data. Single-lead electrocardiogram (ECG) signals were recorded using silver / silver chloride electrodes at a sampling frequency of 250 Hz to extract time-frequency domain features of heart rate variability and cardiac electrical activity information such as the QT interval. Blood oxygen saturation signals were measured using Nonin XPOD 3011 and 8000 sensors at a sampling frequency of 1 Hz to assess nocturnal blood oxygen fluctuations and the oxygen depletion index. Airflow signals were recorded using a Compumedics thermal sensor at a sampling frequency of 10 Hz to assess respiratory rate, respiratory amplitude, and apnea and hypoventilation indices. During signal preprocessing, a 0.05 Hz high-pass filter was applied to the ECG and airflow signals to eliminate baseline drift. Complete time-series data from the PSG recordings were extracted, and the first and last 15 minutes of data were removed to exclude awake or noisy segments. The effective data were segmented into non-overlapping segments with a 30-second window size to construct the final nocturnal physiological signal dataset. The dataset was randomly divided into a development group and an evaluation group in a 7:3 ratio. 10% of the development data was used as the validation set for the training phase. The training set contained 104,511 fragments from 116 subjects, the validation set contained 51,476 fragments from approximately 20 subjects, and the test set contained 76,830 fragments from 58 subjects.

[0064] The model performance was evaluated using accuracy and F1-score as the main metrics. Accuracy was defined as the sum of true positives and true negatives divided by the sum of true positives, false positives, true negatives, and false negatives. The F1-score was the harmonic mean of precision and recall. According to publicly available literature, the SleepCVD-Net architecture referenced in this embodiment achieved an average accuracy of 97.55% on the aforementioned SHHS dataset for a four-class classification task. The F1-scores for each class were 97.97% for the healthy control group, 96.35% for the stroke group, 97.79% for the angina group, and 97.49% for the congestive heart failure group. The overall average accuracy for the training, validation, and test sets were 99.98%, 97.48%, and 97.55%, respectively. These figures represent the literature reports on this reference architecture on publicly available datasets and are used to illustrate the expected performance level of the technical architecture adopted in this application. The confusion matrix is ​​used to assess the consistency between model predictions and cardiologist diagnoses, while the receiver operating characteristic (ROC) curve is used to evaluate the model's classification performance at various classification thresholds; a larger area under the curve indicates higher classification accuracy. The aforementioned literature reports that the multimodal AI method based on nocturnal polysomnography (PSG) signals can effectively achieve automated screening for cardiovascular diseases. It can be applied to nocturnal cardiac activity monitoring in hospitalized patients, outpatients, and home healthcare scenarios, extending the traditional use of PSG for sleep disorder diagnosis to the field of cardiovascular disease detection.

[0065] In a specific embodiment of cardiovascular risk assessment based on fundus images, retinal fundus images of the target subject are acquired using a fundus imaging device. Fundus images, as the only non-invasive window for directly observing the human vascular network, can be used to extract biomarkers such as vascular topology, arteriovenous ratio, and bifurcation angles. This invention employs a cardiovascular risk assessment model based on a deep neural network. This model includes a feature extraction network and at least one output network. The feature extraction network extracts feature information from the fundus images, and the output network outputs the assessment results based on the feature information. In a preferred embodiment, an autoencoder model based on the Transformer architecture, such as the RETFound basic model, can be used to learn general representations from unlabeled retinal images through unsupervised learning, without relying on displayed label information. This method trains the model by receiving one or more sets of training retinal images and corresponding training cardiovascular event labels, inputting the extracted feature dataset into the machine learning model, and training it to output cardiovascular event indicators based on the feature dataset. In another embodiment, the model employs a two-level architecture. A first model analyzes the input fundus images to predict cardiovascular risk and outputs a cardiovascular risk score. A second model receives the cardiovascular risk score predicted by the first model and input disease-type-specific factors to predict the risk of specific cardiovascular diseases. Diagnostic information is provided based on the prediction results of both models. Model output may include some or all of the following: age, gender, systolic blood pressure, diastolic blood pressure, history of diabetes, history of hypertension, high-density lipoprotein (HDL), low-density lipoprotein (LDL), BMI, waist circumference, smoking history, total cholesterol, place of residence, family history, glycated hemoglobin information, and uric acid information. Based on this, a cardiovascular risk value can be calculated, or the model can directly output a cardiovascular risk value. Deep learning models based on fundus images can effectively assess cardiovascular disease risk. Compared with traditional risk prediction methods, multiple cardiovascular risk-related parameters can be obtained through a single fundus image acquisition, simplifying the assessment process and making it suitable for primary healthcare institutions and large-scale population screening scenarios.

[0066] In a specific embodiment of risk prediction based on multimodal physiological rhythm synchronization analysis, standardized time-series data of multiple modalities of the target object are acquired. The multimodal signals include at least electrocardiogram (ECG), pulse wave, and respiratory signals. Frequency domain decomposition is performed on each modal signal to extract the rhythm component time-series signals corresponding to preset physiological frequency bands. Hilbert transform is applied to the rhythm component time-series signals to obtain their instantaneous phases. Stability quantification is performed based on the instantaneous phase differences between different modal rhythm components, and phase synchronization strength is calculated. A multimodal physiological rhythm synchronization dynamic network is constructed using rhythm components as nodes and phase synchronization strength as edge weights. Through in-depth mining of multimodal physiological time-series data within a continuous time window, the dynamic changes in phase synchronization between different modal physiological rhythms are accurately captured, effectively distinguishing normal fluctuations of the physiological system from abnormal signals in the pre-disease stage. Based on the constructed multimodal physiological rhythm synchronization dynamic network, variant edges are identified, and the changes in network topology features are calculated, thereby outputting cardiovascular disease risk prediction results. This method, through the construction of a multimodal physiological rhythm synchronous dynamic network and the identification of variant connections, can capture abnormal changes in the functional coupling of physiological systems before the appearance of clinical symptoms of disease, thus achieving true early risk warning.

[0067] On the other hand, embodiments of the present invention also provide a method for early warning of cardiovascular and cerebrovascular diseases based on multimodal information, comprising: collecting image data, physiological signal data, and structured clinical data of the target object; extracting features from each modality of data and outputting modal feature vectors corresponding to each modality of data; projecting the modal feature vectors to the same latent space, calculating cross-modal attention weights through a multi-head attention mechanism, performing weighted fusion of the modal feature vectors based on the cross-modal attention weights, and outputting a fused feature vector; performing risk stratification based on the fused feature vectors and outputting risk levels and confidence scores; and generating and pushing early warning information according to the risk levels.

[0068] Meanwhile, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for early warning of cardiovascular and cerebrovascular diseases based on multimodal information.

[0069] Through the specific embodiments described above, this invention fully realizes a complete set of technical solutions, from multimodal data acquisition, standardized processing, modality-specific feature extraction, cross-modal multi-head attention fusion, contrastive learning risk stratification to proactive early warning response and closed-loop management throughout the entire process. This invention is not limited to the specific embodiments described above; any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A cardiovascular and cerebrovascular disease early warning system based on multimodal information, characterized in that, include: The data acquisition layer is configured to acquire image data, physiological signal data, and structured clinical data of the target object. The image data includes fundus images, cardiac CT and / or cardiac MRI. The physiological signal data includes at least one of dynamic electrocardiogram signals, pulse wave signals, blood oxygen saturation signals, and polysomnography signals. The image data, physiological signal data, and structured clinical data acquired by the data acquisition layer are collectively referred to as modal data. The intelligent analysis layer, which is communicatively connected to the data acquisition layer, includes a modality-specific feature extraction module, a cross-modal alignment and fusion module, and a risk stratification and classification module. The modality-specific feature extraction module is used to extract features from each modality data and output modality feature vectors corresponding to each modality data. The cross-modal alignment and fusion module is used to project the feature vectors of each modality onto the same latent space, calculate cross-modal attention weights through a multi-head attention mechanism, and perform weighted fusion of the feature vectors of each modality based on the cross-modal attention weights to output a fused feature vector. The risk stratification and classification module is used to perform risk stratification based on the fused feature vector and output the risk level and confidence score. The intervention response layer is communicatively connected to the intelligent analysis layer and is configured to generate and push early warning information and personalized intervention plans based on the risk level.

2. The cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The modality-specific feature extraction module includes: The fundus image processing submodule is configured to segment retinal vessels based on the U-Net++ model, transform the vascular network into a graph structure, update features through a graph neural network, and output a graph-level feature vector of the fundus image. The cardiac CT image processing submodule is configured to use the kernel density fuzzy C-means algorithm for myocardial segmentation, extract the three-dimensional structure of the coronary artery tree through the region growing method and active contour model, extract multi-scale texture and frequency domain features using the squeeze-excitation network and linear frequency modulation transform, identify high-risk plaque features and calculate the pericoronary fat attenuation index, and output cardiac CT feature vectors. The cardiac MRI processing submodule is configured to segment the left ventricular boundary based on deep learning, calculate the ejection fraction and ventricular volume, calculate myocardial strain parameters through feature tracking technology, evaluate the delayed gadolinium enhancement region, and output cardiac MRI feature vectors. The physiological signal processing submodule is configured to preprocess and extract specific features from the electrocardiogram (ECG), airflow, and blood oxygenation signals in polysomnography signals, and output cardiopulmonary coupling feature vectors; perform denoising and QRS detection on dynamic ECG signals, identify abnormal ECG types based on deep convolutional neural networks, and output ECG feature vectors; and extract time-domain and frequency-domain features from pulse wave signals and output hemodynamic parameter vectors. The clinical feature encoding module is configured to encode categorical variables, standardize numerical variables, and handle missing values ​​in structured clinical data, and output clinical feature vectors.

3. The cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The cross-modal alignment and fusion module includes a multi-level fusion unit, which includes: The first-level fusion unit is configured to perform feature concatenation or 3D convolution fusion on multiple sequence data within the same modality, and output comprehensive features of the same modality. The second-level fusion unit is configured to perform attention-weighted concatenation of feature vectors from different modalities through a multi-head attention mechanism. The cross-modal attention weights reflect the contribution of each modality to the current prediction task, are dynamically adjusted, and achieve interpretability. The three-level fusion unit is configured to use soft voting or stacking ensemble learning methods to perform decision-level fusion of the outputs of each modality-independent classifier, and output the final comprehensive risk score.

4. The cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The risk stratification and classification module includes a contrastive learning unit, a stratified decision-making unit, and an output unit. The contrastive learning unit is configured to compare risk features based on the feature space obtained through contrastive learning training. The feature space is constructed during the model training phase by bringing the feature vectors of samples with the same risk level closer together and pushing the feature vectors of samples with different risk levels further apart, thereby enhancing the inter-class discrimination in the feature space. The stratified decision-making unit is configured to first perform binary classification to determine whether it is high-risk. If it is determined to be high-risk, it is further classified into extremely high-risk or high-risk through fine-grained classification. If it is determined to be non-high-risk, it is further classified into medium-risk or low-risk through fine-grained classification. The output unit is configured to output the risk level, confidence score, and ranking of the contribution of each modality.

5. A cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The risk levels are categorized as extremely high, high, intermediate, and low. Extremely high risk is determined when the fused feature vector is classified as high-risk by the risk stratification and classification module, with a confidence score greater than a preset confidence threshold. Simultaneously, the system detects preset critical signs based on imaging or physiological signal data and triggers an immediate emergency response. High risk is determined when the fused feature vector is classified as high-risk by the risk stratification and classification module, with a confidence score greater than a preset confidence threshold, triggering enhanced monitoring and a recommendation for specialist consultation within one week. Intermediate risk is determined when at least two modalities of the feature vectors, after independent assessment, indicate risk and trigger further examination recommendations. Low risk is determined when all modalities do not indicate significant risk and trigger routine health management recommendations.

6. A cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The data acquisition layer includes: The in-hospital data acquisition unit is configured to import DICOM standard image data through the PACS system interface, import structured medical records and laboratory data through the HIS / EMR system interface, and acquire physiological signal data through bedside monitors and dynamic electrocardiogram recorders. The pre-hospital data acquisition unit is configured to monitor heart rate, dynamic electrocardiogram signals, blood pressure, blood oxygen saturation and polysomnography signals in real time through wearable devices, collect home physiological signal data through portable monitoring devices, and transmit the data to the cloud analysis platform via 5G / Wi-Fi / Bluetooth. The symptom-triggered acquisition unit is configured to automatically trigger a high-frequency data acquisition mode when abnormal physiological indicators are detected, thereby shortening the sampling interval.

7. A cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The intervention response layer includes: The cloud-based decision support platform is configured to receive multimodal analysis results in real time via a message queue, perform cross-validation with reference to clinical guidelines, and automatically generate personalized intervention plans based on risk levels and specific risk factors. The proactive warning and response module is configured to call the on-duty physician for manual review when a high-risk signal is detected and the confidence level exceeds a preset threshold. After confirmation, the patient and emergency contact will be notified simultaneously, and the warning information will be pushed to the community health service center. In critical situations, the regional emergency command center will be automatically notified. The resident-side service module is configured to provide access to cardiac health records, personalized health reminders, home data uploads, and online consultation channels, and supports family member binding and early warning information sharing. The primary care empowerment module is configured to automatically identify and record individuals with hypertension, hyperglycemia, hyperlipidemia, and overweight / obesity, recommend primary care observation or referral to higher-level hospitals based on the warning level, provide a real-time overview of the status of high-risk patients within its jurisdiction, automatically generate follow-up plans and provide reminders when they are due.

8. A cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, It also includes an interpretability module and a closed-loop management module; the interpretability module is configured as follows: By analyzing the SHAP values, the overall contribution of each mode and feature to the model prediction is analyzed, and a global interpretation is generated. The LIME algorithm is used to generate local interpretations for a single prediction, and key imaging regions, key signal segments and key clinical indicators are marked with heat maps. It provides counterfactual reasoning to simulate the change in risk level after improvement of target clinical indicators, which include at least one of low-density lipoprotein cholesterol, systolic blood pressure, glycated hemoglobin, body mass index, and pericoronary fat attenuation index. The closed-loop management module is configured as follows: After the intelligent analysis layer issues an AI warning, it records the results of the clinician's manual review and confirmation within a preset time. Record the treatment outcomes, final diagnosis, and result data of the clinical team in implementing the intervention plan; The resulting data is automatically fed back to the model in the intelligent analysis layer for incremental learning, continuously optimizing model performance.

9. A cardiovascular and cerebrovascular disease early warning system based on multimodal information according to claim 1, characterized in that, The data acquisition layer also includes a data standardization and quality control unit, configured as follows: Image data is uniformly converted to DICOM or NIfTI format, physiological signal data is uniformly converted to EDF or CSV format, and structured clinical data follows the HL7 FHIR standard; Perform deduplication, missing value imputation using multiple imputation methods, and outlier detection and labeling; The verification results are: image signal-to-noise ratio greater than or equal to 20dB, effective signal duration greater than or equal to 6 hours, and data integrity rate greater than or equal to 95%. The data is de-identified and anonymized, and the transmission uses TLS 1.3 encryption, while the storage uses AES-256 encryption.

10. A method for early warning of cardiovascular and cerebrovascular diseases based on multimodal information, based on the early warning system for cardiovascular and cerebrovascular diseases based on multimodal information as described in any one of claims 1 to 8, characterized in that, include: Collect imaging data, physiological signal data, and structured clinical data of the target subjects; Feature extraction is performed on each modal data separately, and the corresponding modal feature vectors are output. The modal feature vectors are projected onto the same latent space, cross-modal attention weights are calculated using a multi-head attention mechanism, and the modal feature vectors are weighted and fused based on the cross-modal attention weights to output a fused feature vector. Risk stratification is performed based on the fused feature vector, and the risk level and confidence score are output. Early warning information is generated and pushed out based on the risk level.

Images