Aortic valve stenosis classification and survival risk prediction system, device, and storage medium

Abstract

The present application belongs to the medical technical field, and specifically relates to an aortic valve stenosis typing and survival risk prediction system, equipment and storage medium. The system comprises a multi-modal data acquisition module and a typing and risk prediction module of a target object; the multi-modal data comprises a three-dimensional heart image, a fiber calcification spatial distribution mask generated based on the three-dimensional heart image, and clinical data; the fiber calcification spatial distribution mask is a binary three-dimensional mask for indicating the spatial positions of fibrotic tissue and calcified tissue in the aortic valve; the multi-modal data is taken as the input of a prediction model to obtain the aortic valve stenosis typing result of the target object and the survival risk quantitative value of the target object within a preset time window after transcatheter aortic valve replacement. The present application can eliminate the dependence of manual delineation on observers, quantify spatial distribution heterogeneity, overcome the rough problem of missing data processing, and greatly improve the clinical diagnosis and treatment efficiency.

Description

Technical Field

[0001] This invention belongs to the field of medical technology, specifically relating to a system, device and storage medium for aortic stenosis classification and survival risk prediction. Background Technology

[0002] Aortic stenosis (AS) is a highly heterogeneous valvular disease, giving rise to various hemodynamic classifications. Although transcatheter aortic valve replacement (TAVR) has become the standard treatment for intermediate- and high-risk symptomatic AS, postoperative prognosis varies significantly among patients, and traditional risk scores cannot fully explain the outcomes. Currently, clinical practice primarily classifies AS into four subtypes based on hemodynamic parameters (such as mean pressure gradient, valve orifice area, and ejection fraction) and uses clinical scoring for risk stratification.

[0003] Existing technologies have two major bottlenecks: First, the quantification of fibrous calcification relies on manual delineation, which is time-consuming and subject to significant differences among observers; second, it only focuses on the total volume and ignores the spatial heterogeneity of fibrous calcification tissue, which limits the accuracy of classification and prediction. Summary of the Invention

[0004] The purpose of this invention is to provide a system, device, and storage medium for aortic stenosis classification and survival risk prediction, in order to solve the problems in related technologies, such as reliance on manual delineation for fibrocalcification quantification, neglect of spatial distribution heterogeneity, crude processing of missing clinical data, and inefficiency caused by the separation of classification and risk prediction.

[0005] In a first aspect, the present invention provides a system for classifying aortic stenosis and predicting survival risk, specifically comprising:

[0006] (i) Multimodal data acquisition module for target object; the multimodal data includes a three-dimensional image of the heart, a fibrocalcification spatial distribution mask generated based on the three-dimensional image of the heart, and clinical data; wherein, the fibrocalcification spatial distribution mask is a binary three-dimensional mask used to indicate the spatial location of fibrotic tissue and calcification tissue in the aortic valve;

[0007] (ii) Typing and Risk Prediction Module; the multimodal data serves as input to the prediction model, yielding the aortic stenosis typing result of the target object and the quantitative survival risk value of the target object within a preset time window after transcatheter aortic valve replacement; the prediction model includes an image processing module, a data processing module, an information fusion module, a typing output layer, and a risk output layer; wherein:

[0008] The image processing module is used to guide the multi-scale feature extraction and sequence encoding of local regions of the three-dimensional image of the heart using the spatial distribution mask of fibrous calcification, so as to obtain image features.

[0009] The data processing module is used to perform missing-aware item-by-item encoding and feature screening on the clinical data to obtain the first data feature corresponding to the subtyping task and the second data feature corresponding to the risk prediction task.

[0010] The information fusion module is used to fuse the image features, the first data features and the second data features, and output the aortic stenosis classification result of the target object through the classification output layer, and output the survival risk quantification value through the risk output layer.

[0011] Optionally, the prediction model is trained in the following manner:

[0012] A training sample set is obtained, wherein each training sample in the training sample set contains multimodal data of the sample object, aortic stenosis classification of the sample, and survival status of the sample. The multimodal data of the sample includes a three-dimensional image of the sample heart, a sample fibrocalcification spatial distribution mask generated based on the sample three-dimensional image of the sample heart, and sample clinical data.

[0013] An initial prediction model is constructed, which includes the image processing module, the data processing module, the information fusion module, the classification output layer, and the risk output layer;

[0014] The multimodal data of the training sample set is input into the initial prediction model to calculate the total loss function; wherein, the total loss function includes the classification error of the morphology task, the temporal risk error of the risk prediction task, and the sparse constraint term of feature selection in the data processing module.

[0015] Based on the total loss function, the parameters of the initial prediction model are iteratively updated using an optimization algorithm until the convergence condition is met, thus obtaining the prediction model.

[0016] Optionally, obtaining the training sample set includes:

[0017] Obtain raw data for any sample object, including raw three-dimensional cardiac images, raw clinical data, and follow-up data. The follow-up data is used to record indications of whether the sample object has experienced a preset clinical endpoint event within a preset time window after transcatheter aortic valve replacement.

[0018] The original three-dimensional heart image is subjected to image preprocessing including intensity normalization, region of interest cropping and three-dimensional image block sampling to obtain a sample three-dimensional heart image, and the sample three-dimensional heart image is input into a segmentation model to obtain a sample fibrous calcification spatial distribution mask;

[0019] The original clinical data is preprocessed, including standardization, missing value handling, and categorical variable coding, to obtain sample clinical data.

[0020] Based on the clinical data of the samples, the aortic stenosis type of the samples was determined;

[0021] Based on the follow-up data, the survival status of the samples was determined;

[0022] The three-dimensional image of the sample heart, the spatial distribution mask of the sample fibrocalcification, the sample clinical data, the sample aortic valve stenosis classification, and the sample survival status are used as a training sample.

[0023] The training sample set is obtained based on the training samples corresponding to different sample objects.

[0024] Optionally, the segmentation model includes a first segmentation module and a second segmentation module, wherein inputting the three-dimensional image of the sample heart into the segmentation model to obtain a mask for the spatial distribution of sample fibrous calcification includes:

[0025] The three-dimensional image of the heart is input into the first segmentation module to obtain a mask of the valve complex region.

[0026] The mask of the valve complex region is input into the second segmentation module to obtain a tissue segmentation probability map; the tissue segmentation probability map includes the probability distributions corresponding to different tissue categories.

[0027] Based on the density value and spatial coordinates of each voxel in the tissue segmentation probability map, the tissue segmentation probability map is clustered to obtain the tissue threshold range of the sample object.

[0028] Based on the tissue threshold range, voxels with probability values ​​greater than a preset probability threshold in the tissue segmentation probability map are identified as fibrous calcification regions, and voxels with probability values ​​less than or equal to the preset probability threshold in the tissue segmentation probability map are identified as background, thus obtaining the sample fibrous calcification spatial distribution mask.

[0029] Optionally, the image processing module is specifically used for:

[0030] Based on the aforementioned fiber calcification spatial distribution mask, multiple fiber calcification regions are extracted;

[0031] Calculate the bounding box of the multiple fiber calcification regions, and take the geometric center of the bounding box as the cutting center;

[0032] Using the cropping center as a reference, a three-dimensional sub-region of a preset size is cropped from the three-dimensional image of the heart as a region of interest, the region of interest including the plurality of fibrous calcification regions;

[0033] The region of interest is input into a spatial feature extraction network to obtain a multi-scale feature map, which includes a first-scale feature map with high spatial resolution and a second-scale feature map with strong semantic information.

[0034] Global average pooling is performed on the second-scale feature map to obtain the global context vector;

[0035] The fiber calcification spatial distribution mask is aligned with the spatial dimensions of the first-scale feature map through interpolation to obtain a first alignment mask, and the fiber calcification spatial distribution mask is aligned with the spatial dimensions of the second-scale feature map through interpolation to obtain a second alignment mask.

[0036] For each of the plurality of fiber calcification regions, determine the input sequence corresponding to the fiber calcification region;

[0037] For any fibrous calcification region, if the mass volume of the fibrous calcification region is less than a volume threshold, the feature markers in the input sequence corresponding to the fibrous calcification region are masked to obtain a set of masked input sequences corresponding to the multiple fibrous calcification regions.

[0038] The masked input sequence set and the global context vector are input into the sequence transformation network to obtain the image features.

[0039] Optionally, determining the input sequence corresponding to the fibrous calcification region includes:

[0040] Based on the first alignment mask, the voxels covered by the fibrous calcification region are determined on the first scale feature map to obtain the first voxel set, and the first semantic features of the fibrous calcification region are determined according to the first voxel set.

[0041] Based on the spatial coordinates of each voxel in the first voxel set, the spatial distribution characteristics of the fiber calcification region are calculated.

[0042] Based on the first voxel set, the morphological characteristics of the fibrous calcification region are calculated.

[0043] By concatenating the first semantic feature, spatial distribution feature, and morphological feature, the first-scale feature label of the fibrous calcification region is obtained.

[0044] Based on the second alignment mask, the voxels covered by the fibrous calcification region are determined on the second scale feature map to obtain the second voxel set, and the second semantic features of the fibrous calcification region are determined based on the second voxel set.

[0045] By concatenating the second semantic features, spatial distribution features, and morphological features, the second-scale feature label of the fibrous calcification region is obtained.

[0046] Add a first-scale embedding to the first-scale feature label of each fibrous calcification region to obtain the processed first-scale feature label; add a second-scale embedding to the second-scale feature label of each fibrous calcification region to obtain the processed second-scale feature label.

[0047] The processed first-scale feature labels, processed second-scale feature labels, and classification labels corresponding to the fibrous calcification region are concatenated along the sequence dimension to form the input sequence of the fibrous calcification region.

[0048] Optionally, the clinical data includes continuous variables and categorical variables, and the data processing module is specifically used for:

[0049] A first embedding representation is generated for each continuous variable, and a corresponding second embedding representation is generated for each categorical variable; wherein, the first embedding representation is used to distinguish between the true measured value and the filled value of the continuous variable, and the second embedding representation is used to distinguish between the true measured value and the filled value of the categorical variable;

[0050] The first embedding representations of all continuous variables and the second embedding representations of all categorical variables are concatenated in a preset order to form the target feature sequence;

[0051] The classification task weights are multiplied element-wise by the target feature sequence to obtain the first filtered feature sequence corresponding to the classification task, and the risk prediction task weights are multiplied element-wise by the target feature sequence to obtain the second filtered feature sequence corresponding to the risk prediction task.

[0052] The first filtered feature sequence is input into the genotyping task fusion network to output the first data feature; the second filtered feature sequence is input into the risk prediction task fusion network to output the second data feature.

[0053] Optionally, the aortic stenosis classification result includes one of high-gradient severe stenosis, low-flow low-pressure gradient stenosis, paradoxical low-flow low-pressure gradient stenosis, and normal-flow low-pressure gradient stenosis.

[0054] In a second aspect, the present invention also provides an electronic device comprising a memory and a processor, the memory being used to store executable instructions; the processor being used to operate under the control of the instructions to perform the functions of the system as described in the first aspect of the present invention.

[0055] In a third aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program performing the functions of the system as described in the first aspect when executed by a processor.

[0056] The main technical features and functional advantages of this invention are as follows:

[0057] (1) By obtaining a mask of the spatial distribution of fibrocalcification generated based on the three-dimensional image of the heart to replace manual delineation, the dependence on manual delineation and inter-observer differences are eliminated.

[0058] (2) The image processing module uses the mask to guide the multi-scale feature extraction and sequence encoding of local regions of the three-dimensional heart image, which can capture the spatial distribution relationship of each fibrocalcified region, thereby quantifying the spatial distribution heterogeneity ignored by traditional methods;

[0059] (3) The data processing module performs item-by-item encoding and feature screening for missing data in clinical data, which can generate special embeddings for missing values, enabling the model to distinguish between true values ​​and filled values, thus solving the problem of rough processing of missing data.

[0060] (4) The information fusion module integrates image features with the first data features and the second data features, and outputs the typing results and survival risk quantification values ​​simultaneously through the typing output layer and the risk output layer, thereby realizing the integration of typing and risk prediction and greatly improving the efficiency of clinical diagnosis and treatment. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of an aortic stenosis classification and survival risk prediction system according to an embodiment of the present invention. Detailed Implementation

[0063] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0064] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0065] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0066] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0067] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0068] A block diagram of the hardware configuration of the electronic device 100 according to an embodiment of the present invention is shown below. Figure 1 As shown, electronic device 100 can be, for example, a PC, a laptop, a server, etc.

[0069] In this embodiment, refer to Figure 1 As shown, the electronic device 100 includes a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and so on.

[0070] Processor 1100 may be a mobile processor. Memory 1200 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. Interface device 1300 includes, for example, a USB interface and a headphone jack. Communication device 1400 is capable of wired or wireless communication. Communication device 1400 may include short-range communication devices, such as any device that performs short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, and LiFi. Communication device 1400 may also include long-range communication devices, such as any device that performs WLAN, GPRS, or 2G / 3G / 4G / 5G long-range communication. Display device 1500 is, for example, an LCD screen or a touch screen. Display device 1500 is used to display the aortic stenosis classification results of the target subject and the quantitative value of the survival risk of the target subject within a preset time window after transcatheter aortic valve replacement. Input device 1600 may include, for example, a touch screen or a keyboard. Users can input / output voice information via speaker 1700 and microphone 1800.

[0071] In this embodiment, the memory 1200 of the electronic device 100 is used to store instructions for controlling the processor 1100 to operate in order to at least execute the aortic stenosis classification and survival risk prediction method according to any embodiment of the present invention. Those skilled in the art can design the instructions according to the disclosed scheme of the present invention. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0072] Despite Figure 1 The present invention illustrates multiple devices of electronic device 100, but may refer to only some of these devices. For example, electronic device 100 may refer only to memory 1200, processor 1100 and display device 1500.

[0073] A flowchart illustrating the aortic stenosis classification and survival risk prediction method of this invention is shown below. Figure 2 As shown, the steps include S2100~S2200:

[0074] Step S2100: Obtain multimodal data of the target object, the multimodal data including a three-dimensional image of the heart, a fibrous calcification spatial distribution mask generated based on the three-dimensional image of the heart, and clinical data; wherein, the fibrous calcification spatial distribution mask is a binary three-dimensional mask used to indicate the spatial location of fibrotic and calcified tissues in the aortic valve.

[0075] In this embodiment, the target group can refer to patients who need to undergo aortic stenosis classification and survival risk prediction, usually patients suspected or diagnosed with aortic stenosis who are planning to undergo or have already undergone transcatheter aortic valve replacement (TAVR).

[0076] Multimodal data can refer to various types of data used to comprehensively assess the status of aortic stenosis and patient prognosis, including three-dimensional cardiac images, fibrous calcification spatial distribution masks, and clinical data.

[0077] Three-dimensional cardiac images can be used to visually represent the anatomical structure and fibrous calcification distribution of the aortic valve. These images can be obtained using 3D volumetric data acquired through computed tomography (CTA) angiography (using a ≥64-slice CT scanner, retrospectively ECG-gated or prospectively ECG-triggered scans, reconstructed with a thin slice thickness of 0.5-0.625 mm to ensure adequate visualization of the aortic root and left ventricular cavity, with a target HU value greater than 350), or other three-dimensional imaging techniques such as magnetic resonance imaging (MRI) that can clearly display the aortic valve structure; no specific limitations are specified here.

[0078] The three-dimensional image of the heart can be the original three-dimensional image of the heart after the first preprocessing. The content of the first preprocessing can be found in step S1100.2 later, and will not be described here.

[0079] A fibrosis and calcification spatial distribution mask can be used to accurately indicate the spatial location of fibrotic and calcified tissue in the aortic valve. It is a binary 3D mask (containing only two voxel values, 0 and 1). Regions with a pixel value of 1 represent the location of fibrotic or calcified tissue, while regions with a pixel value of 0 represent the background (non-fibrotic and calcified areas). This mask is automatically generated based on a segmentation model using a 3D image of the heart, eliminating the need for manual drawing.

[0080] Clinical data can refer to tabular data recording the clinical information of the target subject, including baseline variables, hemodynamic parameters, and laboratory indicators. Baseline variables include age, sex, STS score, history of diabetes, history of hypertension, and history of chronic kidney disease. Hemodynamic parameters may include mean pressure gradient, aortic valve orifice area (AVA), left ventricular ejection fraction (LVEF), stroke volume index (SVI), and NYHA functional class. Laboratory indicators include complete blood count, liver and kidney function tests, and coagulation function tests.

[0081] Clinical data can be the original clinical data after undergoing a second preprocessing step. The second preprocessing step can be referred to in the description of the subsequent step S1100.3, which will not be elaborated here.

[0082] For example, the target patient is a 58-year-old male patient who is scheduled to undergo TAVR surgery. His 3D CTA volume data (scanning parameters: 64-slice CT, slice thickness 0.5 mm, HU value 380), the spatial distribution mask of fibrous calcification generated based on the CTA data (binarized, 1 represents the leaflet calcification area, 0 represents the background), and clinical data (age 58 years, male, STS score 4.2, 10-year history of hypertension, mean pressure gradient 38 mmHg, aortic valve orifice area 0.8 cm², etc.) were obtained.

[0083] Step S2200: Input the multimodal data into the prediction model to obtain the aortic stenosis classification result of the target object and the quantitative value of the survival risk of the target object within a preset time window after transcatheter aortic valve replacement.

[0084] The prediction model includes an image processing module, a data processing module, an information fusion module, a typing output layer, and a risk output layer. The image processing module is used to extract multi-scale features and sequence encode local regions of the three-dimensional cardiac image using the fibrous calcification spatial distribution mask to obtain image features. The data processing module is used to perform missing-aware item-by-item encoding and feature filtering on the clinical data to obtain the first data feature corresponding to the typing task and the second data feature corresponding to the risk prediction task. The information fusion module is used to fuse the image features, the first data feature, and the second data feature, and output the aortic stenosis typing result of the target object through the typing output layer, and output the survival risk quantification value through the risk output layer.

[0085] In this embodiment, the acquired multimodal data is first standardized and preprocessed (in accordance with the preprocessing method of subsequent training samples) to ensure that the data format and scale match the input requirements of the prediction model. Then, the preprocessed 3D cardiac image and the fibrocalcification spatial distribution mask are input into the image processing module, and the clinical data is input into the data processing module. The two modules process the data in parallel and output the corresponding features. Feature fusion is completed by the information fusion module, and finally, the results are output synchronously through the two output layers.

[0086] The prediction model can be an end-to-end multimodal, multi-task network model capable of classifying aortic stenosis and predicting survival risk. It consists of five core modules: image processing, data processing, information fusion, classification output layer, and risk output layer, all working collaboratively. The core function of the image processing module is to focus on local regions related to aortic stenosis (AS) in a three-dimensional cardiac image, guided by a mask of the spatial distribution of fibrous calcification. It extracts multi-scale features, performs sequence encoding, and ultimately outputs image features, thereby capturing the spatial heterogeneity of fibrous calcification distribution.

[0087] The core function of the data processing module is to address the common missing data issues in clinical data by encoding each item and filtering out features useful for classification and risk prediction, and outputting the features corresponding to the two tasks respectively.

[0088] The information fusion module is used to fuse the image features output by the image processing module and the first and second data features output by the data processing module, integrating multi-dimensional information to support subsequent output results.

[0089] The genotyping output layer is used to convert the fused features into aortic stenosis genotyping results, outputting the probability of each genotype and the final genotype.

[0090] The classification of aortic stenosis includes one of four types: high-gradient severe stenosis, low-flow low-pressure gradient stenosis, paradoxical low-flow low-pressure gradient stenosis, and normal-flow low-pressure gradient stenosis.

[0091] The risk output layer is used to convert the fused features into a survival risk quantification value.

[0092] The preset time window can refer to the postoperative follow-up period that is of key clinical concern. The preset time window can be, for example, 1 year (i.e., the survival risk within 1 year after TAVR), or it can be adjusted to 6 months, 2 years, etc., according to clinical needs; there is no limitation here.

[0093] Survival risk quantification values ​​can be used to quantify the probability of adverse clinical events (such as all-cause mortality or heart failure readmission) occurring in a target subject within a predetermined time window after aortic valve replacement surgery. The value ranges from 0 to 1, with higher values ​​indicating a higher survival risk after aortic valve replacement surgery. For example, a survival risk quantification value of 0.2 indicates low risk, 0.5 indicates moderate risk, and 0.8 indicates high risk.

[0094] In one embodiment of the present invention, the prediction model is trained through the following steps S1100 to S1400:

[0095] Step S1100: Obtain the training sample set; each training sample in the training sample set contains multimodal data of the sample object, aortic stenosis classification of the sample, and survival status of the sample. The multimodal data of the sample includes a three-dimensional image of the sample heart, a sample fibrocalcification spatial distribution mask generated based on the three-dimensional image of the sample heart, and sample clinical data.

[0096] In this embodiment, the training sample set consists of multiple training samples, and the size of the training sample set needs to meet the model training requirements (usually no less than 100 cases; the larger the sample size, the stronger the model's generalization ability). Each training sample contains input data (sample multimodal data) and labels (sample aortic valve stenosis classification, sample survival status).

[0097] The sample subjects refer to the patients used to construct the training sample set, whose clinical information, examination data, and follow-up data must be complete. The sample subjects are consistent with the target subjects, namely, patients suspected or diagnosed with AS who have undergone TAVR surgery.

[0098] Sample multimodal data refers to the multimodal data corresponding to the sample object, which is consistent with the multimodal data type of the target object. It includes three-dimensional images of the sample heart, spatial distribution masks of sample fibrous calcification, and sample clinical data. It is named only to distinguish the data of the target object.

[0099] The sample aortic stenosis classification can refer to the actual AS hemodynamic classification of the sample subjects, serving as a training label for the model classification task. The sample aortic stenosis classification includes one of four types: high-gradient severe stenosis, low-flow low-pressure gradient stenosis, paradoxical low-flow low-pressure gradient stenosis, and normal-flow low-pressure gradient stenosis.

[0100] Sample survival status refers to the survival of a sample subject within a pre-defined time window after TAVR surgery. Sample survival status includes two states: occurrence of the pre-defined clinical endpoint event and non-occurrence of the pre-defined clinical endpoint event. Sample survival status can be used as training labels for model risk prediction tasks.

[0101] In one embodiment of the present invention, step S1100 of obtaining the training sample set includes: steps S1100.1 to S1100.7.

[0102] Step S1100.1: Obtain the raw data of any sample object. The raw data includes raw three-dimensional cardiac images, raw clinical data, and follow-up data. The follow-up data is used to record indication information of whether the sample object has experienced a preset clinical endpoint event within a preset time window after transcatheter aortic valve replacement.

[0103] In this embodiment, for each sample object, three types of raw data (sample object-related data directly extracted from each system without any preprocessing) are simultaneously extracted from the hospital image storage system, electronic medical record system, and follow-up management system. After extraction, a preliminary verification is performed to ensure that the data is uniquely associated with the sample object (by matching the patient ID), and sample objects with serious data missing or unable to be verified are removed.

[0104] Among them, raw cardiac 3D images refer to cardiac 3D images obtained directly from imaging equipment such as CT and MRI without standardized processing, which may contain equipment differences, artifacts and other problems.

[0105] Raw clinical data refers to clinical data extracted directly from electronic medical records without cleaning or processing, and may contain outliers, missing values, and other issues.

[0106] Follow-up data refers to data recorded by the sample subjects after TAVR surgery through regular outpatient follow-up and telephone follow-up.

[0107] Preset clinical endpoint events refer to adverse events defined in clinical practice that are related to the patient's postoperative prognosis, including all-cause mortality and heart failure readmission. Other events may be added as needed in clinical practice, and no restrictions are imposed here.

[0108] Indication information can refer to a marker used to indicate whether a pre-defined clinical endpoint event has occurred, usually indicated by "1" for occurrence and "0" for non-occurrence (including cases where the pre-defined follow-up time window has not been completed or where the event has not occurred despite loss to follow-up).

[0109] Step S1100.2: Perform image preprocessing on the original three-dimensional heart image to obtain a sample three-dimensional heart image, and input the sample three-dimensional heart image into the segmentation model to obtain a sample fibrous calcification spatial distribution mask.

[0110] In this embodiment, the original 3D heart image is first preprocessed, including intensity normalization, region of interest cropping, and 3D image block sampling, to eliminate the effects of device differences and image artifacts, resulting in a standardized sample 3D heart image. The standardized sample 3D heart image is then input into a pre-trained segmentation model, which automatically processes the image and outputs a sample fibrous calcification spatial distribution mask. The entire process requires no manual intervention.

[0111] The segmentation model refers to the model used to automatically segment the fibrocalcified region of the aortic valve and generate a mask for the spatial distribution of fibrocalcification. The segmentation model is pre-trained and has high segmentation accuracy.

[0112] The segmentation model can be a dual-channel cascaded 3D nnUNet architecture.

[0113] Segmentation models can be obtained through common model training methods, which will not be elaborated here.

[0114] The sample fibrocalcification spatial distribution mask refers to the binary 3D mask obtained after the segmentation model processes the sample heart 3D image. It is consistent with the definition of the target object's fibrocalcification spatial distribution mask and is used to mark the spatial location of fibrocalcified tissue in the aortic valve of the sample object.

[0115] The segmentation model includes a first segmentation module and a second segmentation module. In step S1100.2, the three-dimensional image of the sample heart is input into the segmentation model to obtain a sample fibrous calcification spatial distribution mask, including steps S1100.21 to S1100.24.

[0116] Step S1100.21: Input the three-dimensional image of the heart into the first segmentation module to obtain a mask of the valve complex region.

[0117] In this embodiment, the three-dimensional image of the sample heart after the first preprocessing (i.e., the three-dimensional image of the sample heart obtained in step S1100.2) is input into the first segmentation module (the first segmentation module can be, for example, the first channel of a dual-channel cascaded 3D nnUNet architecture). The first segmentation module identifies and segments the valvular complex region, including the left coronary lobes, right coronary lobes, non-coronary lobes, annulus, and Valsalva sinus, through its network architecture (e.g., a 3D nnUNet architecture), and outputs a valvular complex region mask. The valvular complex region mask is used to define the range of subsequent tissue segmentation and reduce background interference.

[0118] The valvular complex region refers to the core anatomical region related to the aortic valve, including the leaflets, annulus, left ventricular outflow tract, and Valsalva sinus, and is the main distribution area of ​​fibrotic calcification tissue.

[0119] The valve complex region mask refers to the mask used to mark the valve complex region. It is a binary mask, where a pixel value of 1 represents the valve complex region and 0 represents the non-complex region.

[0120] Step S1100.22: Input the mask of the valve complex region into the second segmentation module to obtain a tissue segmentation probability map; the tissue segmentation probability map includes the probability distributions corresponding to different tissue categories.

[0121] In this embodiment, the mask of the valve complex region is input into the second segmentation module. The second segmentation module processes the valve complex region and classifies and identifies the tissue within the region through multi-scale feature extraction and attention gating mechanisms, outputting a tissue segmentation probability map. Each voxel in the tissue segmentation probability map corresponds to a probability value for a different tissue category; a higher probability value indicates a greater likelihood that the voxel belongs to the corresponding tissue category.

[0122] For example, in the tissue segmentation probability map, the probability of a certain voxel corresponding to fibrotic tissue is 0.85, calcified tissue is 0.12, and background is 0.03, indicating that the voxel is likely to be fibrotic tissue.

[0123] The tissue categories can include three types: background, fibrotic tissue, and calcified tissue, without any specific limitation here.

[0124] Step S1100.23: Based on the density value and spatial coordinates of each voxel in the tissue segmentation probability map, perform clustering processing on the tissue segmentation probability map to obtain the tissue threshold range of the sample object.

[0125] In this embodiment, two core parameters of each voxel in the tissue segmentation probability map are extracted: density value (i.e., HU value in CT image) and spatial coordinates. An unsupervised clustering algorithm (e.g., adaptive spectral clustering algorithm) is used to cluster all voxels, grouping voxels with similar density values ​​and spatial coordinates into one class, thereby determining the individualized tissue threshold range for each sample object.

[0126] The tissue threshold range refers to the range of HU values ​​used to distinguish different tissues, mainly targeting fibrotic and calcified tissues. That is, the tissue threshold range of the sample includes the HU value threshold range of fibrotic tissue and the HU value threshold range of calcified tissue, which solves the problem of HU value drift caused by different equipment and contrast agent concentrations.

[0127] The HU value varies among different tissues, with calcified tissue typically having a higher HU value than fibrotic tissue. For example, the HU value range for fibrotic tissue is 150-300, for calcified tissue it is 300-1000, and for background and blood it is -1000-150.

[0128] The threshold range for each sample object can be adaptively adjusted based on its own data, rather than using a fixed threshold.

[0129] Spatial coordinates refer to the three-dimensional coordinates (x, y, z) of a voxel in a three-dimensional image, which are used to reflect the spatial position of the voxel.

[0130] Step S1100.24: Based on the tissue threshold interval, voxels with probability values ​​greater than a preset probability threshold in the tissue segmentation probability map are identified as fibrous calcification regions, and voxels with probability values ​​less than or equal to the preset probability threshold in the tissue segmentation probability map are identified as background, thereby obtaining the sample fibrous calcification spatial distribution mask.

[0131] In this embodiment, a preset probability threshold is first determined (e.g., 0.5, which can be adjusted according to the model's segmentation accuracy). Then, combined with the tissue threshold interval, each voxel in the tissue segmentation probability map is determined: if the probability value of a voxel's fibrotic tissue is greater than the preset probability threshold, and the HU value of the voxel is within the tissue threshold interval corresponding to the fibrotic tissue of the target object, then the voxel is determined to be a fibrotic tissue region (marked as 1). If the probability value of a voxel's calcified tissue is greater than the preset probability threshold, and the HU value of the voxel is within the tissue threshold interval corresponding to the calcified tissue of the target object, then the voxel is determined to be a calcified tissue region (marked as 1). Thus, a fibrocalcified region composed of fibrotic tissue region and calcified tissue region is obtained. If the probability values ​​of both fibrotic tissue and calcified tissue of a voxel are less than or equal to the preset probability threshold, or if the HU value of the voxel is not within the threshold interval of calcified tissue and fibrotic tissue, then it is determined to be background (marked as 0). After all voxels are determined, a binary sample fibrocalcified spatial distribution mask is formed. Finally, outlier cleanup was performed on the binarized sample fiber calcification spatial distribution mask (NaN / Inf values ​​were replaced with 0) to ensure mask quality.

[0132] The preset probability threshold refers to the critical probability value used to distinguish the fibrous calcification area from the background, and its value ranges from 0 to 1.

[0133] The sample fiber calcification spatial distribution mask, which is the final generated binary three-dimensional mask used to mark the spatial location of the fiber calcification tissue of the sample object, is completely consistent with the fiber calcification mask format and definition of the target object, and will not be elaborated here.

[0134] Example: With a preset probability threshold of 0.5, and considering the tissue threshold range of the sample object (fibrosis 150-300 HU, calcification 300-1000 HU), the voxels in the tissue segmentation probability map are determined: a voxel with a calcification probability of 0.91 and a HU value of 380, which is greater than 0.5 and within the calcification threshold range, is determined to be a fibrous calcification region (marked 1); a voxel with a fibrous tissue probability of 0.45 and a HU value of 220, which is less than 0.5, is determined to be the background (marked 0); finally, a sample fibrous calcification spatial distribution mask is obtained.

[0135] Step S1100.3: Perform a second preprocessing on the original clinical data to obtain sample clinical data.

[0136] In this embodiment, the original clinical data is systematically cleaned and standardized, specifically including three core steps: outlier handling, variable classification, and missing value labeling. After processing, sample clinical data is obtained, ensuring that the data meets the requirements for model training while preserving the authenticity and integrity of the data.

[0137] Abnormal values ​​refer to values ​​that exceed the clinically reasonable range, such as blood pressure >200 / 150 mmHg, left ventricular ejection fraction (LVEF) >80%, etc. Such values ​​may be due to measurement errors or recording mistakes and should be deleted.

[0138] Variable classification refers to dividing clinical data into continuous variables (age, blood pressure, LVEF, SVI, etc.) and categorical variables (gender, history of diabetes, NYHA classification, etc.) to facilitate subsequent coding processing.

[0139] Specifically, based on the type of attributes of clinical data, they are divided into two categories: continuous variables and categorical variables. After classification, each category is labeled to facilitate targeted coding processing and ensure the rationality and accuracy of the coding. Continuous variables refer to clinical indicators with continuous numerical values ​​that can be quantified, reflecting specific numerical differences in the indicators. Categorical variables refer to clinical indicators with discrete categories that cannot be quantified; they are typically used to distinguish different clinical states and are further divided into binary and multi-category variables.

[0140] Example: In clinical data, age (58 years), mean pressure gradient (38 mmHg), aortic valve orifice area (0.8 cm²), and left ventricular ejection fraction (55%) are continuous variables; gender (male / female), history of hypertension (present / absent), and NYHA functional class (I / II / III / IV) are categorical variables, with gender being a binary variable and NYHA functional class being a multi-category variable.

[0141] For continuous variables with missing values, first calculate the mean and standard deviation of the continuous variable in the training sample set. Use the mean of the continuous variable to fill in the missing values. Then, standardize all values ​​of the continuous variable (including true measurements and filled values) using the mean and standard deviation, and add missing value labels to each continuous variable. The missing value labels for filled continuous variables indicate that the values ​​of the continuous variable are missing. The missing value labels for continuous variables with true measurements indicate that the values ​​of the continuous variable are not missing. In other words, the missing value labels indicate the missing value status of the continuous variable.

[0142] Add missing value markers to all values ​​in the categorical variables. If a categorical variable has missing values, no numerical imputation is performed; instead, missing value markers are added to indicate that the categorical variable has a missing value.

[0143] For example, a missing information of 0 indicates a missing value, while a missing information of 1 indicates that there is no missing value.

[0144] For example, the original clinical data of the aforementioned 62-year-old female patient underwent a second preprocessing step: outliers were removed (e.g., a recorded systolic blood pressure of 210 mmHg, which exceeded the reasonable range and was therefore removed). Age, mean blood pressure difference, and LVEF were classified as continuous variables, while gender, history of diabetes, and NYHA functional class were classified as categorical variables. Missing values ​​of categorical variables were marked as 0, and the true measurements of categorical variables were marked as 1. Missing values ​​of continuous variables were also marked as 0, and the true measurements of continuous variables were marked as 1, ultimately yielding the sample clinical data.

[0145] Step S1100.4: Based on the clinical data of the sample, determine the aortic valve stenosis type of the sample.

[0146] In this embodiment, the hemodynamic classification of aortic stenosis includes the following four categories: Category 1 is high-gradient severe stenosis; Category 2 is low-flow, low-pressure gradient stenosis; Category 3 is paradoxical low-flow, low-pressure gradient stenosis; and Category 4 is normal-flow, low-pressure gradient stenosis. The determination method for each category is as follows:

[0147] The criteria for severe stenosis are: mean differential pressure greater than or equal to the differential pressure threshold (e.g., 40 mmHg), valve orifice area less than the area threshold (e.g., 1.0), and ejection fraction greater than or equal to the functional threshold (e.g., 50%). The criteria for low-flow, low-differential-pressure stenosis are: valve orifice area less than the area threshold, mean differential pressure less than the differential pressure threshold, ejection fraction less than the functional threshold (e.g., 50%), and cardiac output index less than or equal to the flow threshold (e.g., 35 mL / m²). The criteria for paradoxical low-flow, low-differential-pressure stenosis are: valve orifice area less than the area threshold, mean differential pressure less than the differential pressure threshold, ejection fraction greater than or equal to the functional threshold, and cardiac output index less than or equal to the flow threshold. The criteria for normal-flow, low-differential-pressure stenosis are: valve orifice area less than the area threshold, mean differential pressure less than the differential pressure threshold, ejection fraction greater than or equal to the functional threshold, and cardiac output index greater than the flow threshold.

[0148] Step S1100.5: Determine the survival status of the sample based on the follow-up data.

[0149] In this embodiment, the survival status of the sample is determined based on the clinical endpoint event indication information in the follow-up data (marked as 1 if a preset clinical endpoint event occurs, and marked as 0 if no event occurs).

[0150] Step S1100.6: Use the three-dimensional image of the sample heart, the spatial distribution mask of the sample fibrocalcification, the sample clinical data, the sample aortic valve stenosis classification, and the sample survival status as a training sample.

[0151] Step S1100.7: Obtain the training sample set according to the training samples corresponding to different sample objects.

[0152] In this embodiment, steps S1100.1 to S1100.6 are executed for each sample object to construct a corresponding training sample for each sample object, and all training samples are summarized to form a training sample set.

[0153] In one example, the training sample set can be initially divided (e.g., the training subset and the validation subset can be divided in a 7:3 ratio) to prepare for subsequent model training and validation.

[0154] Step S1200: Construct an initial prediction model, which includes the image processing module, the data processing module, the information fusion module, the classification output layer, and the risk output layer.

[0155] In this embodiment, an initial prediction model is constructed based on a deep learning framework (such as mainstream frameworks like PyTorch and TensorFlow), clarifying the model's network structure, parameter settings for each module, and input / output formats.

[0156] The image processing module and the data processing module are set up in parallel. The information fusion module is connected to the output of the two processing modules (i.e., the image processing module and the data processing module). The classification output layer and the risk output layer are connected to the output of the information fusion module, forming an end-to-end multimodal multitasking network structure.

[0157] The parameters of the initial model are randomly initialized and then gradually optimized through training.

[0158] For example, an initial prediction model is constructed based on the PyTorch framework. The image processing module includes a 3D CNN spatial feature extraction network and a Transformer sequence transformation network. The data processing module includes an embedding layer and a Hard-Concrete L0 feature selection layer. The information fusion module uses a fully connected layer to achieve feature splicing and fusion. The genotyping output layer is a 4-class linear layer, and the risk output layer is a regression linear layer. The parameters of each module are randomly initialized. The model input is multimodal data of the samples, and the output is the sample genotyping and survival risk value.

[0159] Step S1300: Input the multimodal data of the training sample set into the initial prediction model and calculate the total loss function; wherein, the total loss function includes the classification error of the morphology task, the temporal risk error of the risk prediction task, and the sparse constraint term of feature selection in the data processing module.

[0160] In this embodiment, multimodal data (3D images of the heart, fibrocalcification masks, and clinical data) from the training sample set are input in batches into the initial prediction model. The model processes these data through various modules, outputting predicted genotyping results and predicted survival risk quantification values. The prediction results are compared with the sample labels (aortic stenosis genotyping and survival status). The classification error for the genotyping task, the temporal risk error for the risk prediction task, and the sparsity constraint term for feature selection are calculated separately. These three factors are then fused using weighting coefficients to obtain the total loss function. The value of the total loss function reflects the degree of deviation between the model's prediction results and the true labels; the greater the deviation, the higher the loss value.

[0161] The total loss function, used to measure the model's prediction error, is the core basis for updating model parameters and is a weighted fusion of classification error, temporal risk error, and sparse constraint terms. Classification error refers to the deviation between the model's predicted classification and the sample's aortic stenosis classification in the genotyping task; it can be calculated using the cross-entropy loss function and used to optimize the model's classification accuracy. Temporal risk error refers to the deviation between the model's predicted survival risk and the sample's survival status in the risk prediction task; it can be specifically calculated using the Cox partial likelihood loss function and used to optimize the model's risk prediction accuracy (while considering temporal characteristics). The sparse constraint term refers to the loss term used to constrain the feature selection process in the data processing module; it can specifically use the L0 penalty loss to reduce redundant features, avoid model overfitting, and ensure the model's generalization ability. The weight coefficients of the weighted fusion can be adaptively adjusted according to the model's training effect.

[0162] Step S1400: Based on the total loss function, the parameters of the initial prediction model are iteratively updated using an optimization algorithm until the convergence condition is met, thus obtaining the prediction model.

[0163] In this embodiment, a preset optimization algorithm is used to iteratively update all learnable parameters of the initial prediction model based on the gradient direction of the total loss function (backpropagation update). After each iteration, the total loss function value is calculated to determine whether the convergence condition is met. If not, the iteration continues; if it is met, the iteration stops, and the model at this point is the trained prediction model. The optimization algorithm can use the AdamW optimizer (which has weight decay function to avoid overfitting), or other optimization algorithms such as SGD and Adagrad; no limitation is made here.

[0164] Convergence criteria refer to the standards used to judge whether the model is mature. It can be "the total loss function value no longer decreases for 10 consecutive iterations (epochs), and the decrease is less than a preset threshold (such as 1e-5)", or it can be that the number of iterations reaches a preset maximum value (such as 200 epochs).

[0165] In another embodiment of the present invention, the method further includes a model validation step: steps S3100 to S3300, used to validate the performance of the model, ensure the predictive accuracy and generalization ability of the model, and provide a basis for clinical application.

[0166] Step S3100: Divide the training sample set into a training subset and a validation subset according to a preset ratio.

[0167] In this embodiment, a random partitioning method is used to divide the training sample set into a training subset and a validation subset according to a preset ratio. During the partitioning process, it is ensured that the sample distribution of the two subsets is consistent (i.e., the sample proportions of each AS subtype and the sample proportions of survival states are consistent with the original training sample set), avoiding deviations in validation results caused by uneven sample distribution. The preset ratio is preferably set to 7:3 (70% training subset and 30% validation subset), but it can also be adjusted to 8:2 depending on the size of the sample set; no limitation is made here.

[0168] Step S3200: Update the model parameters using the training subset, and periodically evaluate the model performance on the validation subset.

[0169] In this embodiment, the training subset is input into the initial prediction model, and the model parameters are updated according to steps S1300 to S1400. During model training, the validation subset is periodically (preferably every one iteration cycle) input into the model at the current training stage. The model outputs prediction results, and the evaluation metrics for the classification task and risk prediction task are calculated by combining the true labels of the validation subset. The model parameters (such as learning rate and weight coefficients) are adjusted according to the evaluation metrics to ensure that the model performs optimally on the validation subset.

[0170] Evaluation metrics for classification tasks include accuracy and harmonic mean score, while evaluation metrics for risk prediction tasks include ranking consistency index and calibration index.

[0171] Step S3300: Construct a base model containing only clinical baseline variables, and calculate the classification improvement index of the prediction model relative to the base model to verify the added value of image features.

[0172] In this embodiment, a base model is constructed. The input to this model consists only of clinical baseline variables (excluding image features corresponding to the three-dimensional cardiac image and the spatial distribution mask of fibrous calcification). The model structure is consistent with the risk prediction task structure of the prediction model. The validation subset is input into the base model, and the risk prediction evaluation index (C-index, calibration index) of the base model is calculated. Then, the classification improvement index (NRI) of the prediction model relative to the base model is calculated. If the NRI ≥ 0.2, it indicates that image features can significantly improve the predictive performance of the model, verifying the added value of image features. The classification improvement index (NRI) is an index used to measure the difference in predictive performance between two models, with a value ranging from -1 to 1. A positive number indicates that the prediction model is superior to the base model; the larger the value, the more significant the improvement. An NRI ≥ 0.2 indicates that the added value of image features is significant.

[0173] In one embodiment, the image processing module is specifically used to perform the following steps S4100 to S4900. The core purpose is to use a mask to guide the extraction and encoding of multi-scale features of the fibrous calcification region in the three-dimensional image of the heart, thereby obtaining image features and providing support for subsequent fusion and prediction.

[0174] Step S4100: Based on the fiber calcification spatial distribution mask, extract multiple fiber calcification regions.

[0175] In this embodiment, a fibrous calcification spatial distribution mask (either a mask for the target object or a mask for the sample object) is input. A connected component analysis algorithm is used to identify all connected regions in the mask with a pixel value of 1. Each connected region is an independent fibrous calcification region. All connected regions are extracted to obtain multiple fibrous calcification regions. Simultaneously, the location and extent of each fibrous calcification region are recorded to provide a basis for subsequent processing. Here, a fibrous calcification region refers to a connected region in the mask with a pixel value of 1. Each region corresponds to an independent fibrous calcification cluster, which may be a cluster of fibrous tissue, a cluster of calcified tissue, or a mixture of both.

[0176] Example: Input a mask of the spatial distribution of fibrous calcification of the target object, and use a connected component analysis algorithm to identify 3 independent connected regions (located in the left coronal lobe, right coronal lobe, and annulus), thus extracting 3 fibrous calcification regions.

[0177] Step S4200: Calculate the bounding box of the plurality of fiber calcification regions, and take the geometric center of the bounding box as the cutting center.

[0178] In this embodiment, for each fibrous calcification region, its three-dimensional bounding box (i.e., the smallest three-dimensional cuboid that can completely enclose the fibrous calcification region) is calculated. Then, the overall bounding box of all bounding boxes (the smallest three-dimensional cuboid that can completely enclose all fibrous calcification regions) is calculated. Finally, the geometric center of the overall bounding box is calculated.

[0179] Continuing with the example above, the bounding boxes of the three fiber calcification regions have three-dimensional coordinate ranges of x∈[120,180], y∈[90,150], and z∈[60,120], with their geometric center coordinates being (cz=90, cy=120, cx=150), which is the cutting center.

[0180] Step S4300: Using the cropping center as a reference, a three-dimensional sub-region of a preset size is cropped from the three-dimensional image of the heart as a region of interest, the region of interest including the plurality of fibrous calcification regions.

[0181] In this embodiment, a preset size is first determined (set according to the resolution of the 3D cardiac image and the size of the fibrous calcification region, for example, 64×64×64 voxels, which can be adjusted according to the actual situation). Using the cropping center as a reference, the preset size is expanded by half in each of the x, y, and z directions to crop the corresponding 3D sub-region from the 3D cardiac image. If the cropping range exceeds the boundary of the 3D cardiac image, the excess portion is filled with 0s to ensure that the size of the cropped region of interest is uniform and includes all fibrous calcification regions, reducing interference from the background region. The preset size refers to the 3D size (number of voxels) of the region of interest, set to a fixed value to ensure that the size of the region of interest is consistent across all samples, facilitating model processing. The region of interest (ROI) refers to the local 3D sub-region cropped from the 3D cardiac image that contains all fibrous calcification regions.

[0182] Continuing with the example above, with a preset size of 64×64×64 voxels, and using the cropping center (90,120,150) as a reference, extend 32 voxels in each of the x, y, and z directions to crop out a three-dimensional sub-region x∈[118,182], y∈[88,152], and z∈[58,122]. Fill the parts that exceed the boundaries of the original image with 0 to obtain the region of interest, which contains 3 fibrous calcification regions.

[0183] Step S4400: Input the region of interest into the spatial feature extraction network to obtain a multi-scale feature map, which includes a first-scale feature map with high spatial resolution and a second-scale feature map with strong semantic information.

[0184] In this embodiment, the region of interest (ROI) is input into a pre-defined spatial feature extraction network (such as a 3D CNN encoder, composed of residual blocks and learnable downsampling layers; a 3D CBAM attention module may also be incorporated into this encoder). The spatial feature extraction network extracts multi-scale features of the ROI through multiple rounds of convolution and downsampling operations, outputting feature maps at two core scales: a first-scale feature map and a second-scale feature map. These two feature maps work together to comprehensively capture the feature information of the fibrous calcification region.

[0185] Multi-scale feature maps refer to feature maps with different spatial resolutions and semantic information. Multi-scale features can capture the features of the target region more comprehensively, avoiding the limitations of single-scale features. The first-scale feature map, i.e., the high spatial resolution feature map, retains detailed information of the fibrous calcification area (such as the edges of calcified masses and microcalcification points), but its semantic information is weak (it is difficult to distinguish the pathological differences between fibrosis and calcification). The second-scale feature map, i.e., the low spatial resolution feature map, has undergone multiple rounds of downsampling, resulting in reduced detailed information, but its semantic information is strong (it can clearly distinguish fibrotic tissue, calcified tissue, and the background).

[0186] Continuing with the example above, the region of interest is input into a 3D CNN encoder. The encoder outputs a first-scale feature map (32×32×32 voxels, high spatial resolution) and a second-scale feature map (8×8×8 voxels, strong semantic information) through four rounds of convolution and downsampling operations. The first-scale feature map can clearly see the edge details of calcified masses, and the second-scale feature map can distinguish between calcified tissue and fibrotic tissue.

[0187] Step S4500: Perform global average pooling on the second scale feature map to obtain the global context vector.

[0188] In this embodiment, a global average pooling operation is employed to perform average pooling on each channel of the second-scale feature map. The average of all voxel values ​​in each channel is then calculated to obtain a scalar. These scalars from all channels form the global context vector. This global context vector captures the overall features of the region of interest, thereby supplementing the global information of the fibrous calcification region and avoiding the limitations of local features.

[0189] Continuing with the example above, the second-scale feature map consists of 8×8×8 voxels and 64 channels. The average of the 64 voxel values ​​for each channel yields 64 scalars, which together form a 64-dimensional global context vector. This vector reflects the overall characteristics of fibrous calcification within the region of interest.

[0190] Step S4600: Align the spatial distribution mask of fiber calcification with the spatial dimensions of the first scale feature map through interpolation to obtain a first alignment mask; and align the spatial distribution mask of fiber calcification with the spatial dimensions of the second scale feature map through interpolation to obtain a second alignment mask.

[0191] In this embodiment, since the spatial dimensions of the fiber calcification spatial distribution mask are inconsistent with the dimensions of the first-scale feature map and the second-scale feature map (the feature map has been downsampled and its size is smaller than the original mask), a nearest neighbor interpolation algorithm is used to interpolate the original fiber calcification mask, adjusting its size to be completely consistent with the spatial dimensions of the first-scale feature map and the second-scale feature map, resulting in a first aligned mask and a second aligned mask. The aligned mask corresponds one-to-one with the spatial coordinates of the corresponding feature map, ensuring that the features of the fiber calcification region can be accurately extracted from the feature map based on the mask.

[0192] The interpolation operation can use the nearest neighbor interpolation algorithm (which is fast and can preserve the binarization properties of the mask) or the bilinear interpolation algorithm; there is no limitation here.

[0193] The first alignment mask refers to a fibrous calcification mask with the same size as the first-scale feature map, used to extract features of the fibrous calcification region on the first-scale feature map. The second alignment mask refers to a fibrous calcification mask with the same size as the second-scale feature map, used to extract features of the fibrous calcification region on the second-scale feature map.

[0194] Continuing with the example above, the original fiber calcification mask has a size of 256×256×256 voxels, the first-scale feature map has a size of 32×32×32 voxels, and the second-scale feature map has a size of 8×8×8 voxels. Using the nearest neighbor interpolation algorithm, the original mask is interpolated to 32×32×32 voxels (to obtain the first aligned mask) and 8×8×8 voxels (to obtain the second aligned mask), respectively, to ensure that the aligned mask has the same size and coordinates as the corresponding feature map.

[0195] Step S4700: For each of the plurality of fiber calcification regions, determine the input sequence corresponding to the fiber calcification region.

[0196] In this embodiment, the input sequence is used for subsequent sequence encoding to capture the multi-dimensional features of the fibrous calcification region. Multi-dimensional features refer to features reflecting multiple dimensions of the fibrous calcification region, such as semantics, space, and morphology, and can comprehensively characterize the properties of the fibrous calcification region.

[0197] In one embodiment of the present invention, step S4700 determines the input sequence corresponding to the fibrous calcification region, including steps S4700.1 to S4700.9.

[0198] Step S4700.1: Based on the first alignment mask, determine the voxels covered by the fibrous calcification region on the first scale feature map to obtain a first voxel set, and determine the first semantic feature of the fibrous calcification region according to the first voxel set.

[0199] In this embodiment, for a specific fibrous calcification region, the position of the region on the first-scale feature map is located according to the first alignment mask. Voxels on the first-scale feature map corresponding to voxels with a pixel value of 1 in the first alignment mask are extracted to form a first voxel set. The average value of the feature values ​​of all voxels in the first voxel set is calculated, and this average value is used as the first semantic feature of the fibrous calcification region.

[0200] The first voxel set refers to the set of all voxels covered by the fibrous calcification region on the first-scale feature map. Each voxel corresponds to a feature vector of the first-scale feature map, which can reflect the high-resolution detail features of the region.

[0201] The first semantic feature can reflect the detailed semantic features of the fibrous calcification region. It is a one-dimensional vector with the same dimension as the number of channels in the first-scale feature map. It can capture detailed semantic information such as the edge and texture of the fibrous calcification region, providing detailed support for subsequent feature fusion.

[0202] Continuing the example above, for a specific fibrous calcification region (the left coronal flap calcification region), based on the first alignment mask, voxels with a mask pixel value of 1 are extracted from the 32×32×32 voxel first-scale feature map, forming the first voxel set (containing 128 voxels). The average of the feature vectors of these 128 voxels is calculated to obtain a one-dimensional vector with the same number of channels as the first-scale feature map (assumed to be 128-dimensional). This vector is the first semantic feature of the fibrous calcification region, which can reflect the edge detail semantics of the left coronal flap calcification region.

[0203] Step S4700.2: Calculate the spatial distribution characteristics of the fiber calcification region based on the spatial coordinates of each voxel in the first voxel set.

[0204] In this embodiment, the three-dimensional spatial coordinates of each voxel in the first voxel set are extracted, and the spatial distribution characteristics of the fiber calcification region are calculated based on these coordinates. The spatial distribution characteristics include three core parameters: spatial centroid coordinates, spatial dispersion, and spatial extension direction. These three parameters are concatenated to form a spatial distribution feature vector.

[0205] The spatial centroid coordinates refer to the average value of the spatial coordinates of all voxels in the first voxel set, reflecting the center position of the fiber calcification region, and complementing the cutting center in step S4200.

[0206] Spatial dispersion refers to the standard deviation of all voxels in the first voxel set relative to the coordinates of the spatial centroid, reflecting the degree of dispersion of the calcified region of the fiber. The larger the standard deviation, the more dispersed the calcified masses are.

[0207] The spatial extension direction refers to the principal component direction vector obtained by analyzing the spatial coordinates of the first voxel set through principal component analysis (PCA), which reflects the main extension trend of the calcified region of the fiber.

[0208] Continuing with the example above, for the first set of voxels in the calcified region of the left coronal flap, the spatial coordinates of each voxel are extracted, and the spatial centroid coordinates are calculated to be (152, 121, 92), the spatial dispersion is 3.8, and the spatial extension direction vector is (0.72, 0.25, 0.64). The three parameters are concatenated to form a 3+1+3=7-dimensional spatial distribution feature vector, which is the spatial distribution feature of the fibrous calcified region.

[0209] Step S4700.3: Calculate the morphological characteristics of the fibrous calcification region based on the first voxel set.

[0210] In this embodiment, the morphological features of the fibrous calcification region are calculated based on the first voxel set. The morphological features include four key indicators: region volume, surface area, sphericity, and aspect ratio. These four indicators are standardized and concatenated to form a morphological feature vector. The region volume refers to the total number of voxels contained in the first voxel set, multiplied by the volume of a single voxel (calculated based on the resolution of the 3D heart image) to obtain the actual volume of the fibrous calcification region. The surface area refers to the area formed by the voxels on the surface of the fibrous calcification region, reflecting the surface roughness of the calcification mass. Sphericity refers to the ratio of the volume of the fibrous calcification region to the surface area of ​​a sphere of the same volume, ranging from 0 to 1; the closer the value is to 1, the closer the calcification mass is to a sphere. The aspect ratio refers to the ratio of the longest side to the shortest side of the bounding box of the fibrous calcification region, reflecting the degree of stretching of the calcification mass.

[0211] Continuing with the example above, for the calcified area of ​​the left coronary flap, the calculated area volume is 12.6 mm³, the surface area is 28.3 mm², the sphericity is 0.68, and the aspect ratio is 1.8. After standardizing the four indicators to the [0,1] interval, they are spliced ​​together to form a 4-dimensional morphological feature vector, which is the morphological feature of the fibrous calcified area.

[0212] Step S4700.4: The first semantic feature, the spatial distribution feature, and the morphological feature are concatenated to obtain the first scale feature label of the fibrous calcification region.

[0213] In this embodiment, a feature concatenation operation is used to concatenate the first semantic feature, the spatial distribution feature, and the morphological feature in a preset order (e.g., "first semantic feature → spatial distribution feature → morphological feature") into a unified feature vector, which is the first scale feature marker of the fibrous calcification region.

[0214] Among them, the first-scale feature label can focus on the local detailed features of the fibrous calcification region, including the detailed semantics, spatial location and geometric morphology information of the region, which is a comprehensive feature representation of the region at the first scale.

[0215] Continuing with the example above, the first semantic feature of a certain fibrous calcification region is a 128-dimensional vector (from the first-scale feature map of 32×32×32 voxels), the spatial distribution feature is a 7-dimensional vector (including three-dimensional center coordinates, relative position with other fibrous calcification regions, etc.), and the morphological feature is a 4-dimensional vector (including volume, surface area, sphericity, etc.). Through feature concatenation, the three types of features are connected in sequence to obtain a first-scale feature label with a dimension of 128+7+4=139.

[0216] Step S4700.5: Based on the second alignment mask, determine the voxels covered by the fibrous calcification region on the second scale feature map to obtain a second voxel set, and determine the second semantic features of the fibrous calcification region according to the second voxel set.

[0217] In this embodiment, for the fibrous calcification region, its position on the second-scale feature map is located according to the second alignment mask. Voxels corresponding to voxels with a pixel value of 1 in the mask are extracted from the second-scale feature map to form a second voxel set. The average feature value of all voxels in the second voxel set is calculated, and this average value is used as the second semantic feature of the fibrous calcification region. The second voxel set refers to the set of all voxels covered by the fibrous calcification region on the second-scale feature map. The second voxel set has fewer voxels than the first voxel set, but each voxel contains stronger semantic information.

[0218] The second semantic feature can reflect the overall semantic features of the fibrotic calcification region. It is a one-dimensional vector with the same dimension as the number of channels in the second-scale feature map. It can capture the overall semantic information of the pathological type of the fibrotic calcification region (such as fibrosis-predominant or calcification-predominant) and complement the first semantic feature.

[0219] Continuing with the example above, for the calcified region of the left coronary fossa, based on the second alignment mask, voxels with a mask pixel value of 1 are extracted from the second-scale feature map of 8×8×8 voxels to form a second voxel set (containing 16 voxels). The average value of the feature vectors of these 16 voxels is calculated to obtain a one-dimensional vector with the same number of channels as the second-scale feature map (assumed to be 64-dimensional). This vector is the second semantic feature of the fibrocalcified region, which can reflect the overall pathological semantics of the left coronary fossa calcified region, which is dominated by calcified tissue.

[0220] Step S4700.6: The second semantic feature, the spatial distribution feature, and the morphological feature are spliced ​​together to obtain the second scale feature label of the fibrous calcification region.

[0221] The second semantic feature, along with the spatial distribution feature and morphological feature, are concatenated in a predetermined order (e.g., "second semantic feature → spatial distribution feature → morphological feature") to form a unified feature vector, which is the second-scale feature label of the fibrous calcification region. The role of the second-scale feature label is to integrate the multi-dimensional global semantic features of the fibrous calcification region at the second scale, forming a multi-scale complementarity with the first-scale feature label, and comprehensively characterizing the properties of the fibrous calcification region.

[0222] Among them, the second-scale feature marker can focus on the global semantic features of the fibrous calcification region, including the strong semantic, spatial location and geometric morphological information of the region, forming a multi-scale comparison with the first-scale feature marker, and together constituting the complete feature representation of the region.

[0223] Step S4700.7: Add a first-scale embedding to the first-scale feature label of each fibrous calcification region to obtain the processed first-scale feature label; add a second-scale embedding to the second-scale feature label of each fibrous calcification region to obtain the processed second-scale feature label.

[0224] First, two independent scale embedding vectors are predefined: a first-scale embedding (adapted to the first-scale feature label, used to label the "first-scale" feature attribute) and a second-scale embedding (adapted to the second-scale feature label, used to label the "second-scale" feature attribute). The dimensions of the two scale embedding vectors are consistent with the dimensions of the first-scale and second-scale feature labels (ensuring element-wise addition or concatenation is possible). Then, the first-scale embedding is element-wise added to the first-scale feature label (or feature concatenation is performed, with element-wise addition preferred to preserve the original feature information while adding a scale identifier), resulting in the processed first-scale feature label. Similarly, the second-scale embedding is element-wise added to the second-scale feature label, resulting in the processed second-scale feature label.

[0225] By adding explicit scale identifiers to the feature labels at both scales, we can avoid confusing the features at the two scales during subsequent sequence encoding, while improving the discriminative power of the features and helping the sequence transformation network to accurately capture the differences in features at multiple scales.

[0226] Step S4700.8: The processed first-scale feature label, the processed second-scale feature label, and the classification label corresponding to the fiber calcification region are concatenated along the sequence dimension to form the input sequence of the fiber calcification region.

[0227] In this embodiment, the processed first-scale feature labels, processed second-scale feature labels, and classification labels are concatenated along the sequence dimension (i.e., the length direction of the feature vector) in a preset order (e.g., classification label → processed first-scale feature label → processed second-scale feature label) to form a longer sequence vector. This sequence vector is the input sequence corresponding to the current fibrous calcification region. This integrates the multi-scale features (first-scale feature labels and second-scale feature labels) of the fibrous calcification region with the global anchor (classification label) into a unified sequence, adapting to the input format of the subsequent sequence transformation network and achieving sequence encoding of multi-scale features.

[0228] Among them, the classification label (i.e., CLS label) refers to a pre-defined fixed-dimensional feature vector, which is a learnable parameter that does not depend on the features of a specific fiber calcification region. It can serve as a global anchor point for sequence encoding, aggregating the multi-scale features of the fiber calcification region, which facilitates the subsequent sequence transformation network to extract the global comprehensive features of the region.

[0229] Step S4800: For any fibrous calcification region, if the mass volume of the fibrous calcification region is less than a volume threshold, the feature markers in the input sequence corresponding to the fibrous calcification region are masked to obtain a set of masked input sequences corresponding to the multiple fibrous calcification regions.

[0230] First, for each fibrocalcified region extracted in step S4100, the mass volume of that region is calculated (i.e., the total number of voxels contained in the fibrocalcified region, which, combined with the voxel resolution of the 3D cardiac image, can be converted into the actual physical volume). Second, a preset volume threshold is applied, and the mass volume of each fibrocalcified region is compared with the volume threshold to determine the size relationship. If the mass volume of the fibrocalcified region is smaller than the volume threshold, it indicates that the region is a microcalcified mass or a noise region, lacking clinical reference value, and its corresponding input sequence needs to be masked. Specifically, the masking targets are the feature markers in the input sequence (i.e., the processed first-scale feature markers and the processed second-scale feature markers), while classification markers are not masked. The masking method is to set the feature value of the feature marker to 0 (or use a masking mechanism to prevent it from participating in subsequent sequence encoding). If the mass volume of the fibrocalcified region is greater than or equal to the volume threshold, it indicates that the region is a clinically significant fibrocalcified mass, and its corresponding input sequence is not processed and is directly used as input for subsequent steps. Thus, each fibrous calcification region corresponds to a masked input sequence (where the unmasked input sequence is the original input sequence). Finally, the masked input sequences corresponding to all fibrous calcification regions are summarized to form a set of masked input sequences (each element is the masked input sequence of a fibrous calcification region).

[0231] The volume threshold is a critical volume value used to screen clinically significant fibrous calcification masses. It is an adjustable parameter, determined based on clinical data statistics and model training effects. Its core function is to eliminate irrelevant regions such as microcalcifications and noise, reduce redundant information, and improve the accuracy of subsequent sequence coding.

[0232] It should be noted that if the volume of the corresponding fibrocalcified region meets the standard, the masked input sequence is consistent with the original input sequence; if the volume does not meet the standard, only the classification label of the original input sequence is retained in the masked input sequence, and the feature labels (i.e., the first-scale feature label and the second-scale feature label) are masked (set to zero or masked) to ensure that subsequent encoding focuses only on the clinically significant fibrocalcified region.

[0233] Based on the above, the effectiveness of fibrocalcification regions can be screened, eliminating clinically insignificant small clumps and noise, reducing the interference of redundant features on subsequent encoding, ensuring that the sequence transformation network focuses on clinically valuable fibrocalcification regions, and improving the accuracy and effectiveness of image feature extraction.

[0234] Step S4900: Input the masked input sequence set and the global context vector into the sequence transformation network to obtain the image features.

[0235] In this embodiment, the sequence transformation network can be a Transformer encoder, which includes a self-attention layer and a feedforward neural network at its core, and is adapted to the encoding processing of multiple sequence features.

[0236] The sequence transformation network first encodes each masked input sequence in the masked input sequence set. It captures the intrinsic correlations between multi-scale features (first-scale feature labels and second-scale feature labels) of individual fibrous calcification regions through a self-attention mechanism, and simultaneously captures the spatial correlations and mutual influences between different fibrous calcification regions through a cross-attention mechanism. Subsequently, the encoded features of individual fibrous calcification regions are fused with the global context vector, integrating the local features of individual regions with the global features of the region of interest, eliminating the limitations of local features. Finally, after multiple rounds of encoding and fusion processing by the sequence transformation network, a unified feature vector is output. This feature vector represents the comprehensive feature representation of all fibrous calcification regions, which is the final image feature output by the image processing module.

[0237] Based on the above, by uniformly encoding and fusing the sequence features of multi-fiber calcification regions that have been effectively screened, and combining them with global contextual information, comprehensive and accurate image features are formed. This enables the transformation from multi-scale features of a single region to a comprehensive representation of the overall fiber calcification state, laying the foundation for subsequent multi-modal feature fusion and prediction tasks.

[0238] In one embodiment of the present invention, the data processing module is specifically used to perform the following steps S5100 to S5400. The core purpose is to encode and filter features item by item to address the missing characteristics of clinical data, and output clinical features that are suitable for the subtyping task and risk prediction task, respectively, so as to provide accurate clinical data support for subsequent information fusion and solve the problems of rough handling of missing clinical data and feature redundancy in the prior art.

[0239] Step S5100: Generate a first embedding representation for each continuous variable and a corresponding second embedding representation for each categorical variable;

[0240] Wherein, the first embedding represents the method used to distinguish between the true measured value and the filled value of the continuous variable, and the second embedding represents the method used to distinguish between the true measured value and the filled value of the categorical variable.

[0241] First, for each continuous variable, a corresponding first embedding representation is generated through a pre-defined embedding layer. For each categorical variable, a corresponding second embedding representation is generated through an independent embedding layer. The first and second embedding representations clearly distinguish between the true measured value and the imputed value of the variable.

[0242] Specifically, the generation of the first embedding representation needs to consider the characteristics of continuous variables, incorporating missing label information during the embedding process. This allows the embedding vector to simultaneously represent the numerical features and missing states of the continuous variable, thereby effectively distinguishing between the true measured values ​​and the imputed values. The generation of the second embedding representation needs to adapt to the discrete characteristics of categorical variables, also incorporating missing label information. Through the differences in the embedding vectors, it clearly distinguishes between the true discrete values ​​and imputed values ​​of the categorical variable (such as the imputed state corresponding to the predefined missing category). The two embedding processes are independent of each other, and the parameters of the embedding layer are trained and optimized together with the prediction model to ensure that the embedding representation can accurately adapt to the subsequent feature selection and fusion requirements.

[0243] Based on the above, missing data-aware embedding encoding can be completed, converting continuous and categorical variables into embedding representations that can distinguish between true and imputed values. This preserves the core information of the clinical data while accurately capturing the missing data state, laying the foundation for subsequent feature splicing and task adaptation screening.

[0244] Step S5200: The first embedding representations of all continuous variables and the second embedding representations of all categorical variables are concatenated in a preset order to form the target feature sequence.

[0245] First, a pre-defined feature concatenation order is established (e.g., "first embedding representations of all continuous variables → second embedding representations of all categorical variables," which can be adjusted based on model training performance and clinical data characteristics, and this order must remain consistent throughout model training and inference). Second, the first embedding representations corresponding to all continuous variables are aggregated and arranged sequentially according to a pre-defined continuous variable sorting (e.g., sorting by clinical importance). Simultaneously, the second embedding representations corresponding to all categorical variables are aggregated and arranged sequentially according to a pre-defined categorical variable sorting. Finally, all the arranged first and second embedding representations are concatenated along the feature channel dimension to form a unified long feature vector, which is the target feature sequence.

[0246] During the splicing process, it is necessary to ensure that the dimensions of all first and second embedding representations are consistent. If the dimensions are inconsistent, they need to be adjusted to a unified dimension through a linear mapping layer before splicing to avoid splicing failure or feature distortion due to dimensional differences.

[0247] The target feature sequence integrates missing-aware embedding information from all clinical variables and serves as the basic input for subsequent feature selection for different tasks.

[0248] Step S5300: Multiply the weight of the classification task element by element with the target feature sequence to obtain the first filtered feature sequence corresponding to the classification task; and multiply the weight of the risk prediction task element by element with the target feature sequence to obtain the second filtered feature sequence corresponding to the risk prediction task.

[0249] First, two independent task weight vectors are preset: the classification task weight and the risk prediction task weight. The dimensions of both weight vectors are consistent with the dimensions of the target feature sequence, and the weight values ​​are learnable parameters between 0 and 1 (trained and optimized together with the prediction model).

[0250] Subsequently, the classification task weights are multiplied element-wise with the target feature sequence. That is, each feature value in the target feature sequence is multiplied by the corresponding weight value in the classification task weights. The resulting new feature sequence is the first filtered feature sequence corresponding to the classification task. Similarly, the risk prediction task weights are multiplied element-wise with the target feature sequence to obtain the second filtered feature sequence corresponding to the risk prediction task.

[0251] By using element-wise multiplication, task-fit selection of the target feature sequence is achieved: features important to the fractal task have higher weight values, and are strengthened after multiplication. Features irrelevant to the fractal task have weight values ​​close to 0, and are weakened or suppressed after multiplication. The same method is used for feature selection in risk prediction tasks. The two selection processes are independent and can select the same or different features, ensuring that the selected features are highly adapted to the corresponding tasks.

[0252] Step S5400: Input the first filtered feature sequence into the genotyping task fusion network and output the first data feature; input the second filtered feature sequence into the risk prediction task fusion network and output the second data feature.

[0253] In this embodiment, firstly, two independent task fusion networks are preset: a fractal task fusion network and a risk prediction task fusion network. The two networks can be constructed using a fully connected layer combined with an attention mechanism. The structure can be adjusted according to task requirements, but they are independent of each other to avoid interference between tasks.

[0254] Subsequently, the first filtered feature sequence is input into the classification task fusion network. This network further fuses and enhances the filtered classification-specific features, focuses on the most discriminative core features through an attention mechanism, eliminates feature redundancy, and finally outputs a unified feature vector, which is the first data feature corresponding to the classification task.

[0255] Similarly, the second-screened feature sequence is input into the risk prediction task fusion network. This network fuses and enhances the screened risk prediction-specific features, focuses on the core features that are highly correlated with postoperative survival risk, and finally outputs a unified feature vector, which is the second data feature corresponding to the risk prediction task.

[0256] The parameters of the two task fusion networks are trained and optimized together with the prediction model to ensure that they can accurately extract the core features of the corresponding tasks. The output first and second data features will be used as inputs to the subsequent information fusion module, and together with the image features output by the image processing module, they will complete the classification and risk prediction tasks.

[0257] In one embodiment of the present invention, the information fusion module, the typing output layer and the risk output layer are specifically used to perform the following steps S6100 to S6300. The core purpose is to fuse the image features output by the image processing module and the two types of clinical data features output by the data processing module, and simultaneously output the typing results and the survival risk quantification value to achieve the collaborative completion of typing and risk prediction.

[0258] Step S6100: Obtain the image features, the first data features, and the second data features, and perform dimensional alignment processing on the three types of features.

[0259] In this embodiment, image features output by the image processing module and first and second data features output by the data processing module are acquired respectively. The dimensions of the three types of features are then checked for consistency. If the dimensions are inconsistent, a linear mapping layer is used to map the dimensions of the first and second data features to the same dimensions as the image features, thus aligning the dimensions of the three types of features. If the dimensions are consistent, the process proceeds directly to the subsequent fusion step to ensure the feasibility of feature fusion.

[0260] For example, the image features are 320-dimensional, the first data features are 200-dimensional, and the second data features are 180-dimensional. Through two independent linear mapping layers, the first and second data features are mapped to 320 dimensions respectively, thereby achieving dimensional alignment of the three types of features.

[0261] Step S6200: The three types of features after dimension alignment are fused through the information fusion module to obtain fused features.

[0262] In this embodiment, a dual fusion method of "feature stitching + attention fusion" is adopted to fuse the dimension-aligned image features, the first data feature, and the second data feature. First, the three types of features are stitched together to obtain the initial fused features. Then, through the attention mechanism, the attention weights of the three types of features (the weights reflect the importance of each type of feature to the task) are calculated. The attention weights are then weighted and summed with the initial fused features to obtain the final fused features. This ensures that the fused features can fully integrate multimodal information and highlight the role of important features.

[0263] Continuing with the example above, the three features after dimension alignment are all 320-dimensional. First, the three are concatenated to obtain an initial fusion feature of 320×3=960 dimensions. The image feature weight is calculated to be 0.5, the first data feature weight is 0.25, and the second data feature weight is 0.25 through the attention mechanism. The initial fusion feature is then weighted and summed with the weights to obtain the fusion feature of 320 dimensions.

[0264] In step S6300, the fusion features are input into the classification output layer and the risk output layer respectively to obtain the aortic stenosis classification results and the survival risk quantification value.

[0265] In this embodiment, the fusion features obtained in step S6200 are input into the typing output layer and the risk output layer respectively. The two output layers are processed in parallel and output results synchronously. The typing output layer outputs the probability of each AS hemodynamic typing through a classification activation function, and selects the typing with the highest probability as the final typing result. The risk output layer outputs a survival risk quantification value between 0 and 1 through a regression activation function. The larger the value, the higher the survival risk of the patient within the preset time window after surgery.

[0266] The genotyping output layer can use a fully connected layer with a softmax activation function to suit multi-class classification tasks (AS hemodynamic genotyping has 4 classes). The risk output layer can use a fully connected layer with a sigmoid activation function to suit regression tasks (outputting a scalar value between 0 and 1).

[0267] Risk levels are categorized based on a quantitative value of survival risk (e.g., 0-0.3 is low risk, 0.3-0.7 is medium risk, and 0.7-1.0 is high risk).

[0268] Continuing with the example above, the 320-dimensional fusion features are input into the classification output layer and the risk output layer, respectively. The classification output layer outputs the probabilities of four classifications: high-gradient severe stenosis 10%, low-flow low-pressure gradient stenosis 8%, paradoxical low-flow low-pressure gradient stenosis 89%, and normal-flow low-pressure gradient stenosis 3%. The final classification result is "paradoxical low-flow low-pressure gradient stenosis". The risk output layer outputs 0.32, which is a survival risk quantification value of 0.32, belonging to the medium risk category, indicating that the probability of adverse clinical events occurring within 1 year after surgery is 32%.

[0269] Based on the above, firstly, by acquiring a spatial distribution mask of fibrocalcifications generated from a 3D cardiac image to replace manual delineation, the dependence on manual delineation and observer-to-observer differences are eliminated. Secondly, the image processing module uses this mask to guide multi-scale feature extraction and sequence encoding of local regions in the 3D cardiac image, capturing the spatial distribution relationship of each fibrocalcification region, thereby quantifying the spatial distribution heterogeneity ignored by traditional methods. Thirdly, the data processing module performs missing-aware item-by-item encoding and feature screening on clinical data, generating dedicated embeddings for missing values, enabling the model to distinguish between true values ​​and filled values, solving the problem of coarse processing of missing data. Finally, the information fusion module fuses image features with the first and second data features, and simultaneously outputs the typing results and survival risk quantification values ​​through the typing output layer and risk output layer, realizing the integration of typing and risk prediction, and improving clinical diagnosis and treatment efficiency.

[0270] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the methods described in any of the above-described method embodiments.

[0271] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0272] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0273] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0274] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.

[0275] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0276] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0277] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0278] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0279] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0280] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A system for classifying aortic stenosis and predicting survival risk, characterized in that, include: (i) Multimodal data acquisition module for target object; the multimodal data includes a three-dimensional image of the heart, a fibrocalcification spatial distribution mask generated based on the three-dimensional image of the heart, and clinical data; wherein, the fibrocalcification spatial distribution mask is a binary three-dimensional mask used to indicate the spatial location of fibrotic tissue and calcification tissue in the aortic valve; (ii) Typing and Risk Prediction Module; the multimodal data serves as input to the prediction model, yielding the aortic stenosis typing result of the target subject and the quantitative survival risk value of the target subject within a preset time window after transcatheter aortic valve replacement; the prediction model includes an image processing module, a data processing module, an information fusion module, a typing output layer, and a risk output layer; wherein: The image processing module uses the spatial distribution mask of fibrous calcification to guide the multi-scale feature extraction and sequence encoding of local regions of the three-dimensional image of the heart, thereby obtaining image features. The data processing module performs missing-aware item-by-item encoding and feature filtering on the clinical data to obtain the first data feature corresponding to the subtyping task and the second data feature corresponding to the risk prediction task. The information fusion module fuses the image features, the first data features, and the second data features, and outputs the aortic stenosis classification result of the target object through the classification output layer, and outputs the survival risk quantification value through the risk output layer.

2. The system according to claim 1, characterized in that, The prediction model is trained in the following way: Obtain a training sample set; each training sample in the training sample set contains multimodal data of the sample object, aortic stenosis classification of the sample, and survival status of the sample; the multimodal data of the sample includes a three-dimensional image of the sample heart, a sample fibrocalcification spatial distribution mask generated based on the sample three-dimensional image of the sample heart, and sample clinical data. Construct an initial prediction model; the initial prediction model includes an image processing module, a data processing module, an information fusion module, a classification output layer, and a risk output layer; The multimodal data of the training sample set is input into the initial prediction model to calculate the total loss function; the total loss function includes the classification error of the genotyping task, the temporal risk error of the risk prediction task, and the sparse constraint term of feature selection in the data processing module. Based on the total loss function, the parameters of the initial prediction model are iteratively updated using an optimization algorithm until the convergence condition is met, thus obtaining the prediction model.

3. The system according to claim 2, characterized in that, The acquisition of the training sample set specifically includes: Obtain raw data for any sample object; the raw data includes raw 3D cardiac images, raw clinical data, and follow-up data, wherein the follow-up data is used to record indication information of whether the sample object has experienced a preset clinical endpoint event within a preset time window after transcatheter aortic valve replacement. The original three-dimensional heart image is preprocessed, including intensity normalization, region of interest cropping, and three-dimensional image block sampling, to obtain a sample three-dimensional heart image. The sample three-dimensional heart image is then input into a segmentation model to obtain a sample fibrous calcification spatial distribution mask. The original clinical data is preprocessed, including standardization, missing value handling, and categorical variable coding, to obtain sample clinical data; Based on the clinical data of the samples, the aortic stenosis type of the samples was determined; Based on the follow-up data, the survival status of the samples was determined; The three-dimensional image of the sample heart, the spatial distribution mask of the sample fibrocalcification, the sample clinical data, the sample aortic valve stenosis classification, and the sample survival status are used as a training sample. The training sample set is obtained based on the training samples corresponding to different sample objects.

4. The system according to claim 3, characterized in that, The segmentation model includes a first segmentation module and a second segmentation module. The step of inputting the three-dimensional image of the sample heart into the segmentation model to obtain a mask of the spatial distribution of fibrous calcification in the sample specifically includes: The three-dimensional image of the heart is input into the first segmentation module to obtain a mask of the valve complex region. The mask of the valve complex region is input into the second segmentation module to obtain a tissue segmentation probability map; the tissue segmentation probability map includes the probability distributions corresponding to different tissue categories. Based on the density value and spatial coordinates of each voxel in the tissue segmentation probability map, the tissue segmentation probability map is clustered to obtain the tissue threshold range of the sample object. Based on the tissue threshold range, voxels with probability values ​​greater than a preset probability threshold in the tissue segmentation probability map are identified as fibrous calcification regions, and voxels with probability values ​​less than or equal to the preset probability threshold in the tissue segmentation probability map are identified as background, thus obtaining the sample fibrous calcification spatial distribution mask.

5. The system according to claim 1, characterized in that, The image processing module is specifically used for: Based on the aforementioned fiber calcification spatial distribution mask, multiple fiber calcification regions are extracted; Calculate the bounding box of the multiple fiber calcification regions, and take the geometric center of the bounding box as the cutting center; Using the cropping center as a reference, a three-dimensional sub-region of a preset size is cropped from the three-dimensional image of the heart as a region of interest, the region of interest including the plurality of fibrous calcification regions; The region of interest is input into a spatial feature extraction network to obtain a multi-scale feature map, which includes a first-scale feature map with high spatial resolution and a second-scale feature map with strong semantic information. Global average pooling is performed on the second-scale feature map to obtain the global context vector; The fiber calcification spatial distribution mask is aligned with the spatial dimensions of the first-scale feature map through interpolation to obtain a first alignment mask, and the fiber calcification spatial distribution mask is aligned with the spatial dimensions of the second-scale feature map through interpolation to obtain a second alignment mask. For each of the plurality of fiber calcification regions, determine the input sequence corresponding to the fiber calcification region; For any fibrous calcification region, if the mass volume of the fibrous calcification region is less than a volume threshold, the feature markers in the input sequence corresponding to the fibrous calcification region are masked to obtain a set of masked input sequences corresponding to the multiple fibrous calcification regions. The masked input sequence set and the global context vector are input into the sequence transformation network to obtain the image features.

6. The system according to claim 5, characterized in that, The step of determining the input sequence corresponding to the fibrous calcification region specifically includes: Based on the first alignment mask, the voxels covered by the fibrous calcification region are determined on the first scale feature map to obtain the first voxel set, and the first semantic features of the fibrous calcification region are determined according to the first voxel set. Based on the spatial coordinates of each voxel in the first voxel set, the spatial distribution characteristics of the fibrous calcification region are calculated. Based on the first voxel set, the morphological characteristics of the fibrous calcification region are calculated; By concatenating the first semantic feature, the spatial distribution feature, and the morphological feature, a first-scale feature marker of the fibrous calcification region is obtained. Based on the second alignment mask, the voxels covered by the fibrous calcification region are determined on the second scale feature map to obtain a second voxel set, and the second semantic features of the fibrous calcification region are determined according to the second voxel set. By concatenating the second semantic feature, the spatial distribution feature, and the morphological feature, a second-scale feature marker of the fibrous calcification region is obtained. Add a first-scale embedding to the first-scale feature label of each fibrous calcification region to obtain the processed first-scale feature label; add a second-scale embedding to the second-scale feature label of each fibrous calcification region to obtain the processed second-scale feature label. The processed first-scale feature labels, processed second-scale feature labels, and classification labels corresponding to the fibrous calcification region are concatenated along the sequence dimension to form the input sequence of the fibrous calcification region.

7. The system according to claim 1, characterized in that, The clinical data includes continuous variables and categorical variables, and the data processing module is specifically used for: A first embedding representation is generated for each continuous variable, and a corresponding second embedding representation is generated for each categorical variable; wherein, the first embedding representation is used to distinguish between the true measured value and the filled value of the continuous variable, and the second embedding representation is used to distinguish between the true measured value and the filled value of the categorical variable; The first embedding representations of all continuous variables and the second embedding representations of all categorical variables are concatenated in a preset order to form the target feature sequence; The classification task weights are multiplied element-wise with the target feature sequence to obtain the first filtered feature sequence corresponding to the classification task, and the risk prediction task weights are multiplied element-wise with the target feature sequence to obtain the second filtered feature sequence corresponding to the risk prediction task. The first filtered feature sequence is input into the genotyping task fusion network to output the first data feature; the second filtered feature sequence is input into the risk prediction task fusion network to output the second data feature.

8. The system according to claim 1, characterized in that, The aortic stenosis classification result is one of the following: high-gradient severe stenosis, low-flow low-pressure gradient stenosis, paradoxical low-flow low-pressure gradient stenosis, or normal-flow low-pressure gradient stenosis.

9. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, performing the functions of the system as claimed in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, performs the functions of the system according to any one of claims 1-8.

Images