Abdominal aortic aneurysm progression prediction method and system based on cross-modal knowledge distillation

Abstract

The application discloses a kind of abdominal aortic aneurysm progression prediction method and system based on cross-modal knowledge distillation, by collecting the blood of target object;The blood of target object is detected by blood protein fingerprint marker concentration, and blood protein fingerprint marker includes matrix metalloproteinase-12, calcitonin related polypeptide-alpha, urine regulator, nerve injury induced protein-1, glycosylated hemoglobin A1c, fibroblast growth factor-9;Based on blood protein fingerprint marker concentration, Student-net of cross-modal knowledge distillation is called to calculate abdominal aortic aneurysm risk;Wherein, the input of Teacher-net of cross-modal knowledge distillation includes CTA image feature and pathological section feature, and the category probability vector output by Student-net is aligned with the category probability vector output by Teacher-net;Output abdominal aortic aneurysm risk grade and preset year abdominal aortic aneurysm occurrence / progression / rupture three event probability.Precise prediction is carried out to the development of aortic aneurysm disease.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to an abdominal aortic aneurysm progression prediction method and system based on cross-modal knowledge distillation. BACKGROUND

[0002] An abdominal aortic aneurysm (AAA) is a serious life-threatening vascular disease, which refers to the local sustained expansion of the largest abdominal artery blood vessels in the human body. When the diameter of the blood vessel is increased by more than 50% compared with the normal condition, a weak cystic or fusiform bulge is formed, which is diagnosed as an aneurysm.

[0003] Doctors usually diagnose the disease when the abdominal aorta diameter exceeds 3 cm. Many patients may not feel any discomfort in daily life, but once an aneurysm ruptures, like a balloon being blown up, the patient will suddenly have severe abdominal pain, a sharp drop in blood pressure and massive bleeding. This situation is very dangerous and can endanger life in a short time.

[0004] The annual growth rate of untreated abdominal aortic aneurysm is about 2-8 mm, and the risk of rupture is significantly increased when the diameter is >5 cm, with a mortality rate of 70%-90%. The essence is that the smooth muscle and elastic fibers in the middle layer of the arterial wall are destroyed, resulting in a decrease in structural strength due to replacement by collagen.

[0005] Currently, there is no quantifiable method to effectively predict the development of abdominal aortic aneurysm to prevent sudden rupture and life-threatening consequences. SUMMARY

[0006] To solve the above problems, the purpose of the present application is to provide an abdominal aortic aneurysm progression prediction method and system based on cross-modal knowledge distillation, which can accurately predict the development of aortic aneurysm disease so as to intervene in time and prevent sudden rupture and endanger life.

[0007] The technical solution provided by the present application is as follows: an abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation, comprising the following steps:

[0008] Collecting blood of a target object;

[0009] Detecting the concentration of blood protein fingerprint markers of the target object, the blood protein fingerprint markers including matrix metalloproteinase-12, calcitonin-related polypeptide-alpha, uromodulin, nerve injury-induced protein-1, glycosylated hemoglobin A1c, and fibroblast growth factor-9;

[0010] calling the cross-modal knowledge distillation Student-net based on the blood protein fingerprint marker concentration to calculate the abdominal aortic aneurysm risk; wherein the input of the cross-modal knowledge distillation Teacher-net includes CTA image features and pathological section features, and the class probability vector output by the Student-net and the class probability vector output by the Teacher-net alignment;

[0011] outputting the abdominal aortic aneurysm risk level and the probability of three types of events of abdominal aortic aneurysm occurrence / progression / breakage in a preset year.

[0012] Preferably, before training the cross-modal knowledge distillation Teacher-net, the CTA image features, the pathological section features, the blood protein fingerprint marker concentration, the public database information and the text mining AI score need to be preprocessed:

[0013] For the CTA image features, 3D-ResNet-18 migration learning implemented by PyTorch is adopted, and 256 × 1 full connection is output, and SE-Attention is compressed to 128 dimensions;

[0014] For the pathological section features, ViT-Tiny is fine-tuned for 20 epochs, and 64 dimensions are taken after Global Average Pooling, wherein the pathological section is processed into an image of 224 × 224 size, and the image is divided into 16 × 16 patches by ViT-Tiny;

[0015] The blood protein fingerprint marker concentration, the public database information and the text mining AI score are respectively subjected to Z-standardization and missing value filling processing.

[0016] Preferably, before training the cross-modal knowledge distillation Teacher-net, a feature matrix needs to be constructed:

[0017] constructing a feature matrix based on the CTA image features, the pathological section features, the blood protein fingerprint marker concentration, the public database information and the text mining AI score wherein is the sample size, x 1~6D represents matrix metalloproteinase-12, calcitonin-related polypeptide-alpha, uromodulin, nerve injury-induced protein-1, glycosylated hemoglobin A1c, fibroblast growth factor-9 concentration data, D represents CTA image aortic diameter level, I represents CTA image convolution features, H represents abdominal aortic aneurysm risk classification score, P represents pathological section feature vector, at least including inflammatory cell infiltration, elastic fiber rupture, smooth muscle cell apoptosis, and microstructure information, O represents mining features output by the Transformer text encoder of the public literature; the feature matrix is used to input the public database information and the text mining AI score through a fusion layer together with the CTA image features and / or the pathological section features into a teacher model, to provide supplementary knowledge of the mechanism layer and the baseline layer, thereby enhancing the knowledge transfer effect of the distillation process.

[0018] Preferably, the process of training the Teacher-net and the Student-net of the cross-modal knowledge distillation includes:

[0019] Inputting the CTA image into the 3D-CNN+Self-Attention model to output the rupture risk probability ;

[0020] Inputting the pathological section into the ViT-Tiny to output the histopathological risk probability ;

[0021] Calling the DeepSeek-Reasoner API based on the public database information and the text mining AI score to output the clinical risk probability and the public database risk probability ;

[0022] The fusion layer adopts the Transformer output category probability vector ;

[0023] Aligning the category probability vector output by the Student-net with the category probability vector output by the Teacher-net, and using the distillation loss: , where σ is the softmax function, T is the distillation temperature, α is the loss weight coefficient, CE is the cross-entropy loss, KL represents the KL divergence, y true is the real label;

[0024] The Teacher-net receives the CTA and pathological feature output fusion risk vector and the multi-layer attention weight , where is the attention distribution of the lth layer, the element takes a value of 0-1, indicating the importance allocation of different feature regions, satisfying row normalization, represents the probability of abdominal aortic aneurysm occurrence, represents the probability of abdominal aortic aneurysm progression, represents the probability of abdominal aortic aneurysm rupture.

[0025] Preferably, the process of training the Teacher-net and the Student-net for the cross-modal knowledge distillation further comprises:

[0026] Applying target temperature to obtain soft labels, and calculating divergence with the Student-net output;

[0027] Measuring the L2 distance between the attention of the Student-net and the attention of the Teacher-net , focusing on key high-risk patterns has achieved attention transfer.

[0028] Preferably, the attention transfer of the cross-modal knowledge distillation further comprises:

[0029] Extracting key attention maps from the multi-layer attention weights of the Teacher-net;

[0030] Introducing corresponding attention modules in the Student-net, and calculating the attention output thereof;

[0031] Using L2 distance as a loss function to quantify the difference in attention distribution between the two;

[0032] Adding this loss to the total distillation loss, prompting the Student-net to adjust parameters in training to minimize the distance, thereby transferring the attention of the Teacher-net to key features.

[0033] Preferably, the process of training the Teacher-net and the Student-net for the cross-modal knowledge distillation further comprises:

[0034] Applying temperature T=4 to obtain soft labels , and calculating divergence with the Student-net output ;

[0035] At the same time, using the real label y to calculate the cross-entropy CE, ensuring the classification accuracy;

[0036] Freezing the Teacher-net, and only updating the Student-net parameters through backpropagation until the validation set ΔAUC<0.01 converges, where ΔAUC represents the change in the area under the ROC curve, used to evaluate the degree of performance improvement of the binary classification model. ​​

[0037] The application also provides an abdominal aortic aneurysm progression prediction system based on cross-modal knowledge distillation, comprising:

[0038] A collection module is configured to collect blood of a target object.

[0039] A detection module is configured to detect concentrations of blood protein fingerprint markers of the target object, wherein the blood protein fingerprint markers comprise matrix metalloproteinase-12, calcitonin-related polypeptide-alpha, uromodulin, nerve injury-induced protein-1, glycosylated hemoglobin A1c, and fibroblast growth factor-9.

[0040] A risk calculation module is configured to invoke a Student-net of the cross-modal knowledge distillation to calculate an abdominal aortic aneurysm risk based on the concentrations of the blood protein fingerprint markers, wherein input of a Teacher-net of the cross-modal knowledge distillation comprises CTA image features and pathological section features, and a class probability vector output by the Student-net is aligned with a class probability vector output by the Teacher-net. The class probability vector output by the Student-net is aligned with the class probability vector output by the Teacher-net.

[0041] An output module is configured to output an abdominal aortic aneurysm risk level and probabilities of three types of events, i.e., abdominal aortic aneurysm occurrence, abdominal aortic aneurysm progression, and abdominal aortic aneurysm rupture, in a preset year.

[0042] The application also provides an electronic device, comprising:

[0043] A memory is configured to store a processing program.

[0044] A processor is configured to implement the abdominal aortic aneurysm progression prediction method based on the cross-modal knowledge distillation when executing the processing program.

[0045] The application also provides a readable storage medium, wherein the readable storage medium stores a processing program, and the processing program is configured to implement the abdominal aortic aneurysm progression prediction method based on the cross-modal knowledge distillation when executed by a processor.

[0046] The application has the following advantages and positive effects compared with the prior art:

[0047] ​The present scheme only needs a blood sample, completely avoids the limitations of imaging examination, realizes truly non-invasive, low-risk, low-cost, and strong repeatability dynamic monitoring, greatly improves patient compliance and screening / monitoring frequency. Cross-modal knowledge distillation realizes "small model, great wisdom", Teacher-Net (teacher network): input contains rich CTA image features (anatomical structure, tumor size and shape, thrombus, wall thickness, etc.) and pathological section features (structural damage at the cellular level, inflammatory infiltration, calcification, etc.). These modal data have a huge amount of information and can build a very powerful prediction model, but their acquisition is highly dependent on professional equipment, invasive operation (biopsy) and expert interpretation, and it is extremely difficult to apply clinically. Student-Net (student network): only uses easily accessible blood marker concentrations as input. Student-Net can learn and inherit the complex discrimination ability contained in Teacher-Net (based on image + pathology) only with blood data, and its prediction accuracy is much higher than that of traditional models trained only with blood data, and is infinitely close to Teacher-Net using the gold standard multi-modal data. The structure of Student-Net is simpler than that of Teacher-Net which integrates multiple modalities, has small computational load, and is easier to integrate into a clinical laboratory information system or a portable device to realize rapid, real-time bedside or laboratory risk assessment. BRIEF DESCRIPTION OF DRAWINGS

[0048] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings, in which:

[0049] Figure 1 The flow chart of the abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation of the present application;

[0050] Figure 2 The clinical sample quantification of the abdominal aortic aneurysm protein fingerprint of an embodiment of the present application;

[0051] Figure 3 The structure schematic diagram for CTA feature extraction;

[0052] Figure 4 The encoding flow chart of pathological section ViT-Tiny;

[0053] Figure 5 Cross-modal independence verification of the present application;

[0054] Figure 6 The Receiver Operating Characteristic (ROC) curve diagram of the model of the present application;

[0055] Figure 7Calibration curve for the model of the present application;

[0056] Figure 8 Net Benefit Curve (NBC) for the model of the present application. DETAILED DESCRIPTION

[0057] The present application will be further described below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present application will be more apparent from the following description. It should be noted that the accompanying drawings are very simplified and use non-precise ratios, only for the purpose of facilitating, clarifying and assisting in the description of the embodiments of the present application.

[0058] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications will also change accordingly.

[0059] First embodiment

[0060] As shown in the drawings, the present embodiment provides a method for predicting abdominal aortic aneurysm progression based on cross-modal knowledge distillation, comprising the following steps: Figure 1 Collecting blood of a target object;

[0061] Detecting blood protein fingerprint marker concentration of the blood of the target object, the blood protein fingerprint marker including matrix metalloproteinase-12, calcitonin-related polypeptide-alpha, uromodulin, nerve injury-induced protein-1, glycosylated hemoglobin A1c, and fibroblast growth factor-9;

[0062] Based on the blood protein fingerprint marker concentration, calling the Student-net of the cross-modal knowledge distillation to calculate the risk of abdominal aortic aneurysm; wherein the input of the Teacher-net of the cross-modal knowledge distillation includes CTA image features and pathological section features, and the class probability vector output by the Student-net is aligned with the class probability vector output by the Teacher-net.

[0063]

[0064] Outputting abdominal aortic aneurysm risk level and probability of three types of events of abdominal aortic aneurysm occurrence / progression / breaking in a preset year.

[0065] ​​​Traditional abdominal aortic aneurysm monitoring and risk assessment is highly dependent on imaging examinations (such as ultrasound, CTA), this scheme only needs blood samples, completely avoids the limitations of imaging examinations, realizes truly non-invasive, low-risk, low-cost, and strong repeatability of dynamic monitoring, greatly improves patient compliance and screening / monitoring frequency. Cross-modal knowledge distillation realizes "small model, big wisdom", Teacher-Net (teacher network): input contains rich CTA image features (anatomical structure, aneurysm size and shape, thrombus, wall thickness, etc.) and pathological section features (structural damage at the cellular level, inflammation infiltration, calcification, etc.). These modal data have a huge amount of information, and can build a very powerful prediction model, but its acquisition is highly dependent on professional equipment, invasive operation (biopsy) and expert interpretation, and it is extremely difficult to apply clinically. Student-Net (student network): only use easily accessible blood marker concentrations as input. Knowledge distillation process: by aligning the class probability vectors (i.e. the probability distribution of predicting "occurrence / progression / rupture") output by Student-Net and Teacher-Net, Student-Net not only learns the mapping from blood data to labels during training, but also learns to mimic the "judgment" or "knowledge" made by Teacher-Net based on multi-modal rich information. Student-Net can learn and inherit the complex discrimination ability of Teacher-Net (based on image + pathology) based on blood data alone, and its prediction accuracy is much higher than that of traditional models trained only with blood data, and is close to that of Teacher-Net using gold standard multi-modal data. The structure of Student-Net is simpler than that of Teacher-Net which fuses multiple modalities, and the calculation amount is small, and it is easier to integrate into a clinical laboratory information system or a portable device to realize rapid, real-time bedside or laboratory risk assessment. Without the need to obtain expensive CTA and invasive pathological sections during prediction, blood testing alone can achieve a prediction effect close to the use of these advanced information. The output result is not only a simple "high risk / low risk" label, but also a stratification of the current overall risk (such as low, medium, high), which is convenient for rapid classification and management. Probability of three events in a preset year: directly quantify the specific probability of abdominal aortic aneurysm occurrence (for high-risk population), progression (significant aneurysm enlargement), and rupture within a specific time window (such as 1 year, 3 years, 5 years) in the future. This output directly meets the core needs of clinical decision-making.

[0066] The present application uses cross-modal knowledge distillation technology to "compress" and "migrate" the complex disease progression information contained in high-order images and pathological sections to a lightweight model that only needs routine blood testing (specific proteome) to run.

[0067] The training model features include:

[0068] (1) Image group: aortic diameter, wall calcification volume, intramural hematoma signal, vascular remodeling morphology;

[0069] (2) Pathology group: elastin fiber rupture ratio, smooth muscle cell apoptosis rate, and inflammatory cell infiltration degree;

[0070] (3) Proteomics: 6 verified blood protein fingerprint markers;

[0071] (4) Public database information (from public resources): including differential pathway enrichment score, protein interaction network node degree centrality, and reported functional mutation sites, which are standardized, missing value filled, and feature scaled. After key-value alignment based on patient ID or feature label with clinical / experimental data, they can be used for training;

[0072] (5) Text mining AI score: attention score generated by literature abstract and full text through Transformer encoder, and keyword surge score.

[0073] Table 1 - Features for training the model:

[0074]

[0075] The five-dimensional features (molecular layer, organ macroscopic layer, tissue microscopic layer, mechanism layer, and baseline layer) are statistically independent through orthogonal constraints to reduce redundant information and improve generalization performance.

[0076] In the model data input stage, images and pathology need to be deeply extracted (due to high data dimension and complex structure), and the remaining dimensional data has been normalized, encoded, and missing value processed in the early stage, and can be directly used as structured variables to input the fusion layer.

[0077] Inclusion and exclusion criteria: study subjects were selected from clinical visit population, stratified according to abdominal aortic aneurysm size, including abdominal aortic diameter <30mm, abdominal aortic aneurysm 30-55mm, 55-75mm, >75mm population, and the study population was followed up every year. The study complies with the requirements of the Declaration of Helsinki, and written consent was obtained from each subject before baseline investigation and each follow-up investigation, and informed consent was signed. Inclusion criteria: ① 65 years old or older; ② Patients with abdominal aortic diameter <30mm; ③ Patients with abdominal aortic aneurysm 30-55mm; ④ Patients with abdominal aortic aneurysm 55-75mm; ⑤ Patients with abdominal aortic aneurysm >75mm. Exclusion criteria: ① Incomplete study-related data ② No informed consent ③ Patients with other major diseases.

[0078] Protein quantification used Luminex 200 instrument and PeproTech commercial antibody microbead kit (LH-AAA-6Plex-2025), six proteins were detected in the same batch. Standard curve R 2 > 0.995; lower limit of detection (LOD) 0.1-0.5 ng mL -1 .

[0079] Referring to Figure 2 , the clinical sample quantification of abdominal aortic aneurysm protein fingerprint is shown.

[0080] Preferably, the CTA image features, the pathological section features, the blood protein fingerprint marker concentration, the public database information and the text mining AI score need to be preprocessed before training the Teacher-net of the cross-modal knowledge distillation:

[0081] For the CTA image features, 3D-ResNet-18 migration learning implemented by PyTorch is adopted, outputting 256x1 full connection, and SE-Attention is compressed to 128 dimensions;

[0082] For the pathological section features, ViT-Tiny is fine-tuned for 20 epochs, and 64 dimensions are taken after Global Average Pooling, wherein the pathological section is processed into an image of 224x224 size, and the image is divided into 16x16 patches by ViT-Tiny;

[0083] The blood protein fingerprint marker concentration, the public database information and the text mining AI score are respectively subjected to Z-standardization and missing value filling processing.

[0084] Directly use the general feature extraction capability of the pre-trained model on large 3D medical image datasets (such as CT, MRI), avoid the high cost of training from scratch and the risk of overfitting of small samples. 3D convolution can effectively capture the spatial volume information of the blood vessel structure (such as aneurysm shape, thrombus distribution, and pipe wall calcification). Self-adaptive learning of the importance weight of different channel features enables the model to focus on the image area most relevant to the abdominal aortic aneurysm pathology (such as the weak pipe wall and the inflammation area), suppresses irrelevant background noise, and compresses to 128 dimensions, which greatly reduces the feature dimension while retaining discriminative information, reduces the computational burden of subsequent fusion, and improves the model generalization ability. Use lightweight Vision Transformer (ViT-Tiny) to fine-tune on medical pathology images. The self-attention mechanism of ViT can model the global long-distance dependence relationship of the section (such as the inflammatory cell infiltration range and the extracellular matrix destruction pattern), overcoming the limitations of the local receptive field of CNN.

[0085] Figure 3A CTA feature extraction structure schematic diagram is shown, Figure 4 A pathological section ViT-Tiny encoding flowchart is shown. Features in dimensions such as protein fingerprints, public database features, text mining scores, and clinical risk scores are standardized, missing value filled, and the like in the feature engineering stage, and can be directly spliced with deep features in the fusion layer without secondary convolution or Transformer encoding.

[0086] Preferably, a feature matrix needs to be constructed before training the Teacher-net for cross-modal knowledge distillation:

[0087] A feature matrix is constructed based on the CTA image features, pathological section features, blood protein fingerprint marker concentrations, public database information, and text mining AI scores , wherein is the sample size, , x 1~6 represents matrix metalloproteinase-12, calcitonin-related polypeptide-alpha, uromodulin, nerve injury-induced protein-1, glycosylated hemoglobin A1c, and fibroblast growth factor-9 concentration data, D represents CTA image aortic diameter levels, I represents CTA image convolution features, H represents abdominal aortic aneurysm risk classification scores, P represents pathological section feature vectors, at least including inflammatory cell infiltration, elastic fiber rupture, and smooth muscle cell apoptosis microscopic structure information, and O represents mining features output by a Transformer text encoder from public literature; the feature matrix is used to input public database information and text mining AI scores through a fusion layer together with CTA image features and / or pathological section features into a teacher model, providing supplementary knowledge for the mechanism layer and the baseline layer to enhance the knowledge transfer effect of the distillation process.

[0088] By constructing a structured feature matrix as the input core of Teacher-Net, the systematic integration and knowledge complementarity of multi-source heterogeneous data are realized, and the knowledge breadth and depth of the "teacher" model in cross-modal knowledge distillation are significantly improved. Breaking the data island, the traditional fragmented image, pathology, biochemistry, clinical, literature data are unified in a structured matrix (n xk), providing a multi-dimensional disease view for Teacher-Net. Public database information (H, etc.) and text mining features (O) are combined with core image / pathology features (I / P) through a special fusion layer. Text mining features contain molecular mechanisms, pathway associations, and pathophysiological logic reported in the literature (such as "MMP-12 high expression promotes elastin degradation"). Help Teacher-Net understand the biological causal chain between blood markers (x1~x6) and image / pathology performance (I / P), improve the explainability and generalization ability of prediction. Clinical risk score (H) and public database information (such as age, comorbidities) represent clinical consensus and epidemiological evidence. Provide prior risk benchmarks for the model to correct individual bias (such as higher baseline risk for older patients), making predictions more consistent with clinical practice rules. Teacher-Net not only learns statistical associations, but also absorbs the inherent logic of domain knowledge, forming "medical expert-level" judgment ability. The injected mechanism and baseline knowledge make the probability vector output by Teacher-Net contain richer pathological logic and clinical context, providing a better imitation target for Student-Net.

[0089] Matrix coverage covers information from molecule to cell to tissue to organ to clinic, enabling Teacher-Net to learn complex progression patterns that blood markers alone cannot reveal (such as "local inflammation outbreak" shown by pathology indicating short-term rupture risk). Text features (O) explain mechanisms, clinical features (H) set baselines, and image / pathology (I / P) provide direct evidence, while blood markers (x) reflect real-time status — multiple sources of information complement and supplement each other, reducing single-modal noise interference. By embedding literature and clinical knowledge into image / pathology feature representations through fusion layers, the final knowledge representation implicitly contains cross-domain associations. When imitating the probability vector, Student-Net indirectly absorbs these embedded complementary knowledge. More comprehensive and robust feature inputs enable Teacher-Net to achieve prediction accuracy close to the theoretical upper limit, laying the foundation for distillation. Student-Net imitates the probability distribution of the "enhanced" Teacher, and can reproduce the logical judgment achieved by multi-modal fusion with only blood data, breaking the input modal restriction.

[0090] Five-dimensional data (images, pathology, blood, clinical, literature) and mechanism layer (O) and baseline layer (H) knowledge create a "super knowledge base" for Teacher-Net. This not only greatly improves the prediction ability of the teacher model itself, but more importantly, through the distillation process, these implicit cross-domain knowledge (especially text and clinical knowledge) is efficiently transferred to the Student-Net which only relies on blood markers, ultimately realizing the revolutionary effect of "blood test for multi-modal intelligence" - this is the core value of cross-modal knowledge distillation in the medical field.

[0091] Input feature matrix X:

[0092]

[0093] The integrated feature matrix is used in the training process of the teacher model to integrate CTA, pathology, protein fingerprint, public database information and text mining AI score modal features to generate comprehensive teacher knowledge:

[0094] where n is the sample size, .

[0095] In cross-modal knowledge distillation, this matrix ensures that the knowledge extracted by the teacher model from multi-modal data is distilled into the student model that only uses protein fingerprints, where public database information and text mining AI scores are input into the teacher model through the fusion layer along with CTA / pathology features, providing complementary knowledge of the mechanism layer and the baseline layer, thereby enhancing the knowledge transfer effect of the distillation process.

[0096] High-dimensional image and pathology data are extracted by deep network features (3D-ResNet, ViT-Tiny), and low-dimensional structured data is normalized, missing value filled and encoded in the feature engineering stage to avoid training instability caused by input distribution differences between different modalities. Orthogonal constraints and cross-modal independence verification (HSIC / CKA / VIF) are introduced to ensure that the five modalities remain low correlation after fusion, reducing the risk of overfitting. The model outputs the probability distribution of abdominal aortic aneurysm occurrence / progression / breakdown. The multi-dimensional output is adjusted by distillation temperature (T=4) and probability distribution alignment, so that the student model maintains >95% consistency with the teacher model in three types of prediction.

[0097] Preferably, the process of training the Teacher-net and the Student-net for the cross-modal knowledge distillation comprises:

[0098] Input CTA image into 3D-CNN (3D-Convolutional Neural Networks, three-dimensional convolutional neural network) + self-attention model, output rupture risk probability ;

[0099] Pathology slides are input into ViT-Tiny, and the output is the probability of tissue pathology risk ;

[0100] DeepSeek-Reasoner API is called based on public database information and text mining AI score, and the output is the probability of clinical risk and public database risk probability ;

[0101] The fusion layer uses the Transformer to output the category probability vector ;

[0102] The category probability vector output by Student-net is aligned with the category probability vector output by Teacher-net , and the distillation loss is used: , where σ is the softmax function, T is the distillation temperature, α is the loss weight coefficient, CE is the cross-entropy loss, KL represents the KL divergence, y true is the real label; Teacher-net receives the CTA and pathology feature output fusion risk vector

[0103] and its multi-layer attention weight , where is the attention distribution of the lth layer, the element takes value 0-1, indicating the importance allocation of different feature regions, satisfying row normalization, represents the probability of abdominal aortic aneurysm occurrence, represents the probability of abdominal aortic aneurysm progression, represents the probability of abdominal aortic aneurysm rupture.

[0104] ​In this embodiment, the focal point of the blood vessel wall local weak area (such as asymmetric bulge, intramural hematoma), capture the spatial characteristics of mechanical rupture risk, quantify the density of inflammatory cell infiltration, elastic fiber fracture index, etc., reflect the biological activity (microscopic driving force of progression / rupture), integrate epidemiological regularity (such as "diameter>5cm+smoking history=high rupture risk"), literature mechanism (such as "NINJ1 mediated inflammatory cascade"), output the statistical risk supported by evidence-based medicine and the risk associated with pathological mechanism. Three experts independently evaluate the risk from three aspects of physical structure, biological activity and clinical evidence, avoid single modality bias, and form a stereoscopic risk assessment system. Using the self-attention mechanism of Transformer, automatically learn the decision weight of different experts under different samples (such as: focus on imaging experts when the risk of rupture is high, and focus on pathological experts in the early stage of progression). Force the Student-net probability distribution to soften the distribution of Teacher-net (T>1 probability is smoother, implying the relationship between categories), ensure the correctness of the basic label. Teacher-net additionally outputs multi-layer spatio-temporal attention weights: : The attention distribution of the first layer network at time t (0-1 weight, row normalized). Student-net not only learns the probability output, but also approximates by constraining its internal attention distribution, inheriting the "thinking mode" of the teacher in analyzing the problem (such as: which combination of markers triggers a high-risk warning). t can be associated with follow-up time points, so that the model understands the time dependence of risk factors (such as a sudden increase in HbA1c indicating progression within 3 months).

[0105] For the characteristics of the same patient image with lesions and healthy segments, introduce convolution kernel size adaptation to the vascular cross-sectional scale in the CTA feature extraction stage, thereby enhancing the sensitivity to local morphological differences. The pathology modality optimizes Patch Embedding and attention head configuration based on ViT-Tiny to adapt to the color channels and high resolution of pathological sections, improving the recognition accuracy of inflammatory cell aggregation areas and elastic fiber fracture areas.

[0106] Teacher-net as a "super expert", integrates image, pathology, literature, and clinical data, and outputs a risk vector with biological mechanism and clinical operability. Attention weight Explicitly annotated decision basis to meet the explainability requirements of medical AI. Student-net realizes "mind cloning", and can reproduce the decision logic (through attention alignment) and conclusion (through probability alignment) of Teacher based on blood markers. Breakthrough "black box" restrictions: doctors can understand why blood indicators predict risks through the attention weights of Student-net. Through three core technologies of multi-expert collaborative decision-making, attention-enhanced distillation, and spatio-temporal modeling, Student-net becomes a "mind clone" of Teacher-net - it can output accurate predictions comparable to multi-modal fusion models using only blood tests, and has clinically explainable decision logic.

[0107] A "cross-modal knowledge distillation" two-stage framework is adopted. Since blood protein detection is the most accessible in practical applications, while CTA and pathological information is costly to obtain. By having the student model with only protein input learn from the teacher model that uses CTA + pathological features, the diagnostic ability of images / pathology is preserved, and lightweight inference is achieved at the deployment end. For the diverse data types in this study, we introduce both probability distribution alignment and attention weight transfer in the knowledge distillation loss. By applying a temperature parameter to the fusion output of the teacher model to generate soft labels, and calculating the KL divergence with the student model output, we achieve probability distribution matching and knowledge transfer, so that the student model learns the consistency of the teacher model's multi-class probability distribution, ensuring accuracy in multi-dimensional outputs such as occurrence, progression, and rupture) and attention weight transfer, and adjusting the distillation temperature to balance the multi-class probability output.

[0108] It should be noted that Student-net is deployed at the application end, with input: only input (MMP12, CALCA, UMOD, NINJ1, GHBA1C, FGF9, six protein molecules in blood standard expression). The structure includes: the first layer is Conv1D(kernel=3, filters=32) → BatchNorm → ReLU → MaxPool1D; the second layer is Conv1D(kernel=3, filters=64) → BatchNorm → ReLU → GlobalAvgPool; the third layer is FC-1(128 units, ReLU, Dropout=0.2); Softmax(3). Output: (Three types of probabilities of abdominal aortic aneurysm occurrence, progression, and rupture). Student-net fully inherits the high-dimensional knowledge of images / pathology with only 6-dimensional input, achieving lightweight deployment with inference delay ≤2ms and model volume 0.5MB.

[0109] Preferably, the process of training the Teacher-net and the Student-net for the cross-modal knowledge distillation further comprises:

[0110] To Apply target temperature to obtain soft labels, and calculate divergence with the Student-net output;

[0111] Measure the L2 distance of the attention of the Student-net and the attention of the Teacher-net , focus on the key high-risk mode to realize the transfer of attention.

[0112] Through the two core technologies of soft label knowledge rectification and attention focus transfer, the depth and fidelity of knowledge transfer in cross-modal distillation are significantly improved. The target temperature T>1 is applied to the original output probability distribution σ(z_t) of the Teacher-net to generate soft labels. The output distribution σ(z_s) of the Student-net is forced to approximate the soft label distribution (by minimizing the KL divergence). Hard labels (such as [0, 0, 1]) only indicate the final class, while soft labels (such as [0.1, 0.2, 0.7]) reveal the relative confidence of the teacher model for each class. Soft labels provide a smooth transition between classes, avoiding overfitting of the Student to noisy data and improving robustness in real clinical data. The high-risk feature patterns (such as "focal elastin fiber rupture + neutrophil cluster") identified by the Teacher through images / pathology are encoded as attention hotspots. Although the Student only sees blood data, its attention mechanism will automatically lock onto the corresponding high-risk biomarker combination. The Student shows the same pathological logic as the Teacher when predicting - when a specific marker combination appears, it activates a high-risk warning. The attention heat map of the Teacher can be mapped back to the image / pathology area, and the attention weight of the Student can be directly associated with the blood markers, making the prediction of the double model visualized and meeting the needs of clinically reliable AI.

[0113] Preferably, the attention transfer of the cross-modal knowledge distillation further comprises:

[0114] Extracting key attention maps from the multi-layer attention weights of the Teacher-net;

[0115] Introducing corresponding attention modules in the Student-net to calculate its attention output;

[0116] Using L2 distance as a loss function to quantify the difference in attention distribution between the two;

[0117] This loss is added to the total distillation loss, prompting the Student-net to adjust its parameters during training to minimize this distance, thereby transferring the Teacher-net's focus on key features.

[0118] Through cross-modal hard alignment of attention maps, pixel-level transfer of teacher model diagnostic thinking is achieved, making breakthrough progress in explainability reinforcement and high-risk feature transfer. The teacher's locked anatomical / pathological high-risk areas based on multi-modal data are encoded as transferable heat maps. Shallow attention focuses on basic features (such as texture), while deep layers focus on semantic features (such as rupture precursors), achieving full-dimensional knowledge transfer.

[0119] Preferably, the process of training the Teacher-net and the Student-net for the cross-modal knowledge distillation further comprises:

[0120] For Apply temperature T=4 to obtain soft labels , and the Student-net output Calculate divergence;

[0121] At the same time, use the real label y to calculate the cross-entropy CE to ensure classification accuracy;

[0122] Freeze the Teacher-net, and only update the Student-net parameters through backpropagation , until the validation set ΔAUC<0.01 converges, where ΔAUC represents the change in the area under the ROC curve, used to evaluate the performance improvement of the binary classification model.

[0123] The choice of distillation temperature T is based on empirical adjustment, usually T>1 to soften the probability distribution, making it easier for the student model to learn the relative probability relationship of the teacher model rather than hard labels; in experiments, different T values such as 1, 2, 4, 8 are tested through grid search or cross-validation, and T=4 is selected as the best value because it performs best in balancing softening degree and classification sharpness, avoiding distillation failure due to too small T or too uniform probability due to too large T.

[0124] Referring to Figure 5 , cross-modal independence verification includes:

[0125] HSIC (Hilbert-Schmidt Independence Criterion): Except for the same modal, the cross-modal HSIC value is close to 0.00~0.03, indicating that the modal is basically independent in the feature space, and the information complementarity is strong when fused.

[0126] CKA (Centered Kernel Alignment): Except for the diagonal line and a few combinations, the CKA value is generally 0.00~0.03, consistent with the results of HSIC, verifying the decoupling of multi-modal features in the representation space.

[0127] VIF (Variance Inflation Factor): The VIF values of all variables are <3, far below the commonly used threshold of 5, indicating that there is no multicollinearity, and each feature is relatively independent in explaining the result in prediction.

[0128] The three types of indicators consistently prove from the perspectives of nonlinear independence, feature representation similarity, and multicollinearity that the five modalities used are independent and complementary at the feature level, which helps to improve the generalization ability and stability of the fusion model and reduce the risk of overfitting.

[0129] Referring to Figure 6 , Figure 7 , Figure 8 , respectively, are the ROC curve, reliability calibration, and net benefit curve of the model. Discrimination: AUC 0.93, AUPRC 0.88, external 0.91 / 0.85 (95% CI). Calibration: Brier 0.072, CalibrationSlope 0.98. Clinical: DCA increases net benefit by 6-15% in the threshold 2-20% interval. Figure 6 The area under the ROC curve (AUC) in the Chinese ROC curve is close to 1.0, indicating that the model has excellent discrimination ability. The Youden index, at multiple thresholds, the true positive rate (sensitivity) and the true negative rate (specificity) are excellent, further verifying the ability of the model to distinguish positive and negative samples. The model performs outstandingly in identifying high-risk and low-risk individuals. Figure 7 The Brier coefficient value is 0.07, which is much lower than the critical value of 0.25, indicating that the predicted probability is highly consistent with the actual observation. The calibration slope value is 0.98, close to the ideal 1.0, indicating that the model does not have a systematic overestimation or underestimation of risk. The probability value predicted by the model is very reliable and can be directly used for clinical decision support. Figure 8 The net benefit curve (clinical benefit) threshold range is in the risk threshold range of 0%-10%, the net benefit (NetBenefit) of the model is always higher than the "full intervention" and "no intervention" strategies, and is significantly better than the traditional ultrasound method. The maximum net benefit (MAXNB) is 0.75, while the traditional ultrasound is only 0.10, indicating that under the same false positive rate, the model can reduce unnecessary interventions (such as biopsy) by about 85%. The model can significantly reduce over-treatment and improve decision-making efficiency in clinical applications.

[0130] Compared with the traditional CNN model, the input amount of information is reduced, and compared with the single Transformer model, the interpretability and robustness are improved, and the deployment cost is reduced. On the external validation set, the AUC is improved to 0.91 (about 6-8% higher than the single protein model), and the inference delay is ≤2ms, the model volume is only 0.5MB, suitable for bedside deployment.

[0131] Second embodiment

[0132] Based on the same concept, the present application also provides an abdominal aortic aneurysm progression prediction system based on cross-modal knowledge distillation, comprising:

[0133] A collection module is configured to collect blood of a target object.

[0134] A detection module is configured to detect blood protein fingerprint marker concentration of the blood of the target object, wherein the blood protein fingerprint marker comprises matrix metalloproteinase-12, calcitonin-related polypeptide-alpha, uromodulin, nerve injury-induced protein-1, glycosylated hemoglobin A1c, and fibroblast growth factor-9.

[0135] A risk calculation module is configured to invoke a Student-net of the cross-modal knowledge distillation to calculate abdominal aortic aneurysm risk based on the blood protein fingerprint marker concentration, wherein input of a Teacher-net of the cross-modal knowledge distillation comprises CTA image features and pathological section features, and a class probability vector output by the Student-net is aligned with a class probability vector output by the Teacher-net.

[0136] An output module is configured to output abdominal aortic aneurysm risk level and probability of three types of events of abdominal aortic aneurysm occurrence / progression / breaking in a preset year.

[0137] Based on the same concept, the present application also provides an electronic device, comprising:

[0138] A memory is configured to store a processing program.

[0139] A processor is configured to implement the abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation according to any one of the above embodiments when executing the processing program.

[0140] Based on the same concept, the present application also provides a readable storage medium, wherein the readable storage medium stores a processing program, and the processing program is configured to implement the abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation according to any one of the above embodiments when executed by a processor.

[0141] ​​If the abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation is realized in the form of program instructions and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments can be embodied in the form of software, and the computer software is stored in a storage medium, including a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-only memory, ROM), random access memory (Random access memory, RAM), magnetic disk or optical disk, and various storage program codes.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific execution of the system and device described above can refer to the corresponding process in the foregoing method embodiments.

[0143] The embodiments of the present application are described in detail above in combination with the drawings, but the present application is not limited to the above-described embodiments. Even if various changes are made to the present application, as long as the changes belong to the scope of the claims of the present application and its equivalent technologies, they still fall within the protection scope of the present application.

Claims

1. A method for predicting the progression of abdominal aortic aneurysm based on cross-modal knowledge distillation, characterized in that, Includes the following steps: Collect blood from the target subject; The concentration of blood protein fingerprint markers in the blood of the target object was detected. The blood protein fingerprint markers include matrix metalloproteinase-12, calcitonin-related peptide-α, uroregulatoryin, nerve injury-inducing protein-1, glycated hemoglobin A1c, and fibroblast growth factor-9. The risk of abdominal aortic aneurysm is calculated by calling the cross-modal knowledge distillation-based Student-Net based on the concentration of the blood protein fingerprint markers; wherein, the input of the cross-modal knowledge distillation-based Teacher-Net includes CTA image features and pathological slide features, and the output of Student-Net is a class probability vector. Compared with the class probability vector output by Teacher-Net Alignment; Outputs the risk level of abdominal aortic aneurysm and the probability of three types of events: occurrence, progression, and rupture of abdominal aortic aneurysm in a preset year; Before training the Teacher-Net with cross-modal knowledge distillation, a feature matrix needs to be constructed: the feature matrix is ​​constructed based on the CTA image features, the pathological slide features, the concentration of blood protein fingerprint markers, public database information, and text mining AI scoring. ,in For sample size, p represents the total dimension of the feature matrix X. I p represents the dimension of the features of a disease pathological slide image. P The dimension representing CTA imaging features or image-pathology fusion features, p O x represents the feature dimension of the remaining published documents. 1~6 The matrix represents the concentration data of matrix metalloproteinase-12, calcitonin-related peptide-α, uroregulatoryin, neuroinjury-inducing protein-1, glycated hemoglobin A1c, and fibroblast growth factor-9. D represents the aortic diameter grade in CTA images, I represents the convolutional features of CTA images, H represents the abdominal aortic aneurysm risk grading score, P represents the pathological section feature vector, which includes at least microstructural information such as inflammatory cell infiltration, elastic fiber breakage, and smooth muscle cell apoptosis, and O represents the mined features output by the Transformer text encoder from publicly available literature. The feature matrix is ​​used to input public database information and text mining AI scores into the teacher model through a fusion layer along with CTA image features and / or pathological section features, providing supplementary knowledge to the mechanism layer and baseline layer, thereby enhancing the knowledge transfer effect of the distillation process.

2. The method for predicting the progression of abdominal aortic aneurysm based on cross-modal knowledge distillation according to claim 1, characterized in that, Before training the Teacher-Net with cross-modal knowledge distillation, the CTA image features, pathological slide features, blood protein fingerprint marker concentrations, public database information, and text mining AI scores need to be preprocessed: 3D-ResNet-18 transfer learning implemented in PyTorch is used for the CTA image features, outputting a 256x1 fully connected layer, and SE-Attention is compressed to 128 dimensions; The pathological slide features were fine-tuned for 20 epochs using ViT-Tiny, and then 64 dimensions were obtained after Global Average Pooling. The pathological slides were processed into images of size 224×224, and ViT-Tiny divided the images into 16×16 patches. Z-normalization and missing value imputation were performed on the concentration of blood protein fingerprint markers, public database information, and text mining AI scores, respectively.

3. The method for predicting abdominal aortic aneurysm progression based on cross-modal knowledge distillation according to claim 1, characterized in that, The process of training the Teacher-Net and Student-Net with cross-modal knowledge distillation includes: Input CTA images into a 3D-CNN+self-attention model to output the probability of rupture risk. ; Input the pathological slides into ViT-Tiny, and it will output the probability of histopathological risk. ; Based on public database information and text mining AI scoring, the DeepSeek-Reasoner API is invoked to output the clinical risk probability. and the probability of risks in public databases ; The fusion layer uses Transformer to output class probability vectors. ; The class probability vector output by Student-net Compared with the class probability vector output by Teacher-Net Alignment, using distillation loss: Where σ is the softmax function, T is the distillation temperature, α is the loss weighting coefficient, CE is the cross-entropy loss, KL represents the KL divergence, and y true This is a real label; Teacher-net receives risk vectors from CTA and pathological feature outputs. and its multi-layer attention weights ,in Let L be the attention distribution of the l-th layer, with elements ranging from 0 to 1, representing the importance allocation of different feature regions, and satisfying row normalization. This indicates the probability of abdominal aortic aneurysm occurrence. Indicates the probability of abdominal aortic aneurysm progression. This indicates the probability of abdominal aortic aneurysm rupture.

4. The method for predicting the progression of abdominal aortic aneurysm based on cross-modal knowledge distillation according to claim 3, characterized in that, The process of training the Teacher-Net and Student-Net with cross-modal knowledge distillation also includes: right A soft label is obtained by applying a target temperature, and the divergence is calculated using the Student net output. Measuring Student-net Attention Teacher-Net Attention L2 distance, focusing on key high-risk patterns to achieve attention transfer.

5. The method for predicting abdominal aortic aneurysm progression based on cross-modal knowledge distillation according to claim 4, characterized in that, The attention transfer of the cross-modal knowledge distillation further includes: Extracting key attention maps from the multi-layer attention weights of Teacher-Net; Introduce the corresponding attention module into Student-net and calculate its attention output; The L2 distance is used as the loss function to quantify the difference in attention distribution between the two. Adding this loss to the total distillation loss prompts Student-net to adjust its parameters during training to minimize this distance, thereby transferring Teacher-net's focus on key features.

6. The method for predicting abdominal aortic aneurysm progression based on cross-modal knowledge distillation according to claim 4, characterized in that, The process of training the Teacher-Net and Student-Net with cross-modal knowledge distillation also includes: right Soft tags are obtained by applying a temperature of T=4. , with Student-net output Calculate the divergence; Simultaneously, the cross-entropy CE is calculated using the real label y to ensure classification accuracy; Freeze Teacher-net, and only update Student-net parameters during backpropagation. The convergence continues until the validation set ΔAUC < 0.01, where ΔAUC represents the change in the area under the ROC curve, used to evaluate the degree of performance improvement of the binary classification model.

7. A system for predicting the progression of abdominal aortic aneurysm based on cross-modal knowledge distillation, characterized in that, include: The acquisition module is used to collect blood from the target object; The detection module detects the concentration of blood protein fingerprint markers in the blood of the target object. The blood protein fingerprint markers include matrix metalloproteinase-12, calcitonin-related polypeptide-α, uroregulatory protein, nerve injury-inducing protein-1, glycated hemoglobin A1c, and fibroblast growth factor-9. The risk calculation module, based on the concentration of the blood protein fingerprint marker, calls the cross-modal knowledge distillation-based Student-Net to perform abdominal aortic aneurysm risk calculation; wherein, the input of the cross-modal knowledge distillation-based Teacher-Net includes CTA image features and pathological slide features, and the output of Student-Net is a class probability vector. Compared with the class probability vector output by Teacher-Net Alignment; The output module outputs the risk level of abdominal aortic aneurysm and the probability of three types of events: occurrence, progression, and rupture of abdominal aortic aneurysm in a preset year. Before training the Teacher-Net with cross-modal knowledge distillation, a feature matrix needs to be constructed: the feature matrix is ​​constructed based on the CTA image features, the pathological slide features, the concentration of blood protein fingerprint markers, public database information, and text mining AI scoring. ,in For sample size, p represents the total dimension of the feature matrix X, pI represents the dimension of the disease pathological slide image features, pP represents the dimension of the CTA image features or image-pathology fusion features, pO represents the feature dimensions of the remaining published literature, and x 1~6 The matrix represents the concentration data of matrix metalloproteinase-12, calcitonin-related peptide-α, uroregulatoryin, neuroinjury-inducing protein-1, glycated hemoglobin A1c, and fibroblast growth factor-9. D represents the aortic diameter grade in CTA images, I represents the convolutional features of CTA images, H represents the abdominal aortic aneurysm risk grading score, P represents the pathological section feature vector, which includes at least microstructural information such as inflammatory cell infiltration, elastic fiber breakage, and smooth muscle cell apoptosis, and O represents the mined features output by the Transformer text encoder from publicly available literature. The feature matrix is ​​used to input public database information and text mining AI scores into the teacher model through a fusion layer along with CTA image features and / or pathological section features, providing supplementary knowledge to the mechanism layer and baseline layer, thereby enhancing the knowledge transfer effect of the distillation process.

8. An electronic device, characterized in that, include: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores a processing program, which, when executed by a processor, implements the abdominal aortic aneurysm progression prediction method based on cross-modal knowledge distillation as described in any one of claims 1 to 6.

Images