Coronary arterial tumor fistula identification and myocardial scar evaluation system based on multi-modal fusion

Abstract

The invention relates to the field of medical image processing, in particular to a coronary artery fistula recognition and myocardial scar evaluation system based on multi-modal fusion, which comprises a CTA imaging data module, a CAG / DSA imaging data module and an image post-processing module. High-precision recognition and classification of coronary artery tumor and coronary artery sickle tumor are achieved, a CTA imaging data module collects coronary artery images, and a CAG / DSA imaging data module obtains dynamic blood flow data in combination with electrocardiogram monitoring. The image post-processing module completes accurate recognition and evaluation through cooperative work of five sub-modules, namely a CT blood vessel image processing sub-module, a coronary artery tumor recognition sub-module, a coronary artery sickle tumor recognition sub-module, an angiography image analysis sub-module and an electrocardiosignal analysis sub-module.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a coronary aneurysm fistula recognition and myocardial scar evaluation system based on multi-modal fusion, which is especially suitable for the recognition of coronary aneurysm and coronary aneurysmal stenosis and the evaluation of myocardial scar area. BACKGROUND

[0002] Coronary aneurysm refers to the local expansion of the coronary artery wall, with a diameter exceeding 1.5 times that of the adjacent normal blood vessel segment. Coronary aneurysm can occur at any site of the coronary artery, and can be single or multiple, with various shapes including cystic, fusiform and tubular. Coronary aneurysm often accompanies thrombosis, which can lead to severe complications such as myocardial ischemia and myocardial infarction, and has a significant impact on the heart function of patients.

[0003] Currently, the diagnosis of coronary aneurysm mainly relies on imaging examinations such as CTA (computed tomography angiography) and CAG / DSA (coronary angiography / digital subtraction angiography). However, the traditional diagnostic methods have the following shortcomings:

[0004] 1. The information provided by single modality images is limited, making it difficult to comprehensively and accurately evaluate the morphology, size of coronary aneurysm and its impact on myocardial function;

[0005] 2. Traditional image processing methods have low recognition rate and insufficient accuracy for complex morphological coronary aneurysms (such as coronary aneurysmal stenosis);

[0006] 3. There is a lack of comprehensive evaluation of the relationship between coronary aneurysm and myocardial scar, which cannot provide individualized risk stratification and treatment strategies.

[0007] Therefore, it is urgent to develop a system that can fuse multi-modal medical image information, accurately identify coronary aneurysm and coronary aneurysmal stenosis, and comprehensively evaluate the myocardial scar area, providing more comprehensive and accurate information support for clinical diagnosis and treatment. SUMMARY

[0008] The main purpose of the present application is to provide a coronary aneurysm fistula recognition and myocardial scar evaluation system based on multi-modal fusion, which can effectively fuse the image data of CTA and CAG / DSA two different modalities, combine with electrocardiogram signals, realize accurate identification of coronary aneurysm and coronary aneurysmal stenosis, and comprehensively evaluate the myocardial scar area.

[0009] The present application provides a coronary aneurysm fistula recognition and myocardial scar evaluation system based on multi-modal fusion, which comprises:

[0010] A CTA imaging data module for acquiring coronary CTA imaging data and performing routine film reading processing to generate coronary image data;

[0011] The CAG / DSA imaging data module is used to acquire CAG or DSA angiography images, monitor electrocardiogram signals 30 seconds before and after the angiography process, and combine the electrocardiogram signals with the angiography image data into video data.

[0012] The image post-processing module is connected to the CTA imaging data module and the CAG / DSA imaging data module respectively. It is used to receive the coronary artery image data and the video data, and process the coronary artery image data and the video data based on the spatial-frequency domain dual-path attention mechanism and differential geometric curvature flow analysis technology to identify coronary artery aneurysms, coronary artery falciform aneurysms and the imaging range of the left ventricle.

[0013] The image post-processing module includes:

[0014] The CT vascular image processing module is used to segment and extract features from the coronary artery image data;

[0015] The coronary aneurysm identification module is used to identify coronary aneurysms in the coronary artery image data based on a spatial-frequency domain dual-path attention mechanism, and to extract the diameter and volume of the coronary aneurysm;

[0016] A coronary artery falciform aneurysm identification module is used to identify and classify coronary artery falciform aneurysms in the coronary artery image data based on two-stage deep learning;

[0017] The contrast image analysis module is used to identify and quantify the left ventricular contrast area in the video data;

[0018] The electrocardiogram signal analysis module is used to identify the electrocardiogram signals extracted from the video data, and to calculate cardiac function indicators based on coronary artery aneurysm identification and morphological assessment.

[0019] The diagnostic output module is used to output data such as the coronary artery aneurysm, the coronary artery falciform aneurysm, and the left ventricular imaging range to the diagnostic display screen to complete the clinical diagnosis.

[0020] Preferably, the CT vascular image processing module uses a U-Net encoding and decoding network for segmentation processing. The U-Net encoding and decoding network employs medical image multi-level hierarchical fusion technology to perform hierarchical fusion of data from different modalities.

[0021] Preferably, the CT vascular image processing module extracts coronary artery images from the original image dataset through coronary artery image reconstruction, extracts features of the coronary artery images through the U-Net encoding and decoding network model, enhances feature extraction using an attention mechanism, and then restores the original resolution using convolution, pooling, and deconvolution operations.

[0022] Preferably, the coronary aneurysm identification module includes:

[0023] The spatial domain attention pathway is used to extract multi-scale spatial features from coronary artery image data and generate spatial feature maps.

[0024] The frequency domain attention pathway is used to perform frequency domain transformation on coronary artery image data, separate low-frequency, mid-frequency and high-frequency features, and generate frequency domain feature maps;

[0025] A dual-path feature fusion unit is used to adaptively weight the spatial feature map and the frequency domain feature map to generate fused features;

[0026] The differential geometry analysis unit is used to extract the geometric features of the coronary aneurysm surface based on curvature flow theory and generate a geometric invariant feature map.

[0027] An attention control unit is used to dynamically adjust the attention weights of the fused features based on the geometrically invariant feature map to generate enhanced features;

[0028] The feature classification unit is used to analyze the enhanced features and identify the location, shape, and size of the coronary aneurysm.

[0029] Preferably, the spatial domain attention pathway includes:

[0030] The multi-scale feature extraction network adopts an improved DeepLab V-Net network structure, which includes five levels of feature extraction layers, with each level having half the resolution.

[0031] The spatial adaptive attention module is used to calculate the importance weight of each location in the spatial feature map and dynamically adjust the attention intensity.

[0032] Feature enhancement unit, used to enhance spatial feature maps based on spatial attention weights.

[0033] Preferably, the frequency domain attention path includes:

[0034] A multi-channel frequency domain transformation unit is used to perform two-dimensional discrete Fourier transform on coronary artery image data;

[0035] A frequency band division unit is used to divide the spectrum into low-frequency, mid-frequency, and high-frequency regions.

[0036] The frequency band selective attention module is used to analyze the contribution of each frequency band to coronary aneurysm identification and dynamically allocate attention weights.

[0037] The frequency domain feature enhancement unit is used to enhance the frequency domain feature map based on frequency band attention weights.

[0038] Preferably, the differential geometry analysis unit includes:

[0039] The three-dimensional reconstruction unit is used to construct a three-dimensional mesh model of the coronary arteries and aneurysms based on coronary artery image data;

[0040] The curvature calculation unit is used to calculate the principal curvature, mean curvature, and Gaussian curvature of each vertex in a 3D mesh model.

[0041] A geometric invariant mapper is used to generate shape indices and curvature measures based on curvature information and to construct a geometric invariant feature map.

[0042] The curvature flow evolution analyzer is used to simulate morphological evolution under curvature flow, identify key evolution points, and generate curvature flow feature maps.

[0043] Preferably, the angiography image analysis module and the coronary aneurysm recognition module work in parallel. The angiography image analysis module performs noise reduction, binarization, residual background removal, erosion and hole removal, and edge detection processing on the CAG / DSA angiography image data in sequence. It extracts the heart's initial and final contours using grayscale projection and image segmentation techniques, dividing the image extracted from the angiography video data into left ventricular imaging and non-left ventricular imaging regions. By calculating the number of imaging pixels in the left ventricular imaging and non-left ventricular imaging regions, it determines the ratio of the left ventricular anteroposterior diameter and the end-systolic and end-diastolic volumes, thereby achieving a quantitative analysis of left ventricular function.

[0044] Preferably, the ECG signal analysis module obtains the frequency domain information of the ECG signal from the CAG / DSA angiography video data through Fast Fourier Transform, and uses Fourier linear interpolation to compare and analyze the frequency domain information before and after angiography.

[0045] Preferably, the coronary aneurysm recognition module trains and tests the coronary aneurysm dataset using cross-validation to construct a 3D multi-target region of interest for the coronary aneurysm. The trained model is then used to recognize the coronary aneurysm and its morphology. Bilinear interpolation is used to unify feature maps of different resolutions to the size of the original image. The dimensions of the feature maps are stacked and input into a classifier for prediction to obtain the diameter measurement of the coronary aneurysm.

[0046] The method for coronary aneurysm feature recognition includes: constructing a 3D region of interest (ROI) for the coronary aneurysm; constructing a coronary aneurysm segmentation dataset based on the 3D ROI; constructing training and testing data for the coronary aneurysm based on the segmentation dataset; constructing a coronary aneurysm recognition network, including a segmentation network, a recognition network, and a classification network; and accurately segmenting and classifying the coronary aneurysm.

[0047] The present invention has the following beneficial effects:

[0048] 1. Multimodal data fusion: This invention effectively fuses image data from two different modalities, CTA imaging and CAG / DSA angiography, overcoming the limitations of insufficient information from a single modality and providing more comprehensive diagnostic evidence.

[0049] 2. Improved recognition accuracy: Through the synergistic effect of spatial-frequency domain dual-path attention mechanism and differential geometric curvature flow analysis technology, the recognition accuracy of coronary aneurysm of the present invention is improved from 75% to 80% of the traditional method to 92% to 95%, especially the recognition ability of atypical coronary aneurysms is greatly improved.

[0050] 3. Enhanced Morphological Analysis Depth: Based on the morphological parameters (curvature distribution, shape index, etc.) provided by differential geometry theory, the system can more comprehensively characterize the geometric characteristics of coronary artery aneurysms, refining the classification of coronary artery aneurysms to 5-7 subtypes, which is far superior to the simple classification of traditional methods.

[0051] 4. Significantly improved diagnostic efficiency: This invention realizes a fully automated analysis process, reducing the diagnostic time from 15-20 minutes of manual analysis to 30-60 seconds, while improving diagnostic consistency through standardized processes.

[0052] 5. Personalized risk assessment: Based on a comprehensive analysis of the morphological characteristics of coronary artery aneurysms and myocardial function indicators, this invention can provide personalized risk grading and treatment recommendations, providing a more accurate basis for clinical decision-making. Attached Figure Description

[0053] Figure 1 This is a diagram illustrating the overall framework of the coronary artery aneurysm fistula identification and myocardial scar assessment system based on multimodal fusion of the present invention.

[0054] Figure 2 This is a schematic diagram of the CT vascular image processing module of the present invention. Detailed Implementation

[0055] Please refer to Figure 1 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0056] like Figure 1 As shown, the coronary artery aneurysm fistula identification and myocardial scar assessment system based on multimodal fusion provided by the present invention includes a CTA imaging data module 1, a CAG / DSA imaging data module 2, and an image post-processing module 3.

[0057] CTA imaging data module 1 is used to acquire the patient's coronary artery CTA imaging data and perform routine image reading processing in the clinical diagnostic system. After generating coronary artery image data, it is transmitted to image post-processing module 3. In a preferred embodiment of the invention, CTA imaging uses a 64-slice or higher multi-slice spiral CT scanner, with scanning parameters set as follows: slice thickness 0.5-0.625 mm, tube voltage 120 kV, tube current 350-450 mA, pitch 0.2-0.3, and reconstruction interval 0.4 mm. These parameter settings ensure the acquisition of high-resolution coronary artery images, providing a foundation for subsequent accurate identification.

[0058] CAG / DSA imaging data module 2 is used to acquire CAG or DSA angiography images of the patient. Depending on the patient's condition, electrocardiogram (ECG) signals are monitored for 30 seconds before and after the angiography procedure. The ECG signals are combined with the angiography image data to form video data, which is then transmitted to image post-processing module 3. Preferably, the CAG / DSA angiography uses a standard angiography procedure, with a contrast agent injection rate of 3-5 ml / s, a total volume of 5-10 ml, and an acquisition frequency of 30 frames / second to ensure the complete capture of the coronary artery dynamic perfusion process.

[0059] Image post-processing module 3 performs precise analysis of CTA and CAG / DSA imaging data to achieve accurate identification and quantitative assessment of coronary artery aneurysms, coronary artery falciform aneurysms, and the extent of left ventricular contrast enhancement. Specifically, image post-processing module 3 includes a CT vascular image processing module, a coronary aneurysm identification module, a coronary artery falciform aneurysm identification module, an angiography image analysis module, an electrocardiogram signal analysis module, and a diagnostic output module.

[0060] like Figure 2 As shown, the CT vascular image processing module uses the U-Net encoding and decoding network for segmentation processing. This U-Net encoding and decoding network employs medical image multi-level hierarchical fusion technology to perform hierarchical fusion of data from different modalities.

[0061] In one embodiment of the present invention, the CT vascular image processing module extracts the coronary artery image from the original image dataset through coronary artery image reconstruction, extracts the features of the coronary artery image through the U-Net encoding and decoding network model, enhances the feature extraction using an attention mechanism, and then restores the original resolution using convolution, pooling and deconvolution operations.

[0062] Specifically, the U-Net encoder-decoder network consists of an encoding path and a decoding path. The encoding path comprises four stages, each consisting of two 3×3 convolutional layers and a 2×2 max-pooling layer, with the number of channels starting at 64 and doubling in each stage. The decoding path also comprises four stages, each consisting of a 2×2 upsampling layer, a feature connection corresponding to the layer in the encoding path, and two 3×3 convolutional layers. Finally, a 1×1 convolutional layer is used to obtain the segmentation result.

[0063] To enhance the feature extraction capability of the network, this invention adds an attention mechanism after each convolutional layer of U-Net. This attention mechanism enhances key regions by calculating the importance weights of each location in the feature map, thereby improving the ability to identify small structures in the coronary arteries.

[0064] In practical applications, the processing flow of the CT vascular image processing module is as follows:

[0065] 1) Input: Standardized preprocessed CTA image data;

[0066] 2) Feature extraction: Extract multi-scale features through the encoding path;

[0067] 3) Feature enhancement: Apply attention mechanisms to enhance key features;

[0068] 4) Feature decoding: Restore the spatial resolution of the feature map through the decoding path;

[0069] 5) Output: Generates the segmentation results of the coronary arteries and their lesions.

[0070] The training of this module employs a combined loss function, including binary cross-entropy loss and Dice loss, to balance the class imbalance problem:

[0071] ,

[0072] in: For the total loss, For binary cross-entropy loss, For Dice's loss, The weighting factor is set to 0.5.

[0073] The binary cross-entropy loss is defined as:

[0074] ,

[0075] in: For binary cross-entropy loss, Total number of pixels For pixels The true label (0 or 1). For pixels The probability of being predicted as a prospect.

[0076] Dice loss is defined as:

[0077] ,

[0078] in: For Dice's loss, Total number of pixels For pixels The probability of being predicted as the foreground. For pixels The true label (0 or 1).

[0079] The coronary aneurysm identification module is the core innovation of this invention. It uses the synergistic effect of a spatial-frequency domain dual-path attention mechanism and differential geometric curvature flow analysis technology to achieve high-precision identification and classification of coronary aneurysms.

[0080] The coronary aneurysm identification module includes a spatial domain attention pathway, a frequency domain attention pathway, a dual-path feature fusion unit, a differential geometric analysis unit, an attention regulation unit, and a feature classification unit.

[0081] The spatial domain attention pathway is used to extract multi-scale spatial features from coronary artery image data and generate spatial feature maps. This pathway includes a multi-scale feature extraction network, a spatial adaptive attention module, and a feature enhancement unit.

[0082] The multi-scale feature extraction network employs an improved DeepLab V-Net network structure, containing five levels of feature extraction layers, with each level reducing the resolution by half. Preferably, the network has a 3×3 kernel size and a stride of 1, and uses BatchNormalization and ReLU activation functions to enhance training performance and feature representation capabilities. To capture features at different scales, residual connections are added between each layer to avoid loss of deep information.

[0083] The spatial adaptive attention module calculates the importance weight of each location in the spatial feature map and dynamically adjusts the attention intensity. This module generates a spatial weight map through a two-layer convolutional network, and the weight values ​​are mapped to the [0,1] interval using the Sigmoid function.

[0084] ,

[0085] in: For spatial location Attention weights This is the feature vector at that location. and For learnable convolution kernel parameters, For the Sigmoid function, To modify the activation function of the linear unit, This indicates a convolution operation.

[0086] The feature enhancement unit enhances the spatial feature map based on spatial attention weights. The enhancement operation employs a residual structure to ensure that the original features are not completely lost.

[0087] ,

[0088] in: For the enhanced features, Original features For attention weights, This represents element-wise multiplication.

[0089] The frequency domain attention pathway is used to perform frequency domain transformation on coronary artery image data, separating low-frequency, mid-frequency, and high-frequency features to generate a frequency domain feature map. This pathway includes a multi-channel frequency domain transformation unit, a frequency band division unit, a frequency band selective attention module, and a frequency domain feature enhancement unit.

[0090] The multi-channel frequency domain transform unit performs a two-dimensional discrete Fourier transform on the coronary artery image data. For three-dimensional image data, processing is performed layer by layer along the slice direction. The Fourier transform maps spatial domain information to the frequency domain, enabling the system to analyze the contributions of different frequency components.

[0091] ,

[0092] in: For frequency domain representation, For spatial domain images, and Image size, and For frequency variables, The imaginary unit, is the base of the natural logarithm.

[0093] The frequency band division unit divides the spectrum into a low-frequency region (0%–15%), a mid-frequency region (15%–50%), and a high-frequency region (>50%). This non-uniform division takes into account the different morphological characteristics of coronary aneurysms and their different manifestations in different frequency bands: the low-frequency region reflects the overall shape, the mid-frequency region reflects texture features, and the high-frequency region contains edge and detail information.

[0094] A frequency band-selective attention module analyzes the contribution of each frequency band to coronary aneurysm identification and dynamically allocates attention weights. This module first extracts global features for each frequency band through global pooling, and then calculates the importance of each frequency band through a fully connected layer.

[0095] ,

[0096] in: For frequency band Attention weights This is a global pooling operation. For frequency band Features and These are learnable weight parameters. For Sigmoid, ReLU is the modified linear unit activation function. This represents matrix multiplication.

[0097] The frequency domain feature enhancement unit enhances the frequency domain feature map based on frequency band attention weights. The enhancement operation is performed in the frequency domain, and then returned to the spatial domain via inverse Fourier transform.

[0098] ,

[0099] in: Features enhanced in the frequency domain , and These represent low-frequency, mid-frequency, and high-frequency characteristics, respectively. , and The attention weights for each frequency band are defined by IFFT, which stands for Inverse Fourier Transform. Represents element-wise multiplication. This indicates element-wise addition.

[0100] The dual-path feature fusion unit adaptively weights and fuses the spatial and frequency domain feature maps to generate fused features. The fusion process first performs feature alignment and normalization to ensure that the spatial and frequency domain features have the same dimensionality and numerical range.

[0101] Adaptive weight calculation dynamically adjusts the contribution ratio of spatial and frequency domain features based on the characteristics of the input image.

[0102] ,

[0103] ,

[0104] Where: α and β are fusion weights, satisfying α + β = 1; G_{pool} is the global pooling operation; F_{spatial} and F_{freq} are the spatial and frequency domain features, respectively; W_{} are the learnable weight parameters; σ is the Sigmoid function; and [⋅,⋅] represents feature connectivity. This represents matrix multiplication. The final fused features are obtained through weighted summation:

[0105] ,

[0106] in: The characteristics after fusion For spatial features, For frequency domain characteristics, + indicates element-wise multiplication, and + indicates element-wise addition.

[0107] In practical applications, the fusion weight α is usually dynamically adjusted between 0.4 and 0.6, and the weight is adaptively allocated according to the characteristics of the specific image to give full play to the complementary advantages of the two features.

[0108] The differential geometry analysis unit is another core innovation of this invention, used to extract the geometric features of the coronary aneurysm surface based on curvature flow theory and generate a geometrically invariant feature map. This unit includes a 3D reconstruction unit, a curvature calculation unit, a geometrically invariant mapper, and a curvature flow evolution analyzer.

[0109] The 3D reconstruction unit constructs a 3D mesh model of the coronary arteries and aneurysms based on coronary artery image data. This unit employs a watershed-based surface reconstruction algorithm, combined with morphological post-processing, to optimize mesh quality. To ensure accurate reconstruction of critical areas, the system adaptively refines the mesh in the coronary aneurysm region, with a mesh density of 0.2-0.3 mm, significantly higher than the 0.5-0.8 mm density in ordinary areas.

[0110] The curvature calculation unit calculates the principal curvature, mean curvature, and Gaussian curvature of each vertex in the 3D mesh model. For each vertex p on the mesh, the curvature is estimated by fitting a local quadratic surface:

[0111] ,

[0112] ,

[0113] ,

[0114] in: and Let p be the principal curvature of vertex p. Let p be the shape operator matrix. For the mean curvature, Let be the Gaussian curvature, and let Eigenvalues ​​represent the functions used to calculate the eigenvalues ​​of the matrix. Represents multiplication. Shape operator matrix. The fitting error is calculated using the 1-ring neighborhood of vertex p, and iterative optimization ensures that the fitting error is less than 0.001.

[0115] The geometric invariant mapper generates shape indices and curvature measures based on curvature information, constructing a geometric invariant feature map. Shape indices and curvature measures are important indicators describing local surface morphology and are unaffected by the viewing angle.

[0116] ,

[0117] ,

[0118] in: The shape index, with values ​​ranging from [-1, 1]. Curvature is a measure that characterizes the "degree of curvature" of a surface. Let π be the mathematical constant, and arctan be the arctangent function. The square root is represented. The shape index classifies the local morphology of coronary artery aneurysms into seven basic types: concave spherical (-1.0 to -0.75), concave groove (-0.75 to -0.5), concave saddle (-0.5 to -0.25), saddle (-0.25 to 0.25), convex saddle (0.25 to 0.5), convex groove (0.5 to 0.75), and convex spherical (0.75 to 1.0), providing a geometric basis for refined classification.

[0119] The curvature flow evolution analyzer simulates morphological evolution under curvature flow, identifies key evolution points, and generates curvature flow feature maps. Curvature flow describes the process of a surface moving along its normal at an average curvature rate, revealing the intrinsic stability of the morphology.

[0120] ,

[0121] in: Let p be the evolution rate of vertex p. For the mean curvature, For unit normal vector, This represents the dot product. The system simulates the evolution process using discrete time steps, with an adaptive step size of 0.01-0.05 and a maximum number of iterations of 50. It can also terminate early when the average displacement is less than 0.01 mm.

[0122] By analyzing curvature flow evolution, the system can identify morphologically unstable regions, which typically correspond to weak points or potential rupture risk areas of coronary aneurysms, providing important evidence for clinical risk assessment.

[0123] The attention control unit dynamically adjusts the attention weights of fused features based on geometrically invariant feature maps to generate enhanced features. This unit enables deep interaction between geometric features and image features, allowing the system to adaptively focus on key regions of coronary artery aneurysms of different morphologies.

[0124] First, the system generates a geometry-guided attention map based on geometric features:

[0125] ,

[0126] in: Let p be the geometric attention weight. The shape index, For curvature measurement, For curvature flow displacement, These are learnable weight parameters. For the Sigmoid function, Indicates feature connection, This represents matrix multiplication.

[0127] Then, the system applies geometric attention to the fused features to achieve geometrically guided feature enhancement:

[0128] ,

[0129] in: Features that are geometrically enhanced As a feature of fusion, Represents element-wise multiplication. This indicates element-wise addition.

[0130] In practical applications, the system incorporates a feedback mechanism to dynamically update attention weights based on recognition results, thereby optimizing the attention allocation strategy. Experiments show that the geometry-guided attention mechanism effectively improves the system's accuracy in recognizing complex coronary artery aneurysms, particularly significantly enhancing its ability to identify atypical morphologies.

[0131] The feature classification unit is used to analyze enhanced features and identify the location, shape, and size of coronary artery aneurysms. This unit employs a three-level cascaded classifier design to perform aneurysm detection, morphological classification, and parameter measurement, respectively.

[0132] During the aneurysm detection phase, the system employs a Faster R-CNN-based target detection algorithm to identify the location and extent of coronary artery aneurysms. The detection network backbone uses ResNet-50, with a Feature Pyramid Network (FPN) for multi-scale feature fusion and a Region Proposal Network (RPN) to generate candidate regions. A detection threshold of 0.75 is set to ensure high-precision detection.

[0133] In the morphological classification stage, the system classifies coronary artery aneurysms into several types, including saccular, tubular, and fusiform. The classification network adopts a multilayer perceptron structure, containing three fully connected layers with 512, 256 neurons, and a number of categories (7). The classification uses the Softmax function to calculate the probability of each category:

[0134] ,

[0135] in: Feature F belongs to category The probability, For category The corresponding weight vector, For input features, The total number of categories, is the base of the natural logarithm. Indicates the inner product. This indicates a summation.

[0136] During the parameter measurement phase, the system calculates key parameters of the tumor, such as diameter, volume, and surface area, based on the segmentation results. Diameter measurement employs the maximum inscribed sphere algorithm, volume calculation uses the voxel accumulation method, and surface area calculation uses the triangular mesh face accumulation method. The system samples 20-30 measurement points in each direction, taking the maximum value as the tumor diameter, achieving a measurement accuracy better than 0.3 mm.

[0137] The coronary artery falciform aneurysm identification module uses a two-stage deep learning network for identification and classification. The specific workflow of this module is as follows:

[0138] In the first stage, the model fuses feature maps of different resolutions through a feature pyramid pooling module, an attention module, and a deconvolution module to obtain coronary artery feature maps. Then, the model is trained using the cross-entropy function, and the loss function training formula is as follows:

[0139] ,

[0140] in: The value of the loss function. and Indicates category index, For true labels (0 or 1). To predict the probability of class j for the model, For the number of categories, It is the natural logarithm function.

[0141] In the second stage, the coronary artery falciform aneurysm identification module uses cross-validation to extract the 3D Region of Interest (ROI) of the coronary artery aneurysm through image segmentation, and establishes training and testing sets. The loss function of the two-stage network is constructed and the network is trained. This network is then used to segment the coronary arteries and identify coronary artery falciform aneurysms.

[0142] In practical applications, the coronary artery falciform aneurysm recognition module first trains a segmentation deep learning model using a loss function. Then, the model segments the test data. Based on the segmentation results, a 3D region of interest (ROI) for the coronary artery aneurysm is established. Next, slice images of the coronary artery falciform aneurysm are extracted based on the 3D ROI. Finally, the segmentation model is trained using a loss function and tested to achieve 3D image segmentation of the coronary artery falciform aneurysm.

[0143] Preferably, the neural network structure of this module includes two parts: an encoder and a decoder. The encoder contains four downsampling blocks, each consisting of two 3×3 convolutional layers, a batch normalization layer, a ReLU activation function, and a max-pooling layer. The decoder contains four upsampling blocks, each consisting of a deconvolutional layer, a feature connection, two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function.

[0144] The module is trained using the Adam optimizer with an initial learning rate of 0.001, which decays to 0.1 every 50 epochs. The batch size is 8, and the training lasts for 1000 epochs. Cross-validation uses a 5-fold method to ensure the model's generalization ability.

[0145] The angiography image analysis module and the coronary aneurysm identification module work in parallel to identify and quantify the left ventricular contrast area in the video data. This module first performs noise reduction, binarization, residual background removal, erosion to remove holes, and edge detection on the CAG / DSA angiography image data sequentially. Then, it extracts the initial and final contours of the heart using grayscale projection and image segmentation techniques, dividing the extracted image from the angiography video data into left ventricular contrast and non-left ventricular contrast regions. Finally, by calculating the number of imaging pixels in the left ventricular contrast and non-left ventricular contrast regions, it determines the ratio of the left ventricular anteroposterior diameter and the end-systolic and end-diastolic volumes, thus achieving a quantitative analysis of left ventricular function.

[0146] Specifically, noise reduction uses Gaussian filtering with a kernel size of 5×5 and a standard deviation of 1.0; binarization uses an adaptive thresholding method with a local window size of 11×11; erosion and hole removal use a 3×3 structuring element with 2 iterations; and edge detection uses the Canny operator with a low threshold of 50 and a high threshold of 150.

[0147] Gray-scale projection involves accumulating image pixel values ​​along the horizontal and vertical directions to obtain a gray-scale distribution curve. The location of the heart contour is determined by analyzing the peaks and troughs of the curve.

[0148] ,

[0149] ,

[0150] in: For the horizontal direction in position The grayscale projection value at that location, In the vertical direction at position The grayscale projection value at that location, For position Image pixel values ​​at that location, and These represent the width and height of the image, respectively.

[0151] Quantitative indicators of left ventricular function include:

[0152] 1) Left ventricular ejection fraction (LVEF):

[0153] ,

[0154] 2) Left ventricular anteroposterior diameter ratio (LAD / LAD):

[0155] ,

[0156] 3) Left ventricular end-systolic volume (ESV) and end-diastolic volume (EDV):

[0157] ,

[0158] ,

[0159] in: Left ventricular ejection fraction, This refers to the end-diastolic volume of the left ventricle. This refers to the end-systolic volume of the left ventricle. The ratio of the anteroposterior diameter of the left ventricle. This represents the final value of the anteroposterior diameter of the left ventricle. This is the initial value of the anteroposterior diameter of the left ventricle. This is a set of pixels at the end of left ventricular systole. This is the set of pixels during the end-diastolic phase of the left ventricle. Voxel volume Represents pixels It belongs to a set.

[0160] These indicators provide quantitative evidence for assessing cardiac function and are of great value in the clinical management of patients with coronary artery disease.

[0161] The ECG signal analysis module obtains the frequency domain information of the ECG signal from the CAG / DSA angiography video data through Fast Fourier Transform, and uses Fourier linear interpolation to compare and analyze the frequency domain information before and after angiography.

[0162] Specifically, the ECG signal is first bandpass filtered (0.5-40Hz) to remove baseline drift and high-frequency noise, and then the time-domain signal is transformed to the frequency domain using a Fast Fourier Transform (FFT).

[0163] ,

[0164] in: For the frequency domain representation of the first One frequency component, The first in the time-domain electrocardiogram signal The value of each sampling point, This represents the signal length (number of sampling points). The imaginary unit, is the base of the natural logarithm. Pi This indicates a summation.

[0165] The system uses Fourier linear interpolation to interpolate frequency domain information at different time points, achieving a smooth transition of frequency domain features.

[0166] ,

[0167] in: For interpolation at frequency Frequency value at that location, The first in the original spectrum The value of each frequency component For the first One original frequency point, For sine interpolation kernel function, This represents summation. The sine interpolation kernel function is defined as follows: .

[0168] By comparing the spectral changes of electrocardiogram signals before and after angiography, the system can detect the impact of changes in coronary blood flow on myocardial electrical activity, providing electrophysiological evidence for myocardial function assessment. The system pays particular attention to changes in the low-frequency components (0.04-0.15 Hz) and high-frequency components (0.15-0.4 Hz) of the power spectrum, which are closely related to autonomic nervous system regulation.

[0169] In addition, this module calculates a series of time-domain parameters, such as RR interval, QT interval, and ST segment shift, and combines these with frequency-domain analysis results to comprehensively assess cardiac function. These parameters are of great value in identifying ischemic areas, assessing infarction risk, and guiding clinical intervention.

[0170] The diagnostic output module outputs data such as coronary artery aneurysms, coronary falciform aneurysms, and the extent of left ventricular contrast enhancement to the diagnostic display screen, completing the clinical diagnosis. This module integrates the analysis results of the aforementioned modules, generating standardized diagnostic reports and providing comprehensive support for clinical decision-making.

[0171] The diagnostic report includes the following key information:

[0172] 1) Location, number, type, and size of coronary artery aneurysms;

[0173] 2) Identification and classification of coronary artery falciform aneurysms;

[0174] 3) Assessment of left ventricular function parameters and myocardial scarring;

[0175] 4) Risk assessment and treatment recommendations based on comprehensive analysis.

[0176] The system also provides 3D visualization capabilities, allowing doctors to observe the morphology and location of coronary artery aneurysms from different angles, enhancing the intuitiveness and accuracy of diagnosis.

[0177] The implementation details of the core components of the coronary aneurysm identification module are explained below.

[0178] The multi-scale feature extraction network for the spatial domain attention pathway employs an improved DeepLab V-Net architecture, comprising five levels of feature extraction layers, each with a resolution halved. This network uses dilated convolutions to increase the receptive field while maintaining feature map resolution; the convolution kernels are 3×3, with dilation rates of 1, 2, 4, 8, and 16, respectively. The network input is a preprocessed CTA image with dimensions of 512×512×(16-32), and the output consists of multi-level feature maps with dimensions of 1 / 1, 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image size.

[0179] The spatial adaptive attention module generates a spatial weight map through a two-layer convolutional network. The first layer uses a 1×1 convolution to reduce the number of channels to the middle channel (64), and the second layer uses a 1×1 convolution to reduce the number of channels to 1. Finally, the weights are mapped to the [0,1] interval using a sigmoid function. The threshold for dynamically adjusting the attention intensity is set to 0.3. When the feature complexity is higher than the threshold, the attention weight is increased, and vice versa, ensuring that the focus is accurately on complex regions.

[0180] The frequency band division of the frequency domain attention pathway adopts a non-uniform division strategy, taking into account the characteristic distribution characteristics of coronary artery aneurysms: the low-frequency region (0%–15%) contains overall shape information, the mid-frequency region (15%–50%) contains texture features, and the high-frequency region (>50%) contains edge and detail information. This division method has been experimentally verified and can separate different types of morphological features to the greatest extent.

[0181] The 3D reconstruction of the differential geometric analysis unit employs a watershed-based surface reconstruction algorithm combined with morphological post-processing. To ensure accurate reconstruction of the coronary aneurysm region, the system adaptively refines the mesh for this region with a mesh density of 0.2-0.3 mm, compared to 0.5-0.8 mm for ordinary regions. Curvature calculation utilizes a local quadratic surface fitting method, iteratively optimized until the fitting error is below 0.001.

[0182] The attention control unit enables deep interaction between geometric features and image features. The geometry-guided attention map generation employs a fully connected layer, with inputs including shape indices, curvature measures, and curvature flow displacement, and outputs attention weights. The system incorporates a feedback mechanism to dynamically update the attention weights based on the recognition results, optimizing the allocation strategy. Experiments show that the geometry-guided attention mechanism can improve the recognition accuracy of atypical coronary artery aneurysms from 82% to 94%.

[0183] The coronary aneurysm recognition module uses cross-validation to train and test the coronary aneurysm dataset, constructs a 3D multi-target region of interest for coronary aneurysms, and uses the trained model to achieve the recognition and morphological identification of coronary aneurysms.

[0184] Specifically, the method for identifying coronary artery aneurysm features includes the following steps:

[0185] 1) Construct the 3D region of interest for the coronary aneurysm: Based on the segmentation results, establish a bounding box around the coronary aneurysm and extend it by a certain margin (usually 20% of the aneurysm diameter) to ensure that the aneurysm and its surrounding tissues are completely included;

[0186] 2) Construct a coronary aneurysm segmentation dataset based on the 3D region of interest (ROI) of the coronary aneurysm: normalize the ROI region, adjust the resolution to a uniform size (64×64×64), and normalize the brightness range to [-1,1].

[0187] 3) Construct training and testing data for coronary artery aneurysms based on the coronary artery aneurysm segmentation dataset: 5-fold cross-validation was used, and data augmentation included random rotation (±15°), translation (±5 pixels), scaling (0.9-1.1), and contrast adjustment (0.9-1.1).

[0188] 4) Construct a coronary aneurysm recognition network, including a segmentation network, a recognition network, and a classification network: the segmentation network uses 3D U-Net, the recognition network uses an improved ResNet-18, and the classification network uses a multilayer perceptron;

[0189] 5) Precise segmentation and classification of coronary artery aneurysms: The segmentation network outputs aneurysm segmentation mask, the recognition network determines the location and boundaries of the aneurysm, and the classification network classifies the aneurysm into cystic, tubular, fusiform, and other types.

[0190] Bilinear interpolation is used to unify feature maps of different resolutions to the size of the original image.

[0191] ,

[0192] in: For position The interpolated eigenvalues, For position The original feature value at that location, Distance The four nearest grid points, It represents the absolute value.

[0193] The dimensions of the feature maps are stacked and fed into a classifier for prediction to obtain the diameter measurement of the coronary artery aneurysm. The diameter measurement uses the maximum inscribed sphere algorithm, sampling 20-30 measurement points in each direction and taking the maximum value as the aneurysm diameter, with a measurement accuracy better than 0.3 mm.

[0194] In practical applications, this method can accurately identify coronary artery aneurysms of various morphologies, including common saccular aneurysms (45%), tubular aneurysms (30%), fusiform aneurysms (15%), and irregular aneurysms (10%). For mixed aneurysms, the system uses morphological analysis to determine the primary and secondary components, providing more refined classification results.

[0195] The coronary artery aneurysm fistula identification and myocardial scar assessment system based on multimodal fusion provided by this invention has shown significant advantages in clinical applications:

[0196] 1) Recognition accuracy: The system achieves an accuracy rate of over 95% in recognizing common coronary artery aneurysms and over 90% in recognizing atypical aneurysms, far exceeding the 75%–80% accuracy rate of traditional methods;

[0197] 2) Morphological analysis accuracy: The system can subdivide coronary artery aneurysms into 7 subtypes, with morphological parameter measurement error less than 0.3 mm and volume measurement error less than 5%;

[0198] 3) Processing efficiency: The system can complete a comprehensive analysis of a patient in just 30-60 seconds, which is significantly shorter than the 15-20 minutes required for traditional manual analysis;

[0199] 4) Diagnostic consistency: The system provides standardized analysis processes and report formats, which improves the consistency and comparability of diagnostic results, increasing the diagnostic consistency among doctors from 70% to over 90%.

[0200] Clinical validation studies have demonstrated that this system possesses excellent identification and analysis capabilities for coronary artery aneurysms of different populations and morphologies, providing a powerful tool for the precision diagnosis and treatment of coronary artery diseases. Particularly for complex cases, the system's differential geometric analysis capabilities offer physicians deeper morphological insights, assisting in the development of personalized treatment plans.

[0201] The coronary artery aneurysm fistula identification and myocardial scar assessment system based on multimodal fusion provided by this invention adopts a modular design and can be seamlessly integrated into existing medical imaging workstations. The system has moderate hardware requirements and can run on standard workstations equipped with NVIDIA RTX 3080 or equivalent performance GPUs, with processing speeds meeting the needs of real-time clinical analysis.

[0202] The system supports the DICOM standard and is compatible with all mainstream CT and angiography equipment, requiring no additional hardware investment. The software architecture adopts a front-end / back-end separation design, supporting local deployment and cloud service modes, facilitating flexible application in medical institutions of different sizes.

[0203] This system has analyzed over 500 cases of coronary artery aneurysms, demonstrating excellent stability and reliability. User feedback indicates that the system significantly improves diagnostic efficiency and accuracy, and its ability to analyze complex cases has been highly praised by clinical experts.

[0204] In summary, the coronary artery aneurysm and fistula identification and myocardial scar assessment system based on multimodal fusion provided by this invention innovatively integrates a spatial-frequency domain dual-path attention mechanism with curvature flow analysis technology based on differential geometry, achieving high-precision identification and analysis of coronary artery aneurysms and coronary artery falciform aneurysms. Furthermore, by combining the synergistic analysis of electrocardiogram signals and angiographic images, it provides accurate assessment of myocardial function, offering advanced technical support for the precise diagnosis and treatment of coronary artery diseases.

[0205] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A system for coronary aneurysm fistula identification and myocardial scarring assessment based on multi-modal fusion, characterized in that, include: The CTA imaging data module is used to acquire coronary artery CTA imaging data and perform routine image processing to generate coronary artery image data. The CAG / DSA imaging data module is used to acquire CAG or DSA angiography images, monitor electrocardiogram signals 30 seconds before and after the angiography process, and combine the electrocardiogram signals with the angiography image data into video data. The image post-processing module is connected to the CTA imaging data module and the CAG / DSA imaging data module respectively. It is used to receive the coronary artery image data and the video data, and process the coronary artery image data and the video data based on the spatial-frequency domain dual-path attention mechanism and differential geometric curvature flow analysis technology to identify coronary artery aneurysms, coronary artery falciform aneurysms and the imaging range of the left ventricle. The image post-processing module includes: The CT vascular image processing module is used to segment and extract features from the coronary artery image data; The coronary aneurysm identification module is used to identify coronary aneurysms in the coronary artery image data based on a spatial-frequency domain dual-path attention mechanism, and to extract the diameter and volume of the coronary aneurysm; A coronary artery falciform aneurysm identification module is used to identify and classify coronary artery falciform aneurysms in the coronary artery image data based on two-stage deep learning; The contrast image analysis module is used to identify and quantify the left ventricular contrast area in the video data; The electrocardiogram signal analysis module is used to identify the electrocardiogram signals extracted from the video data, and to calculate cardiac function indicators based on coronary artery aneurysm identification and morphological assessment. The diagnostic output module is used to output the data of the coronary artery aneurysm, the coronary artery falciform aneurysm, and the left ventricular imaging range to the diagnostic display screen to complete the clinical diagnosis.

2. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 1, wherein, The CT vascular image processing module uses a U-Net encoding and decoding network for segmentation processing. The U-Net encoding and decoding network employs medical image multi-level hierarchical fusion technology to perform hierarchical fusion of data from different modalities.

3. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 2, wherein, The CT vascular image processing module extracts coronary artery images from the original image dataset through coronary artery image reconstruction, extracts features of the coronary artery images through the U-Net encoding and decoding network model, enhances feature extraction using an attention mechanism, and then restores the original resolution using convolution, pooling, and deconvolution operations.

4. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 1, wherein, The coronary aneurysm identification module includes: The spatial domain attention pathway is used to extract multi-scale spatial features from coronary artery image data and generate spatial feature maps. The frequency domain attention pathway is used to perform frequency domain transformation on coronary artery image data, separate low-frequency, mid-frequency and high-frequency features, and generate frequency domain feature maps; A dual-path feature fusion unit is used to adaptively weight the spatial feature map and the frequency domain feature map to generate fused features; The differential geometry analysis unit is used to extract the geometric features of the coronary aneurysm surface based on curvature flow theory and generate a geometric invariant feature map. An attention control unit is used to dynamically adjust the attention weights of the fused features based on the geometrically invariant feature map to generate enhanced features; The feature classification unit is configured to analyze the enhanced features and identify the location, shape and size of the coronary aneurysm.

5. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 4, wherein, The spatial domain attention path comprises: The multi-scale feature extraction network adopts an improved DeepLab V-Net network structure and comprises five feature extraction layers, each of which has a resolution reduced by half; The spatial adaptive attention module is configured to calculate the importance weight of each position in the spatial feature map and dynamically adjust the attention intensity. The feature enhancement unit is configured to enhance the spatial feature map based on the spatial attention weight.

6. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 4, wherein, The frequency domain attention path comprises: The multi-channel frequency domain transformation unit is configured to perform two-dimensional discrete Fourier transform on the coronary image data; The frequency band division unit is configured to divide the frequency spectrum into a low-frequency region, a medium-frequency region and a high-frequency region; The frequency band selective attention module is configured to analyze the contribution of each frequency band to the identification of the coronary aneurysm and dynamically allocate the attention weight. The frequency domain feature enhancement unit is configured to enhance the frequency domain feature map based on the frequency band attention weight.

7. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 4, wherein, The differential geometry analysis unit comprises: The three-dimensional reconstruction unit is configured to construct a three-dimensional mesh model of the coronary artery and aneurysm based on the coronary image data; The curvature calculation unit is configured to calculate the principal curvature, mean curvature and Gaussian curvature of each vertex of the three-dimensional mesh model; The geometric invariant mapper is configured to generate a shape index and a curvature measure based on the curvature information and construct a geometric invariant feature map; The curvature flow evolution analyzer is configured to simulate the morphological evolution under the curvature flow, identify the key evolution points and generate a curvature flow feature map.

8. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 1, wherein, The contrast image analysis module works in parallel with the coronary aneurysm identification module, and the contrast image analysis module sequentially performs noise reduction, binarization, residual background removal, corrosion hole removal and edge detection processing on the CAG / DSA contrast image data; the contrast video data is extracted by gray projection and image segmentation technology to extract the starting contour and ending contour of the heart, divide the extracted image into left ventricular visualization and non-left ventricular visualization regions, calculate the number of imaging pixel points in the left ventricular visualization and non-left ventricular visualization regions, determine the left ventricular anteroposterior diameter ratio and systolic and diastolic volume, and realize quantitative analysis of left ventricular function.

9. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 1, wherein, The electrocardiogram signal analysis module obtains the frequency domain information of the electrocardiogram signal by fast Fourier transform according to the electrocardiogram signal in the CAG / DSA contrast video data, and compares and analyzes the frequency domain information before and after the contrast using the Fourier linear interpolation method.

10. The multi-modal fusion based coronary fistula identification and myocardial scarring assessment system as claimed in claim 1, wherein, The coronary aneurysm identification module trains and tests the coronary aneurysm data set by the cross-validation method, constructs a 3D multi-target region of interest of the coronary aneurysm, uses the trained model to realize the identification of the coronary aneurysm and the morphological identification of the coronary aneurysm, uses the bilinear interpolation method to unify the feature maps of different resolutions to the size of the original image, stacks the dimensions of the feature maps, inputs them into the classifier for prediction, and obtains the diameter measurement of the coronary aneurysm. The method for identifying the characteristics of the coronary aneurysm comprises the following steps: constructing a 3D region of interest of the coronary aneurysm; constructing a coronary aneurysm segmentation dataset based on the 3D region of interest of the coronary aneurysm; constructing training data and test data of the coronary aneurysm based on the coronary aneurysm segmentation dataset; constructing a coronary aneurysm identification network, including a segmentation network, an identification network and a classification network; and performing accurate segmentation and classification on the coronary aneurysm.

Images