Coronary angiography calcification recognition method and system based on deep learning, and storage medium

Abstract

The application discloses a kind of coronary angiography calcification identification method, system and storage medium based on deep learning, it is related to medical cardiovascular image processing technical field.The present application comprises the following steps: the comprehensive evaluation of different blood vessel angiography projection position of the to-be-tested coronary artery;According to the CT angiography, mark the calcification lesion on the original digital subtraction angiography image independently;Using U-net neural network combined with focal loss function learning and identifying the marked calcification lesion;The U-net neural network output result is input into the LSTM network to calculate the calcification score.The present application improves the accuracy of coronary artery calcification detection and improves the prediction performance of machine learning algorithm.

Description

Technical Field

[0001] This invention relates to the field of medical cardiovascular image processing technology, and more specifically to a method, system, and storage medium for identifying coronary angiography calcifications based on deep learning. Background Technology

[0002] ECG-gated CT scans are generally considered the gold standard for quantifying coronary artery calcification; however, their widespread use is limited by factors such as cost, socioeconomic status, and medical conditions. Non-ECG-gated, non-angiography chest CT scans, with lower usage requirements and greater accessibility, have been shown to identify coronary artery calcification with comparable efficiency to ECG-gated CT scans, thus obtaining coronary artery calcification scores more efficiently. However, CT-derived calcification scores are often inaccurate in clinical practice and contain unavoidable subjective biases in treatment decisions. Coronary angiography is the gold standard for diagnosing coronary artery disease. Patients with severe calcification scores based on CT scans often require coronary angiography for final diagnosis and intervention. In clinical practice, the interpretation of coronary angiography images by different operators is highly variable and subjective. Determining the degree of coronary artery calcification during coronary angiography relies primarily on operator experience, which is unreliable and inaccurate and cannot be correlated with patient prognosis or conventional coronary artery calcification scoring. Summary of the Invention

[0003] In view of this, the present invention provides a method, system and storage medium for identifying coronary angiography calcifications based on deep learning, in order to solve the problems existing in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A deep learning-based method for identifying coronary angiography calcifications includes the following steps:

[0006] A comprehensive evaluation of the different imaging positions for coronary angiography is conducted.

[0007] Calcified lesions on raw digital subtraction angiography images were independently marked based on CT angiography.

[0008] U-net neural network combined with focus loss function was used to learn and identify labeled calcified lesions;

[0009] The output of the U-net neural network is input into the LSTM network to calculate the calcification score.

[0010] Optionally, the U-net neural network works as follows: four sets of convolution and downsampling operations are performed on the input image to obtain image feature information, and the image is magnified by four sets of deconvolution and upsampling on the right; skip connections are added between the downsampling and upsampling operations using the same number of channels, and the output of the left convolution block is connected to the input of the right deconvolution block of the same depth.

[0011] Optionally, the U-net neural network uses a loss function for supervision during network learning and training, providing feedback to the model on the quality of the training results.

[0012] Optionally, the U-Net neural network uses a focus loss function, which is expressed as follows:

[0013] ;

[0014] in Indicates the predicted probability. Represents the true value. Used to adjust the loss weights of the model for positive class samples. , These represent adjustment factors, used to adjust the penalty level of the model for positive and negative class predictions, respectively.

[0015] Optionally, by feeding all patient locations into an LSTM network, the dependence of calcification on location can be understood, and image fusion and calcification score prediction can be performed.

[0016] This invention also discloses a deep learning-based coronary angiography calcification identification system, comprising the following steps:

[0017] Comprehensive evaluation module: used to comprehensively evaluate different projection positions for coronary angiography of the arteries to be tested;

[0018] Calcification lesion annotation module: used to independently label calcification lesions on raw digital subtraction angiography images based on CT angiography;

[0019] Deep learning module: used to learn and identify labeled calcified lesions using the U-net neural network combined with a focus loss function;

[0020] Calcification scoring module: Used to input the output of the U-net neural network into the LSTM network to calculate the calcification score.

[0021] Finally, this invention discloses a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of any one of the deep learning-based coronary angiography calcification identification methods described above.

[0022] As can be seen from the above technical solutions, compared with the prior art, the present invention provides a method, system and storage medium for coronary angiography calcification identification based on deep learning, which improves the accuracy of coronary artery calcification detection and enhances the predictive performance of machine learning algorithms. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the overall architecture of the present invention;

[0025] Figure 2 This is a structural diagram of the deep learning method of the present invention;

[0026] Figure 3 This is a diagram showing the calcification segmentation results obtained using the improved U-Net according to the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This invention discloses a deep learning-based method for identifying coronary angiography calcifications, comprising the following steps:

[0029] S1. Conduct a comprehensive evaluation of different angiography projection positions for the coronary arteries to be examined;

[0030] S2. Independently mark calcified lesions on the original digital subtraction angiography images based on CT angiography;

[0031] S3. Use the U-net neural network combined with the focal loss function to learn and identify labeled calcified lesions;

[0032] S4. Input the output of the U-net neural network into the LSTM network to calculate the calcification score.

[0033] like Figure 1 As shown, the method in this embodiment is divided into two parts: calcification segmentation and identification, and calcification integral calculation.

[0034] Calcification Segmentation and Recognition: To improve the accuracy of coronary artery calcification detection and enhance the predictive performance of machine learning algorithms, calcification was annotated in at least two projection positions of the major vessels for each patient. Based on the annotated projection positions, a comprehensive evaluation of the coronary arteries was performed using at least five different angiographic projection positions. At least three projection positions were used to evaluate left coronary artery vessels, including CRA (head position), CRA+LAO (head + left anterior oblique), CRA+RAO (head + right anterior oblique), CAU (foot position), CAU+LAO (foot + left anterior oblique), and CAU+RAO (foot + right anterior oblique); at least two projection positions were used to evaluate right coronary artery lesions, including LAO, RAO, LAO+CRA, and LAO+CAU. The projection standards used in this invention are consistent with international clinical standards. Three cardiologists independently labeled calcified lesions on the original DSA images according to CTA locations. U-Net combined with a focus loss function method was used to learn and identify the calcified lesions labeled on coronary angiography.

[0035] LSTM Calculation of Calcification Indices: A single image cannot accurately reflect a patient's true calcification status. Generally, 3-5 images from different locations are needed for analysis. How to store and process this information is crucial for the accuracy of calcification score calculation. LSTM (Long Short-Term Memory) is a special type of Recurrent Neural Network (RNN) that can solve the gradient vanishing and exploding problems in traditional RNNs, thus better handling long sequence data. By modeling image sequences, LSTM can capture long-term dependencies in images, thereby improving model accuracy. By convolving all patient images in a certain order to extract features and then feeding them into the LSTM network, the network understands the dependency of calcification on location and performs image fusion and calcification score prediction.

[0036] In this embodiment, U-Net first performs four sets of convolution and downsampling operations on the input image to obtain image feature information, and then enlarges the image through four sets of deconvolution and upsampling on the right. Figure 2U-Net adds skip connections between downsampling and upsampling operations using the same number of channels, which helps the decoder better recover target details. The output of the left convolutional block is connected to the input of the right deconvolutional block of the same depth. The input feature map size of the deconvolutional block remains unchanged, but the number of channels is doubled. During network learning and training, the loss function acts as a supervisor, providing feedback to the model on the quality of the training results. The model continuously adjusts and optimizes the training strategy based on the feedback to achieve the best training effect. The U-Net network uses the pixel cross-entropy loss function CE (Cross Entropy Loss), which is expressed as follows:

[0037] (1);

[0038] in Indicates the predicted probability. This represents the true value. When the segmentation is completely correct, It is 0. The larger the pixel size, the worse the segmentation performance of the network. When the number of pixels in the current scene is much smaller than the number of pixels in the background, the background element component in the loss function dominates, causing the model to heavily favor the background, resulting in poor model training and prediction performance. Segmentation of calcified regions in coronary angiography is a binary segmentation problem, only applicable to the segmentation of calcified regions and background regions; therefore, the traditional U-Net network performs poorly in this area.

[0039] The key to solving this problem is to make the model more focused on learning the foreground objective. The focus loss function is designed based on this, and its expression is as follows:

[0040] (2);

[0041] set up Substituting into formula (2), we obtain formula (3):

[0042] (3);

[0043] Similarly, (1) can be unified as:

[0044] (4);

[0045] This represents a value close to the true value. The larger the value and the closer it is to y, the more accurate the classification. Used to adjust the loss weights of the model for positive class samples. , These are adjustment factors, used to adjust the penalty level of the model for positive and negative class predictions, respectively. Comparing expressions (3) and (4), the focus loss has an additional adjustment factor compared to cross-entropy, namely... For samples with accurate classification, Close to 1, and Close to 0; for inaccurately classified samples, Approaching 1 Close to 0.

[0046] Compared to cross-entropy loss, the focus loss value remains unchanged for inaccurately classified samples, but decreases for accurately classified samples. It increases the weight of inaccurately classified samples in the loss function. Therefore, during training, the weight of accurately classified background continues to decrease, while the weight of foreground objects continues to increase. Focus loss achieves better training results by balancing the learned weights of positive and negative samples during training.

[0047] Multiple patient images are sequenced in a fixed order and convolutional operations are performed to extract features. These images are then fed into an LSTM network, where nested localization is performed, and computation is conducted through fully connected (FC) linear layers. The final calcification score is output as a score called CACag (Coronary artery calcium score based on angiography).

[0048] The data includes 4955 annotated coronary angiography images, with at least 5 projections per patient. Images were rotated, flipped, and denoised during analysis to increase sample size. A U-Net network with a focus loss function was used to balance calcification segmentation between positive and negative samples, such as... Figure 3 As shown in Table 1, the calcification segmentation performance of the U-Net network with the addition of a focus loss function is significantly better than that of Deep Lab v3+ and Pyramid Scene-Parsing Network (Psp Net). To verify the effectiveness of the focus loss, ablation experiments were conducted. Alpha is the balancing parameter of the focus loss function, used to adjust the model's attention to learn positive and negative samples. Before introducing the focus loss, the U-Net network focused too much on the background, making it difficult to segment calcified regions. Introducing the focus loss and adjusting the balancing parameter significantly improved the calcification segmentation performance. It was found that the algorithm performed best when Alpha was 0.9 (see Table 2).

[0049] Table 1. Comparison of coronary artery calcification segmentation results using the improved U-Net network with segmentation results from other models.

[0050]

[0051] Table 2. Parameter Adjustment and Performance Comparison in the Improved U-Net Network

[0052]

[0053] Finally, in the development group, 90 patients completed the labeling process to obtain the predicted CACag, and in the validation group, all 371 patients were able to calculate CACag. Due to artifacts during angiography, four patients failed to reach CACag. The linear regression model results indicated a linear relationship between CACag and the actual CAC value (Coef = 0.249, p < 0.05). This demonstrates the rationality of the method proposed in this invention.

[0054] This embodiment also discloses a deep learning-based coronary angiography calcification identification system, including the following steps:

[0055] Comprehensive evaluation module: used to comprehensively evaluate different projection positions for coronary angiography of the arteries to be tested;

[0056] Calcification lesion annotation module: used to independently label calcification lesions on raw digital subtraction angiography images based on CT angiography;

[0057] Deep learning module: used to learn and identify labeled calcified lesions using the U-net neural network combined with a focus loss function;

[0058] Calcification scoring module: Used to input the output of the U-net neural network into the LSTM network to calculate the calcification score.

[0059] Finally, this embodiment discloses a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of any one of the deep learning-based coronary angiography calcification identification methods.

[0060] Taking the coronary artery of the heart as an example, the technical solution of the present invention is further explained, specifically including the following steps:

[0061] S1. Obtain coronary angiography images;

[0062] S2. Use the U-Net neural network and Focal loss function to learn calcification features;

[0063] S3. Use LSTM neural network to predict patient calcification score.

[0064] Furthermore, in S1, a coronary angiography image dataset is used. This dataset consists of Dicom coronary digital subtraction angiography files from a number of patients (e.g., 100) with coronary artery disease. Each patient has multiple Dicom files in different positions, and each Dicom file contains several frames of coronary angiography. The Dicom files are exported as single-frame JPG images. For the pre-contrast images, the physician annotates the calcified areas based on the differences between the pre- and post-contrast images. This data is then used to train a network model, which is subsequently used to predict the patient's calcification score.

[0065] In S2, the U-net neural network is structured as follows:

[0066] The input layer receives preprocessed coronary angiography images, with an image size of 512x512 pixels.

[0067] The encoder (downsampling path) is used to extract image features using VGG16 or ResNet50 as the backbone network. The features are gradually reduced in dimensionality through successive convolutional and pooling layers.

[0068] A bridging layer is used for feature fusion between the encoder and decoder.

[0069] The decoder (upsampling path) is used to gradually restore the low-dimensional feature map to the resolution of the original image through progressive upsampling and convolution operations.

[0070] The output layer is used to output the calcified region segmentation result of the image through a 1x1 convolutional layer, with the number of output channels equal to the number of categories (e.g., background and calcified regions).

[0071] The training process for the U-net neural network is as follows:

[0072] Dataset selection: A dataset of cardiac coronary angiography images with calcification markers was used, and the images were preprocessed, including normalization, resizing, and data augmentation (such as rotation, scaling, and flipping).

[0073] Training Details: Loading a Pre-trained Model: If you choose to use a pre-trained model, you need to load the corresponding pre-trained weights. This can be achieved by setting "pretrained=True" when calling the network initialization function.

[0074] Loss function: The cross-entropy loss function (dice) and Focal loss are used, which are suitable for calcification segmentation tasks.

[0075] Optimizer: The Adam optimizer is used, which has the ability to dynamically adjust the learning rate.

[0076] Training cycles: Based on the size and complexity of the dataset, perform a sufficient number of training cycles to ensure convergence.

[0077] More specifically, in this embodiment, coronary angiography images are input into the U-Net network. The U-Net network can choose either VGG16 or ResNet50 as its backbone. In this embodiment, ResNet50 is chosen as the backbone because it performs well in image segmentation tasks. The network's entry point is a series of convolutional layers used to extract preliminary features from the image. Taking the first convolution as an example, the first convolutional layer typically has 64 kernels, a kernel size of 7×7, and a stride of 2. This step is followed by a max pooling operation, reducing the image size from 512×512 to 256×256. Each convolutional operation is usually followed by an activation function, such as ReLU, to introduce non-linearity.

[0078] Following the convolutional layers, the network applies batch normalization to stabilize the learning process and reduce internal covariate shifts. These features are then further activated using activation functions such as ReLU. The U-Net network fuses feature maps from deep and shallow layers through skip connections. Before upsampling (or upinterpolation), feature maps from deeper layers are concatenated with those from shallower layers; this step is called a "skip connection." This fusion helps restore the spatial resolution of the image and preserves more image detail. In the U-Net network, upsampling (or transposed convolution) is used to gradually increase the size of the feature maps until the desired output size is reached. During upsampling, feature alignment is achieved by concatenating deep feature maps with the upsampled shallow feature maps.

[0079] After feature alignment, the network further refines these fused features through a series of convolutional layers to facilitate more precise calcification region segmentation. Finally, a 1x1 convolutional layer maps the refined features to the calcification segmentation categories, yielding the final segmentation result.

[0080] During training, since calcified regions are relatively small samples compared to the background, focal loss is used as the loss function. This function assigns greater weight to calcified regions that are difficult to classify, while reducing the weight of easily classified samples (such as irrelevant background). The segmentation results are optimized through post-processing steps, including applying a threshold to determine the final location of the calcified regions. The threshold can be selected based on clinical criteria or by determining the optimal value experimentally.

[0081] In S3, the LSTM neural network includes an input layer, LSTM units, fully connected layers, and an output layer.

[0082] The input layer is used to input data: the data input into the LSTM network is usually sequence data. In this embodiment, the input is the calcification segmentation results of 3-5 body positions of a patient, which are extracted from the calcification regions segmented from U-Net.

[0083] LSTM cells include configuration, hiding, and cells.

[0084] Configuration: An LSTM network consists of multiple LSTM units, each containing three gating structures—a forget gate, an input gate, and an output gate. These gating mechanisms help the network learn when to forget old information and when to update new information.

[0085] Hidden State: Each LSTM unit maintains a hidden state, which is propagated during sequence processing and can capture the temporal dynamics of the sequence.

[0086] Unit State: Each unit also maintains a unit state, which allows the network to pass state information across time steps, solving the gradient vanishing problem of traditional recurrent neural networks (RNNs).

[0087] Fully connected layers are used to process data through one or more fully connected (dense) layers to form the final output. These layers can be tailored to specific tasks, such as classification or regression.

[0088] Furthermore, the purpose of the fully connected layer is to transform the output of the LSTM unit into a specific prediction value, such as the prediction of the calcium integral.

[0089] The output layer is responsible for generating the final prediction, such as a calcified Agatston score. The output can be a continuous numerical value, suitable for regression tasks.

[0090] It should be noted that the output layer uses an appropriate activation function to ensure that the output values ​​conform to the expected range or format. In calcification integral prediction, if it is a regression problem, an activation function may not be needed or a linear activation function may be used.

[0091] In this embodiment, mean squared error (MSE) or other loss functions suitable for regression tasks are used to train the network. The Adam optimizer is then used to efficiently update the network weights and minimize the loss function.

[0092] The output of the U-Net network is used as the input to an LSTM network to predict the calcification integral. The LSTM network can process the sequential feature data provided by the U-Net network and predict the calcification integral. The entire network is trained on a dataset of segmented coronary angiography images. The network weights are updated using a backpropagation algorithm and an optimizer (such as Adam) to minimize the loss function.

[0093] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0094] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A deep learning-based method for identifying coronary angiography calcifications, characterized in that, Includes the following steps: A comprehensive evaluation of the different imaging positions for coronary angiography is conducted. Calcified lesions on raw digital subtraction angiography images were independently labeled based on CT angiography; wherein, calcifications were labeled in multiple projection positions for each patient's vessels; the multiple projection positions were used to evaluate the left and right coronary arteries. Using the multiple imaging positions as input, the U-net neural network combined with the focal loss function is used to learn and identify the marked calcified lesions, and the segmentation results of the multiple imaging positions are output. The output of the U-net neural network is input into the LSTM network to calculate the calcification score.

2. The method for identifying coronary angiography calcifications based on deep learning according to claim 1, characterized in that, The U-net neural network works as follows: it performs four sets of convolution and downsampling operations on the input image to obtain image feature information, and then enlarges the image through four sets of deconvolution and upsampling on the right. It adds skip connections between the downsampling and upsampling operations using the same number of channels, and connects the output of the left convolution block to the input of the right deconvolution block of the same depth.

3. The method for identifying coronary angiography calcifications based on deep learning according to claim 1, characterized in that, During the learning and training process, the U-net neural network uses the Loss function for supervision, providing feedback to the model on the quality of the training results.

4. The method for identifying coronary angiography calcifications based on deep learning according to claim 1, characterized in that, The U-Net neural network uses a focal loss function, which is expressed as follows: ； in Indicates the predicted probability. Represents the true value. Used to adjust the loss weights of the model for positive class samples. , These represent adjustment factors, used to adjust the penalty level of the model for positive and negative class predictions, respectively.

5. The method for identifying coronary angiography calcifications based on deep learning according to claim 1, characterized in that, By feeding all patient locations into an LSTM network, the dependence of calcification on location can be understood, and image fusion and calcification score prediction can be performed.

6. A deep learning-based coronary angiography calcification identification system, characterized in that, Includes the following steps: Comprehensive evaluation module: used to comprehensively evaluate different angiography projection positions of the coronary arteries to be tested; Calcified lesion annotation module: used to independently annotate calcified lesions on raw digital subtraction angiography images based on CT angiography; wherein, calcification annotation is performed on multiple projection positions of the vessels for each patient; the multiple projection positions are used to evaluate the left coronary artery and the right coronary artery; Deep learning module: Takes the multiple imaging positions as input, uses the U-net neural network combined with the focal loss function to learn and identify the marked calcified lesions, and outputs the segmentation results of multiple imaging positions; Calcification scoring module: Used to input the output of the U-net neural network into the LSTM network to calculate the calcification score.

7. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of a deep learning-based coronary angiography calcification identification method as described in any one of claims 1-5.

Images