Method, system and storage medium for automatic extraction of coronary arteries based on deep learning

Abstract

The application discloses a kind of based on deep learning automatic extraction coronary artery method, system and storage medium, it is related to medical image processing field.The present application includes the following steps: ICA image is input into ICASEG-Net, and the output obtained is converted into probability graph by sigmoid function;By comparing the difference between probability graph and preset binary artery mask, the network weight is optimized and the optimal weight is saved, and the coronary artery extraction model is obtained;The image to be extracted is input into the trained coronary artery extraction model, and the output probability graph is converted into binary artery segmentation mask using OTSU algorithm, wherein, pixel value 1 indicates coronary artery, and pixel value 0 describes background.Compared with existing methods, the present application significantly improves the performance of vessel segmentation and has high robustness.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and more specifically to a method, system, and storage medium for automatically extracting coronary arteries based on deep learning. Background Technology

[0002] Coronary artery disease (CAD) is a type of coronary artery disease that includes angina and acute myocardial infarction. It can lead to coronary artery stenosis due to the accumulation of atherosclerotic plaques in the epicardial arteries. Coronary artery stenosis results in relative myocardial hypoxia, often leading to ischemia. Severe myocardial ischemia can cause serious symptoms such as angina or even myocardial infarction. Invasive coronary angiography (ICA) is an important tool for assessing arterial stenosis and plaque characteristics and is used in the evaluation and treatment of CAD. Automated extraction of coronary arteries from the ICA is the first step before stenosis detection and plays a crucial role in clinical practice in interventional cardiology. However, accurately extracting coronary arteries from the ICA is a challenging task due to: 1) complex noise caused by non-uniform illumination; 2) poor signal-to-noise ratio; 3) non-uniform intensity; and 4) semantic information confusion. Automated vessel extraction techniques can be broadly classified into two categories: traditional image processing and deep learning-based methods. The former includes methods based on Gaussian or Gabor filters, model-based methods, and line-tracking methods, primarily focusing on low-noise and high-contrast vessel extraction. However, these methods fail to achieve satisfactory results when dealing with extremely noisy ICA (Intracoronary Artery Aspect) problems. The latter primarily employs convolutional neural network (CNN)-based methods, demonstrating superior feature extraction performance. Yang designed a method for automatic coronary artery removal using correspondence matching and CNN, reporting a Dice score of 0.8007. Nasr-Esfahani used pixel-based plaques to evaluate ICA and designed a CNN to identify the artery and background. To further improve segmentation accuracy, E. Nasr et al. combined two CNNs for primary and secondary features, achieving a Dice score of 0.8151. CNN-based image segmentation methods, such as U-Net and U-Net++, have also achieved remarkable results. Huang et al. proposed a novel U-Net 3+, a fully connected U-Net for medical image segmentation. Zhao et al. proposed FP-U-Net++ by introducing feature pyramid techniques into U-Net++, achieving a Dice score of 0.8899.

[0003] Although existing methods have achieved positive results, the accuracy of artery segmentation can be further improved. Summary of the Invention

[0004] In view of this, the present invention provides a method, system, and storage medium for automatically extracting coronary arteries based on deep learning. It employs a network structure based on ICASEG-Net to extract sufficient information from full-scale data. First, an encoder structure based on residual and Inception modules is considered, and then transfer learning is used to improve learning efficiency. To achieve excellent performance, a hybrid loss function that encourages network convergence and penalizes discrepancies is utilized during the training phase.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for automatically extracting coronary arteries based on deep learning includes the following steps:

[0007] The ICASEG-Net neural network was trained using ICA images to obtain a model for extracting coronary arteries;

[0008] The image to be extracted is input into the coronary artery extraction model, and the output is converted into a probability map using the sigmoid function;

[0009] The network weights are optimized by comparing the differences between the probability map and the preset binary artery mask;

[0010] The OTSU algorithm is used to convert the probabilistic map into a binary artery segmentation mask, where a pixel value of 1 represents a coronary artery and a value of 0 represents the background.

[0011] The ICASEG-Net neural network consists of an encoder and a decoder. The encoder comprises convolutional layers, max-pooling layers, and Inception A, Inception-ResNet-A, Inception-ResNet-B, and Inception-ResNet-C blocks. The encoder is used to extract feature maps from the input ICA image and to extract high-level semantic information. The decoder module consists of convolutional layers and upsampling. The decoder is used to recover the high-level semantic information extracted from the feature encoder; low-level feature maps extracted from the encoder path are concatenated to the decoder path. ICASEG-Net achieves this through full-scale skip connections, ensuring that each decoder layer contains feature maps from both the encoder and decoder at different scales.

[0012] Optionally, the Inception A block includes four branches; one branch includes an average ensemble layer and a convolutional layer connected in sequence; the other three branches are composed of convolutional layers with different kernel sizes connected in sequence.

[0013] Optionally, the Inception-ResNet-A block includes four branches: the first branch outputs directly; the second branch is processed by a 1×1 kernel 2D convolution; the third branch is processed by 1×1 kernel and 3×3 kernel 2D convolutions in sequence; and the fourth branch is processed by 1×1 kernel 2D convolution and two 3×3 kernel 2D convolutions in sequence. The outputs of the second, third, and fourth branches are connected, processed by a 1×1 kernel 2D convolution, and then connected to the output of the first branch.

[0014] Optionally, the Inception-ResNet-B block has three branches: the fifth branch outputs directly, the sixth branch is processed by a 1×1 kernel 2D convolution, and the seventh branch is processed by 1×1 kernel, 1×7 kernel and 7×1 kernel 2D convolution in sequence; the outputs of the sixth branch and the seventh branch are connected, processed by a 1×1 kernel 2D convolution, and then connected to the output of the fifth branch.

[0015] Optionally, the Inception-ResNet-C block has three branches: the eighth branch outputs directly, the ninth branch is processed by a 1×1 kernel 2D convolution, and the tenth branch is processed by 1×1 kernel, 1×3 kernel and 3×1 kernel 2D convolution in sequence; the outputs of the ninth and tenth branches are connected, processed by a 1×1 kernel 2D convolution, and then connected to the output of the first branch.

[0016] A system for automatically extracting coronary arteries based on deep learning, comprising:

[0017] Coronary artery model building module: used to train the ICASEG-Net neural network using ICA images to obtain a model for extracting coronary arteries;

[0018] Probability graph conversion module: This module takes the image to be extracted as input into the coronary artery extraction model and converts the output into a probability graph using the sigmoid function.

[0019] Network weight optimization module: used to optimize network weights by comparing the differences between the probability map and the preset binary artery mask;

[0020] The segmentation result module is used to input the image to be extracted into the trained coronary artery extraction model. The OTSU algorithm is used to convert the output probability map into a binary artery segmentation mask, where a pixel value of 1 represents the coronary artery and a pixel value of 0 represents the background.

[0021] A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the methods for automatically extracting coronary arteries based on deep learning.

[0022] As can be seen from the above technical solution, compared with the prior art, this invention provides a method, system, and storage medium for automatically extracting coronary arteries based on deep learning, used for automatically extracting coronary arteries from the ICA; the model of this invention includes full-scale skip connections and full-scale deep supervision, and incorporates Inception and residual modules into the encoder; the coronary artery extraction model was trained and validated on a dataset, achieving a Dice score of 0.8942, surpassing state-of-the-art deep learning models. By employing transfer learning, the model trained with a hybrid loss function exhibits excellent performance on the coronary artery extraction task; it has great potential for clinical applications. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the process of the present invention;

[0025] Figure 2 This is a structural diagram of the ICASEG-Net of the present invention;

[0026] Figure 3 This is a structural diagram of the Inception A block of the present invention;

[0027] Figure 4 This is a structural diagram of the Inception-ResNet-A block of the present invention;

[0028] Figure 5 This is a structural diagram of the Inception-ResNet-B block of the present invention;

[0029] Figure 6 This is a structural diagram of the Inception-ResNet-C block of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention discloses a method for automatically extracting coronary arteries based on deep learning, such as... Figure 1 As shown, it includes the following steps:

[0032] S1: Use ICA images to train the ICASEG-Net neural network to obtain a model for extracting coronary arteries;

[0033] S2: Input the image to be extracted into the coronary artery extraction model, and convert the output into a probability map using the sigmoid function;

[0034] S3: Optimize network weights by comparing the differences between the probability map and the preset binary artery mask;

[0035] S4: The OTSU algorithm is used to convert the probability map into a binary artery segmentation mask, where a pixel value of 1 represents the coronary artery and a value of 0 represents the background.

[0036] Furthermore, in this embodiment, a retrospective analysis was performed on 210 patients (100 men and 110 women) who underwent ICA (616 images). This HIPAA-compliant retrospective analysis was approved by the Institutional Review Board and did not require informed consent. Each ICA scan yielded 512×512 pixels with a pixel pitch of 0.200–0.390 mm. The ICA images were divided into training and testing sets; the training set was used to train the ICASEG-Net neural network, and the testing set was used to test the ICASEG-Net neural network.

[0037] In S1, ICASEG-Net is as follows: Figure 2 The diagram shows an encoder and a decoder, with low-level feature maps extracted from the encoder path connected to the decoder path.

[0038] The encoder also includes the Inception A block, the Inception-ResNet-A block, the Inception-ResNet-B block, and the Inception-ResNet-C block.

[0039] in, Figure 3 The Inception A block is shown, consisting of four branches. The first branch contains an average ensemble layer, followed by a convolutional layer. The other three branches consist of convolutional layers with different kernel sizes, such as 1×1, 3×3, and 5×5.

[0040] Figure 4The structure of the Inception-ResNet-A block is shown, divided into four branches. The first branch outputs directly; the second branch is processed by a 1×1 kernel 2D convolution; the third branch is processed by 1×1 kernel and 3×3 kernel 2D convolutions; and the fourth branch is processed by a 1×1 kernel 2D convolution and two 3×3 kernel 2D convolutions. The outputs of the second, third, and fourth branches are concatenated, processed by a 1×1 kernel 2D convolution, and then concatenated with the output of the first branch.

[0041] Figure 5 The structure of the Inception-ResNet-B block is shown, divided into three branches. The first branch outputs directly, the second branch is processed by a 1×1 kernel 2D convolution, and the third branch is processed by 1×1 kernel, 1×7 kernel, and 7×1 kernel 2D convolutions. The outputs of the second and third branches are concatenated, then processed by a 1×1 kernel 2D convolution, and finally concatenated with the output of the first branch.

[0042] at last, Figure 6 The Inception-ResNet-C block is structured into three branches. The first branch outputs directly, the second branch is processed by a 1×1 kernel 2D convolution, and the third branch is processed by 1×1, 1×3, and 3×1 kernel 2D convolutions. The outputs of the second and third branches are concatenated, processed by a 1×1 kernel 2D convolution, and then concatenated with the output of the first branch.

[0043] Furthermore, full-scale depth supervision was used to supervise each scale of the decoder. The side outputs of the 5-layer decoder were fed into a regular 3×3 convolutional layer, then upsampled with coefficients of 16, 8, 4, and 2 respectively, and then transformed into a 512×512 probability map by a sigmoid function.

[0044] Furthermore, this invention employs a hybrid loss function to control backpropagation. It includes binary cross-entropy loss (BCE_loss), Dice loss, and an L2 regularization term. Binary cross-entropy is one of the most common evaluation methods in binary tasks and can effectively address the convergence speed issue. Coronary artery extraction is a single-label binary classification task aimed at identifying arterial targets from the ICA. To achieve fast convergence and applicability to binary classification tasks, BCE_loss is utilized. BCE_loss is defined in the following formula:

[0045]

[0046] Where y represents the gold standard with manual annotation. This represents the predicted binary artery tree.

[0047] Artery contours occupy only a tiny fraction of the total pixels in ICA. To mitigate the issue of data sample dissimilarity, a set similarity measure, Dice, is introduced. It is typically used to calculate the similarity between two samples, allocating greater training weights to smaller samples in the dataset. The Dice score (DSC) is calculated using the following formula:

[0048]

[0049] in Represents a set The intersection with y. And |y| represents the number of its elements. The larger the DSC, the smaller the loss. A DSC of 1 indicates that there is no difference between the ground truth and the predicted value, while a DSC of 0 indicates that they are completely dissimilar.

[0050] In coronary artery extraction tasks, many complex samples in the foreground class are difficult to identify correctly, such as finer vessel branches. To address the imbalanced distribution of sample difficulty, it is necessary to make the model focus more on learning complex samples. This can be achieved by reducing the contribution of simple samples to the total loss and increasing the penalty weight for complex samples. Therefore, the total loss defined by the following formula is used as the loss function of the backbone network:

[0051]

[0052] Due to the unique characteristics of ICAs in coronary artery extraction tasks, which suffer from low contrast and high noise, greater emphasis should be placed on multi-scale information. The loss function should be applied to each branch of the deep supervision to control the network training. The branch loss function for deep supervision is defined as:

[0053]

[0054] The hybrid loss function is defined in the following formula, serving as the target loss function:

[0055]

[0056] s is the number of layers in all the lateral outputs of the deep neural network.

[0057] In this invention, five metrics are used to evaluate the accuracy of coronary artery extraction: DSC, sensitivity (SN), specificity (SP), Hausdorff distance (HD), and average surface distance (ASD).

[0058] SN represents the ratio of pixels predicted as positive to all true positive pixels, and SP represents the ratio of pixels predicted as negative to all true negative pixels. The Hausdorff distance represents the similarity between two sets of points. Assume there are two sets C = {a1...an} and D = {b1...bn}. The Hausdorff distance between these two sets is defined as:

[0059] H(C,D)=max(h(C,D),h(D,C))(6);

[0060] Where h(C,D) and h(D,C) are defined as follows:

[0061]

[0062]

[0063] Surface distance is a function that describes the average difference between the predicted result and the actual ground condition. The shortest distance from any pixel v to P(A) is defined as follows, where P(A) represents the set of surface pixels of A.

[0064]

[0065] Where ||.|| represents the following:

[0066]

[0067] This embodiment also discloses a system for automatically extracting coronary arteries based on deep learning, including:

[0068] Coronary artery model building module: used to train the ICASEG-Net neural network using ICA images to obtain a model for extracting coronary arteries;

[0069] Probability graph conversion module: This module takes the image to be extracted as input into the coronary artery extraction model and converts the output into a probability graph using the sigmoid function.

[0070] Network weight optimization module: used to optimize network weights by comparing the differences between the probability map and the preset binary artery mask;

[0071] The segmentation result module is used to input the image to be extracted into the trained coronary artery extraction model. The OTSU algorithm is used to convert the output probability map into a binary artery segmentation mask, where a pixel value of 1 represents the coronary artery and a pixel value of 0 represents the background.

[0072] Finally, this embodiment discloses a computer storage medium storing a computer program, which, when executed by a processor, implements any one of the steps of a method for automatically extracting coronary arteries based on deep learning.

[0073] This embodiment proposes a novel deep learning model for accurate coronary artery extraction. Full-scale skip connections explore more semantic information and capture more subtle details, overcoming the shortcomings of U-Net and U-Net++ in effectively utilizing feature semantics, thus improving segmentation accuracy. Comprehensive deep supervision not only avoids overfitting but also allows the CNN to learn richer arterial information representations, which is beneficial for segmenting arterial details. Furthermore, the Inception module performs feature dimensionality reduction on large matrices and then concatenates visual information of different sizes for feature extraction at different scales. The Residual module accelerates neural network training and improves neural network performance. The Inception and Residual modules in the encoder structure can improve segmentation accuracy by expanding the receptive field and reducing feature loss during convolution. The model of this invention can learn arterial semantic information from aggregated feature maps of different scales, enhancing the extraction of arterial boundary information, thereby improving the performance of coronary artery extraction.

[0074] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0075] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for automatically extracting coronary arteries based on deep learning, characterized in that, Includes the following steps: The ICA image is input into ICASEG-Net, and the output is converted into a probability map by the sigmoid function. The ICASEG-Net neural network includes an encoder and a decoder. The encoder includes convolutional layers, max pooling layers, Inception A blocks, Inception-ResNet-A blocks, Inception-ResNet-B blocks, and Inception-ResNet-C blocks. By comparing the differences between the probability map and the preset binary artery mask, a hybrid loss function is used to optimize the network weights and save the optimal weights to obtain the coronary artery extraction model; the hybrid loss function The formula includes binary cross-entropy loss, Dice loss, and L2 regularization, as follows: ； In the formula, s represents the number of layers in all lateral outputs of the deep neural network; Let be the loss function of the backbone network. For deep supervision, the branch loss function is: ； In the formula, The gold standard for manual annotation; For the predicted binary artery tree; The image to be extracted is input into the trained coronary artery extraction model. The OTSU algorithm is used to convert the output probability map into a binary artery segmentation mask, where a pixel value of 1 represents the coronary artery and a pixel value of 0 represents the background.

2. The method for automatically extracting coronary arteries based on deep learning according to claim 1, characterized in that, The Inception A block comprises four branches; one branch includes a cascaded average layer and a convolutional layer connected in sequence; the other three branches are each composed of convolutional layers of different kernel sizes connected in sequence.

3. The method for automatically extracting coronary arteries based on deep learning according to claim 1, characterized in that, The Inception-ResNet-A block includes four branches. The first branch outputs directly. The second branch is processed by a 2D convolution with a 1×1 kernel. The third branch is processed by 2D convolutions with a 1×1 kernel and a 3×3 kernel in sequence. The fourth branch is processed by a 2D convolution with a 1×1 kernel and two 2D convolutions with a 3×3 kernel in sequence. The outputs of the second, third, and fourth branches are connected, processed by a 2D convolution with a 1×1 kernel, and then connected to the output of the first branch.

4. The method for automatically extracting coronary arteries based on deep learning according to claim 1, characterized in that, The Inception-ResNet-B block has three branches: the fifth branch outputs directly, the sixth branch is processed by a 1×1 kernel 2D convolution, and the seventh branch is processed by 1×1 kernel, 1×7 kernel and 7×1 kernel 2D convolution in sequence; the outputs of the sixth branch and the seventh branch are connected, processed by a 1×1 kernel 2D convolution, and then connected to the output of the fifth branch.

5. The method for automatically extracting coronary arteries based on deep learning according to claim 1, characterized in that, The Inception-ResNet-C block has three branches: the eighth branch outputs directly, the ninth branch is processed by a 1×1 kernel 2D convolution, and the tenth branch is processed by 1×1 kernel, 1×3 kernel and 3×1 kernel 2D convolution in sequence; the outputs of the ninth and tenth branches are connected, processed by a 1×1 kernel 2D convolution, and then connected to the output of the first branch.

6. A system for automatically extracting coronary arteries based on deep learning, used to implement the method for automatically extracting coronary arteries based on deep learning as described in any one of claims 1-5, characterized in that, include: Coronary artery model building module: used to train the ICASEG-Net neural network using ICA images to obtain a coronary artery extraction model; Probability graph conversion module: This module takes the image to be extracted and inputs it into the coronary artery extraction model. The output is then converted into a probability graph using the sigmoid function. Network weight optimization module: used to optimize network weights by comparing the differences between the probability map and the preset binary artery mask; The segmentation result module is used to input the image to be extracted into the trained coronary artery extraction model. The OTSU algorithm is used to convert the output probability map into a binary artery segmentation mask, where a pixel value of 1 represents the coronary artery and a pixel value of 0 represents the background.

7. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for automatically extracting coronary arteries based on deep learning as described in any one of claims 1-5.

Images