Deep learning method for coronary vessel segmentation with focus on edge and topological features

Abstract

The present application relates to a kind of DSA coronary vessel segmentation deep learning method of edge and topological feature, comprising the following steps: obtaining DSA coronary angiography image dataset and corresponding coronary vessel label dataset;Data is input into deep learning network, the deep learning network is end-to-end U type deep learning network, include improved bidirectional Transformer module and multi-scale semantic fusion module, and encoder and decoder are connected by jump connection and multi-scale semantic fusion module;Coronary vessel segmentation model is trained in three stages using the composite loss function comprising Dice loss, Focal loss, edge loss and topological loss, the segmentation model of the training completion can be carried out complete segmentation to the coronary vessel in DSA coronary angiography picture;Using composite evaluation index, the overall segmentation accuracy of segmentation model, edge segmentation accuracy and the ability of keeping topological consistency are comprehensively evaluated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image segmentation, in particular to a DSA coronary angiography blood vessel segmentation deep learning method, which focuses on the network structure of the model, the training method and the quantitative evaluation of the segmentation effect of the model, and proposes an innovative design. BACKGROUND

[0002] Coronary heart disease belongs to a kind of cardiovascular disease, and the coronary arteries around the heart of patients with coronary heart disease are narrowed, blocked or thrombosed due to atherosclerosis, so that the heart cannot be supplied with sufficient blood and oxygen in time, and the disability rate and mortality rate of coronary heart disease are extremely high. Coronary angiography (CAG) realized by digital subtraction angiography (DSA) technology is the only effective "gold standard" for the diagnosis of coronary heart disease in clinical practice, has the advantages of clear image and high resolution, can provide clear images of the coronary arteries of patients from various angles for doctors, and provides convenience for observing the coronary atherosclerotic lesion and the degree of stenosis, is an important image basis for the diagnosis and treatment of coronary heart disease, and provides effective guidance for surgical treatment.

[0003] The traditional method of diagnosing coronary heart disease requires doctors to have rich clinical experience, is low in efficiency, and has great subjectivity. Computer-aided diagnosis can improve the efficiency and scientificity of diagnosis. And blood vessel segmentation is the first step to realize computer-aided diagnosis. At present, the medical image segmentation model based on U-shaped network greatly improves the accuracy of segmentation. CN202211515553.1 invented a coronary blood vessel segmentation model including Unet network, multi-scale feature splicing module and attention mechanism module, which fuses feature information of different scales and improves the accuracy of coronary artery segmentation. However, the morphological structure of the blood vessels in coronary angiography is special, the blood vessels are very thin and complex, and the diseased blood vessels are narrowed, so the existing segmentation method still has the limitations of a large number of over-segmentation and under-segmentation, especially it is difficult to maintain the continuity and edge accuracy of coronary blood vessel segmentation. The diagnosis of coronary heart disease is performed by directly observing the blood vessel morphology in the coronary angiography image, so it is necessary to maintain the connectivity and accurate blood vessel edge of the segmented blood vessels for accurate diagnosis. SUMMARY

[0004] In view of the above problems, the purpose of the present application is to provide a DSA coronary blood vessel segmentation deep learning method focusing on edge and topological features. The present application is realized by the following technical scheme,

[0005] A DSA coronary blood vessel segmentation deep learning method focusing on edge and topological features, comprising the following steps:

[0006] Step one: Obtain the DSA coronary angiography image dataset and the corresponding coronary vessel label dataset, which are used as input for the deep learning network. The DSA coronary angiography image dataset contains multiple body position coronary angiography images.

[0007] Step two: Input the data into the deep learning network, which is an end-to-end U-shaped deep learning network containing an improved bidirectional Transformer module and a multi-scale semantic fusion module. Feature encoding and decoding are performed using the deep learning network containing the improved bidirectional Transformer module to generate feature representations at each level and concise semantic maps at each level. The feature encoding part is called the encoder, and the feature decoding part is called the decoder. The encoder and decoder are connected through a skip connection and a multi-scale semantic fusion module.

[0008] Step three: Train the coronary vessel segmentation model using a composite loss function containing Dice loss, Focal loss, edge loss, and topological loss in three stages. The segmentation model is trained in three stages to achieve the best convergence state while avoiding overfitting. The trained segmentation model can completely segment the coronary vessels in the DSA coronary angiography images. The constructed segmented composite loss function is as follows:

[0009]

[0010] where L d is the Dice loss, L f is the Focal loss, L b is the edge loss, L t is the topological loss, a is the weight of the topological loss, and x1 and x2 are the set boundary points.

[0011] Step four: Use the composite evaluation index to comprehensively evaluate the overall segmentation accuracy, edge segmentation accuracy, and ability to preserve topological consistency of the segmentation model.

[0012] 2. Further, the encoder is divided into four layers to encode the input data:

[0013] ① The first layer contains two 3x3 convolution layers and four downsampling operations, which model the local semantic relationship of the input image and reduce the resolution of the input image by 4x. The first-level feature representation is obtained, and a concise semantic map is generated through a semantic map generation module mainly composed of 3x3 convolution, which is used to reduce the computational complexity when calculating the cross-attention.

[0014] ② The second layer consists of an improved bidirectional Transformer module. The input is the first-level feature representation and the first-level concise semantic map, and the output is the second-level feature representation and the second-level concise semantic map.

[0015] ③The third layer and the fourth layer are both composed of one batch fusion block for completing the downsampling operation and two improved bidirectional Transformer modules, respectively generating the third and fourth level feature representations and the third and fourth level concise semantic maps.

[0016] Further, the decoder is divided into four layers to decode the features extracted by the encoder, and the last layer outputs the segmentation result:

[0017] ①The first layer contains one upsampling block to perform the first feature decoding on the fourth level feature representation and the fourth level concise semantic map generated by the encoder;

[0018] ②The second layer contains two improved bidirectional Transformer modules and one upsampling block to perform the second feature decoding on the feature decoding result of the first layer; the third layer contains one improved bidirectional Transformer module to perform the third feature decoding;

[0019] ③The fourth layer contains four upsampling operations, and reduces the channel number to 2 through a 1x1 convolution to output the final segmentation result.

[0020] Further, the encoder and the decoder of the U-shaped deep learning network are connected through a skip connection and a multi-scale semantic fusion module to reduce the loss of feature information, including:

[0021] ①Skip connection: the feature representations of each level of the encoder are connected with the results of the corresponding feature decoding of each level of the decoder through a skip connection, and the connection method is to add the pixel values of the corresponding pixel points;

[0022] ②Multi-scale semantic fusion module: the second, third and fourth level concise semantic maps of the encoder are flattened and connected into a sequence, then the one-dimensional sequence is fed into a standard Transformer module composed of multi-head self-attention mechanism (MHSA) and feedforward network (FFN) for multi-scale semantic fusion, and then reshaped into three semantic maps, which are fed into the corresponding layers of the decoder. Figure One

[0023] Further, the bidirectional Transformer modules in each layer of the encoder and the decoder are used to calculate the cross attention to promote the feature fusion of the feature representations and the concise semantic maps, and to filter out background noise, including:

[0024] ①The feature representations output by each layer of the encoder and the decoder are downsampled through a batch fusion block and then mapped to different spaces using a 1x1 convolution operation, and converted into a one-dimensional sequence through a flatten operation, i.e. Figure One

[0025] ​​Q / K / V = flatten(Conv(X, k))

[0026] Q' / K' / V' = flatten(Conv(M, k))

[0027] where X represents the feature representation of each level, M represents the two-dimensional concise semantic graph of each level, Conv represents convolution, k represents the size of the convolution kernel, Q(Q'), K(K'), V(V') represent query, key and value space respectively;

[0028] ②Calculate cross attention: use dot product to calculate the similarity score between query and key and normalize to get attention weight, then multiply the attention weight with the value and sum to get the weighted value X' and M' as the output of the bidirectional Transformer module:

[0029]

[0030]

[0031] where Q'K T =(QK' T ) T .

[0032] Further, the specific formulas of Dice loss, Focal loss, edge loss and topological loss are as follows:

[0033] ①

[0034] where G represents the blood vessel label image, and S represents the model segmentation result image.

[0035] ②L f =-(1-p t ) γ log(p t ), where

[0036] P t is the prediction probability function of the model, the value of p is between 0 and 1, which is the probability of the model predicting as blood vessels, and γ is the modulation factor

[0037] ③L b =∫ Ω φ G (q)s θ (q)dq, where

[0038] where q∈S represents that point q is segmented into a blood vessel region, represents that any point q in the blood vessel contour region of the segmentation result image is the nearest point to the blood vessel contour region in the label image distance between the two points, s θ (q) is a softmax probability output of the network model;

[0039] ④

[0040] where Dgm(f) is the persistence homology diagram of the segmentation result map, t is an arbitrary point on Dgm(f), b(t), d(t) represent the time of the appearance and disappearance of an arbitrary topological structure t in the persistence homology diagram respectively, and γ * is the best match; the persistence homology diagrams Dgm(f) and Dgm(g) of the segmentation result map and the blood vessel label map are calculated respectively, and the best match γ * of all points in Dgm(f) in Dgm(g) is found; the matching rule is that k corner points are determined in Dgm(g), then k points closest to the corner points in Dgm(f) are found and matched to the corresponding points in Dgm(g), and the remaining points in Dgm(f) are all matched to the diagonal line; the sum of the squared distances of all points in Dgm(f) to the best match is calculated to obtain L t .

[0041] Further, α = 0.1, and the modulation factor γ = 2.

[0042] Further, in step three, the method for training the segmentation model in three stages is as follows: using five-fold cross-validation training, setting the initial learning rate, training n epochs, saving the best model with the minimum validation set loss after training, setting x1 and x2, and the method is as follows:

[0043] ① The first stage: using L = L d + L f as the loss function for network training, so that the network reaches a preliminary convergence state;

[0044] ② The second stage: using L = L d + L f + L b as the loss function for network training, so that the network reaches a further convergence state by preserving more edge features during training;

[0045] ③ The third stage: using L = L d + L f + L b + αL t as the loss function for network training, so that the network reaches the final convergence by focusing on topological consistency during training.

[0046] Further, step four: using the composite evaluation index Comprehensive evaluation of the overall segmentation accuracy of the segmentation model, the edge segmentation accuracy and the ability to retain topological consistency, including:

[0047] 1) The composite evaluation index VSF is the weighted sum of the four basic index items Dice, IoU, BIoU and Betti error; wherein, ε, φ, and γ are the weights of the four basic index items,

[0048] 2) Dice and IoU measure the overall similarity of the segmentation result and the label image, quantitatively evaluate the pixel-by-pixel accuracy of the coronary artery segmentation model, the larger the Dice and IoU, the higher the overall segmentation accuracy of the coronary artery segmentation model, the calculation method is:

[0049] ① Calculate the confusion matrix of the segmentation result: TP when both the prediction result and the label value are blood vessels, TN when both the prediction result and the label value are background, FP when the prediction result is blood vessels and the label value is background, and FN when the prediction result is background and the label value is blood vessels.

[0050] ②

[0051] ③

[0052] 3) BIoU measures the similarity of the segmentation result and the label image from the perspective of the edge of the coronary artery, quantitatively evaluates the edge segmentation accuracy of the coronary artery segmentation model, the larger the BIoU, the higher the edge segmentation accuracy of the segmentation model, the calculation method is:

[0053] ① The prediction result S and the label image G are respectively ablated d pixels from the blood vessel edge to obtain S d and G d , and the boundary region is obtained by subtracting the original image from the ablation image, and the representation of BIoU is obtained by calculating the intersection over union of the boundary region of the prediction result and the label image.

[0054] ②

[0055] 4) Betti error is the inverse of the error of the Betti number of the segmentation result and the label image, which measures the similarity of the segmentation result and the label image from the perspective of topological structure, quantitatively evaluates the ability of the coronary artery segmentation model to retain topological consistency, the larger the Betti error, the stronger the ability of the model to retain topological consistency, the calculation method is:

[0056] ① Calculate the Betti number of the segmentation result and the blood vessel label image:

[0057]

[0058] wherein, Bn denotes the Betti number of n-dimensional simplices, denotes the boundary map in topology, which can map an n-dimensional simplex to an n-1-dimensional simplex; denotes the kernel of the boundary map, that is, the set of n-dimensional simplices mapped to zero vectors; denotes the image of the boundary map, that is, the set of n+1-dimensional simplices mapped to n-dimensional simplices; dim denotes the dimension of the vector space;

[0059] ②Calculate B1 and B2 of the segmentation result image and the blood vessel label image, and take the sum as the Betti number of the segmentation result and the blood vessel label, wherein B1 represents the number of connected components, B2 represents the number of holes, and the calculation formula of Betti error is:

[0060]

[0061] wherein B S denotes the Betti number of the segmentation result, B G denotes the Betti number of the label image;

[0062] The larger the composite evaluation index VSF is, the better the coronary vessel segmentation model performs in terms of overall segmentation accuracy, edge segmentation accuracy and topological consistency preservation, and the fewer the island problems and fracture phenomena in the segmentation result.

[0063] The application also provides a DSA coronary vessel segmentation device for implementing the above method steps and a computer readable storage medium storing program instructions.

[0064] The application has the advantages that: based on a deep learning network structure comprising an improved bidirectional Transformer module and a multi-scale semantic fusion module, the application innovatively proposes a novel composite loss function and a three-stage training method focusing on blood vessel edges and topological features, effectively solves the problems of a large number of over-segmentation and under-segmentation of coronary vessels, and improves the edge segmentation accuracy of blood vessels and the topological continuity of blood vessels in the DSA coronary vessel segmentation result, while providing a model segmentation effect quantitative evaluation method for a DSA coronary vessel segmentation model. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 FIG. 1 is a flowchart of a DSA coronary vessel segmentation deep learning method focusing on edge and topological features in an embodiment of the application.

[0066] Figure 2 FIG. 2 is a schematic block diagram of a segmentation model network structure in an embodiment of the application.

[0067] Figure 3The composite loss function and the three-stage training schematic diagram used for an example of the present application.

[0068] Figure 4 The schematic block diagram of the improved bidirectional Transformer module in an example of the present application.

[0069] Figure 5 The schematic block diagram of the multi-scale semantic fusion module in an example of the present application.

[0070] Figure 6 The comparison diagram of the partial segmentation result obtained in an example of the present application and the segmentation result of other advanced models. DETAILED DESCRIPTION

[0071] The present application is further illustrated in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present application and are not used to limit the scope of the present application. In addition, it should be understood that after reading the content taught by the present application, those skilled in the art can make various modifications or changes to the present application, and these equivalent forms also fall within the scope defined by the present application.

[0072] The present application innovatively designs a deep learning network structure containing an improved bidirectional Transformer module and a multi-scale semantic fusion module, simultaneously designs a new composite loss function and a three-stage training method based on the topological features of the blood vessel edge features and the blood vessel continuity, and further innovatively proposes a coronary artery segmentation model evaluation method which comprehensively evaluates the overall segmentation accuracy, the edge segmentation accuracy and the ability to retain topological consistency.

[0073] Due to the existence of bone, guide wire and other artifacts in the background of DSA coronary angiography images, and the complex structure of coronary blood vessels and the existence of stenosis in diseased blood vessels, the existing deep learning medical image segmentation methods often have low accuracy when segmenting DSA coronary angiography blood vessels, and have little reference significance for clinical auxiliary diagnosis. The deep learning image segmentation method combining CNN and Transformer uses CNN to extract local features in a hierarchical manner, and uses Transformer to enhance the extraction of long-distance features, which has achieved excellent performance in many studies. However, due to the very thin coronary blood vessels and the existence of stenosis in diseased blood vessels, there are often many discontinuity problems and island problems in the segmentation results, which is fatal for clinical diagnosis of coronary heart disease. Therefore, the present application proposes a DSA coronary blood vessel segmentation deep learning method which focuses on edge and topological features.

[0074] The present application is realized by the following technical solutions, and the flowchart is as shown in Figure 1 The specific steps of the present application are as follows:

[0075] Step one: obtain the DSA coronary angiography image dataset and the corresponding coronary vessel label dataset for input into the deep learning network, the DSA coronary angiography image dataset contains multiple body position coronary angiography images, the specific steps are as follows:

[0076] 1) Obtain DSA coronary angiography video data from the hospital, and convert it into two-dimensional images. The converted two-dimensional DSA coronary angiography image is a 512x512 size grayscale image.

[0077] 2) With the help of the chief physician of the cardiology department, obtain the vessel label by manual labeling.

[0078] Step two: input the data into the deep learning network, the batch_size size is 16, the deep learning network is an end-to-end U-shaped deep learning network, which contains an improved bidirectional Transformer module and a multi-scale semantic fusion module, as shown in Figure 2 , specifically including:

[0079] Use the deep learning network containing the improved bidirectional Transformer module for feature encoding and feature decoding to generate feature representations at each level and concise semantic maps at each level. The feature encoding part is called the encoder, and the feature decoding part is called the decoder. The encoder and the decoder are connected through the skip connection and the multi-scale semantic fusion module, thereby reducing the loss of feature information.

[0080] 1) The encoder is divided into four layers to encode the input data:

[0081] (4) The first layer contains two 3x3 convolution layers and four downsampling operations, which model the local semantic relationship of the input image, reduce the resolution of the input image by 4x, and obtain the first level feature representation. At the same time, a semantic map generation module mainly composed of 3x3 convolution is used to map the first level feature representation into a concise semantic map, which is used to reduce the computational complexity when calculating the cross attention. In an example of the present application, the size of the concise semantic map is set to 32x32.

[0082] (5) The second layer is composed of an improved bidirectional Transformer module, the input is the first level feature representation and the first level concise semantic map, and the output is the second level feature representation and the second level concise semantic map.

[0083] (6) The third layer and the fourth layer are both composed of a batch fusion block that completes the downsampling operation and two improved bidirectional Transformer modules, which respectively generate the third and fourth level feature representations and the third and fourth level concise semantic maps.

[0084] 2) The decoder is divided into four layers to decode the features extracted by the encoder, and the last layer outputs the segmentation result:

[0085] (4) The first layer comprises an upsampling block to perform the first feature decoding on the fourth-level feature representation and the fourth-level concise semantic map generated by the encoder;

[0086] 5) The second layer comprises two improved bidirectional Transformer modules and an upsampling block to perform the second feature decoding on the feature decoding result of the first layer; the third layer comprises an improved bidirectional Transformer module to perform the third feature decoding;

[0087] (6) The fourth layer comprises four upsampling operations and reduces the channel number to 2 through a 1x1 convolution to output the final segmentation result.

[0088] 3) Unlike the conventional U-shaped segmentation model and the DenseNet cross-layer connection model, the U-shaped deep learning network of the present application connects the features between the encoder and the decoder through a skip connection and a multi-scale semantic fusion module, reduces the loss of feature information, and specifically comprises:

[0089] 3) The skip connection: the feature representation of each level of the encoder is connected with the result of the feature decoding of the corresponding decoder, and the connection mode is to add the pixel values of the corresponding pixel points;

[0090] 4) The multi-scale semantic fusion module: as shown in Figure 5 , the second, third and fourth-level concise semantic Figure One maps of the encoder are converted into a sequence and then fed into a standard Transformer module composed of a multi-head self-attention mechanism (MHSA) and a feedforward network (FFN) for multi-scale semantic fusion, and then reshaped into three semantic maps, which are fed into the corresponding layers of the decoder.

[0091] 4) The bidirectional Transformer module in each layer of the above-mentioned encoder and decoder is used to calculate the cross-attention, promote the feature fusion of the feature representation and the concise semantic map, filter out the background noise, and make the model pay attention to more useful feature information, as shown in Figure 4 , and specifically comprises:

[0092] 3) The feature representation output by each layer of the encoder and the decoder is completed after down-sampling by a batch fusion block and the concise semantic Figure One is used to map to different spaces through a 1x1 convolution operation, and is converted into a one-dimensional sequence through a flatten operation, that is:

[0093] Q / K / V = flatten(Conv(X, k))

[0094] Q' / K' / V' = flatten(Conv(M, k))

[0095] Wherein, X represents each level feature representation, M represents each level two-dimensional concise semantic graph, Conv represents convolution, k represents the size of convolution kernel, Q(Q'), K(K'), V(V') represent query, key and value space respectively;

[0096] (4) Then calculate the cross attention, the specific steps are to use dot product to calculate the similarity score between query and key and normalize to get attention weight, then multiply the attention weight with value and sum to get weighted value X' and M', as the output of the bidirectional Transformer module:

[0097]

[0098]

[0099] Wherein, Q'K T =(QK' T ) T

[0100] Step three: train the coronary artery segmentation model in three stages using a composite loss function containing Dice loss, focal loss, edge loss and topology loss, and the completed segmentation model can completely segment the coronary artery in the DSA coronary angiography picture, as shown in Figure 3 The specific steps are as follows:

[0101] 1) Construct a segmented composite loss function, use conventional Dice and focal loss to solve the problem of uneven distribution of blood vessels and background pixels in DSA coronary angiography pictures, use edge loss to make the model pay attention to the edge features of blood vessels and improve the edge segmentation accuracy of the model, and use topology loss to make the model pay attention to the continuity and other topological features of blood vessels and improve the topological consistency of the segmentation result and the blood vessel label, the segmented loss function is as follows:

[0102]

[0103] Wherein, L d is the Dice loss, L f is the focal loss, L b is the edge loss, L t is the topology loss, and alpha is the weight of the topology loss, in an example of the present application, alpha = 0.1. The specific formulas of Dice loss, focal loss, edge loss and topology loss are as follows:

[0104]

[0105] Wherein, G represents the blood vessel label image, and S represents the model segmentation result image.

[0106] f ​​= -(1 - p t ) γ log(p t ), where

[0107] P t is the predicted probability function of the model, and p is the value between 0 and 1, which is the probability predicted by the model as a blood vessel. In an example of the present application, the value of the modulation factor γ in the focal loss is 2.

[0108] ⑦L b = ∫ Ω φ G (q) s θ (q) dq, where

[0109] where q ∈ S represents that the point q is segmented into a blood vessel region, represents the distance between any point q in the blood vessel contour region in the segmentation result map and the nearest point s θ (q) is the softmax probability output of the network model;

[0110] ⑧

[0111] where Dgm(f) is the persistent homology graph of the segmentation result map, t is any point on Dgm(f), b(t), d(t) respectively represent the time when any topological structure t appears or disappears in the persistent homology graph, and γ * is the best match. In an example of the present application, in order to calculate the topological loss, first, the persistent homology graphs Dgm(f) and Dgm(g) of the segmentation result map and the blood vessel label map are calculated respectively, and the best match γ * of all points in Dgm(f) in Dgm(g) is found. The matching rule is: assuming that there are k points in the upper left corner in Dgm(g), find the k points closest to the corner points in Dgm(f), and match them to the corresponding points in Dgm(g), and match the remaining points in Dgm(f) to the diagonal line. Then calculate the sum of the squared distances of all points in Dgm(f) to the best match.

[0112] 2) Train the segmentation model in three stages to make the model achieve the best convergence state while avoiding overfitting, an example of the present application uses five-fold cross-validation training, the initial learning rate is set to 5e-4, and the model is trained for 50 epochs. After training, the best model with the minimum validation set loss is saved. Referring to Figure 2 , in an example of the present application, x1 = 15 and x2 = 30, and the specific steps are:

[0113] (4) The first stage: the first 15 epochs use L = L d + L f The network is trained as a loss function to achieve a preliminary convergence state of the network.

[0114] (5) The second stage: the 16th to 30th epochs use L = L d + L f + L b The network is trained as a loss function to retain more edge features during the training process, so as to achieve a further convergence state of the network.

[0115] 6) The third stage: the 31st to 50th epochs use L = L d + L f + L b + aL t The network is trained as a loss function to focus on topological consistency during the training process, so as to achieve the final convergence of the network.

[0116] Step four: use the composite evaluation index The overall segmentation accuracy, edge segmentation accuracy and ability to retain topological consistency of the segmentation model are comprehensively evaluated, specifically including:

[0117] 5) The composite evaluation index VSF is the weighted sum of the four basic index items Dice, IoU, BIoU and Betti error; wherein, ε, φ, and γ are the weights of the four basic index items,

[0118] 6) Dice and IoU measure the overall similarity between the segmentation result and the label image, and quantitatively evaluate the pixel-by-pixel accuracy of the coronary vessel segmentation model. The larger the Dice and IoU, the higher the overall segmentation accuracy of the coronary vessel segmentation model, specifically including:

[0119] 1) In order to calculate Dice and IoU, first calculate the confusion matrix of the segmentation result: TP (both the prediction result and the label value are blood vessels), TN (both the prediction result and the label value are background), FP (the prediction result is blood vessels and the label value is background), and FN (the prediction result is background and the label value is blood vessels).

[0120] 2) The Dice coefficient is calculated as follows:

[0121] 3) The IoU coefficient is calculated as follows:

[0122] 7) BIoU measures the similarity between the segmentation result and the label image from the perspective of the edge of the coronary vessel, and quantitatively evaluates the edge segmentation accuracy of the coronary vessel segmentation model. The larger the BIoU, the higher the edge segmentation accuracy of the segmentation model, specifically including:

[0123] ①In order to calculate BIoU, the prediction result image S and the label image G are respectively ablated from the blood vessel edge to the inside by d pixels to obtain S d and G d , and the boundary region is obtained by subtracting the ablation image from the original image, and the representation of BIoU is obtained by calculating the intersection over union of the boundary region of the prediction result image and the label image, and the value d in an example of the application is 2.

[0124] ②

[0125] 8) Betti error is the inverse of the error of the Betti number of the segmentation result and the label image, which measures the similarity of the segmentation result and the label image from the perspective of topological structure, and quantitatively evaluates the ability of the coronary artery segmentation model to retain topological consistency. The greater the Betti error, the stronger the ability of the model to retain topological consistency, and specifically includes:

[0126] ①In order to calculate Betti error, the Betti number of the segmentation result image and the blood vessel label image is calculated first:

[0127]

[0128] Wherein, B n represents the n-dimensional Betti number, represents the boundary mapping of an n-dimensional simplex in topology, which can map an n-dimensional simplex to an n-1-dimensional simplex; represents the kernel of the boundary mapping, that is, the set of n-dimensional simplexes mapped to zero vectors; represents the image of the boundary mapping, that is, the set of n+1-dimensional simplexes mapped to n-dimensional simplexes; dim represents the dimension of the vector space;

[0129] ②An example of the application calculates B1 and B2 of the segmentation result image and the blood vessel label image, and takes the sum as the Betti number of the segmentation result and the blood vessel label, wherein B1 represents the number of connected components, and B2 represents the number of holes. The calculation formula of Betti error is:

[0130]

[0131] Wherein, B S represents the Betti number of the segmentation result, and B G represents the Betti number of the label image;

[0132] 9) In summary, the greater the composite evaluation index VSF, the better the performance of the coronary artery segmentation model in terms of overall segmentation accuracy, edge segmentation accuracy and retention of topological consistency, and the fewer the isolated island problems and fracture phenomena in the segmentation result.

[0133] The test experiment of the method of the application is as follows:

[0134] The segmentation model trained by the DSA coronary vessel segmentation deep learning method focusing on edge and topological features is named UT-BTNet. Based on the test of more than 100 clinical data, UT-BTNet is compared with advanced UNet, SwinUnet and SegNeXt models, and the composite evaluation index VSF is used. The comparison results are as follows:

[0135]

[0136]

[0137] From the comparison results, it can be seen that the segmentation model trained by the DSA coronary vessel segmentation deep learning method focusing on edge and topological features provided by the application has the optimal performance in overall segmentation accuracy (Dice and IoU scores), edge segmentation accuracy (BIoU score) and topological consistency (Betti error score) compared with UNet, SwinUnet and SegNext models. See Figure 6 In most cases, the application of the technical solution of the application effectively solves the problems of excessive segmentation and under-segmentation of coronary vessels, and the vascular edge details and topological continuity are well guaranteed.

[0138] Meanwhile, the DSA coronary vessel segmentation deep learning method focusing on edge and topological features provided by the application uses the composite evaluation index VSF score result to comprehensively evaluate the overall segmentation accuracy, edge segmentation accuracy and topological consistency of the model. From the comparison results, the segmentation model trained by the DSA coronary vessel segmentation deep learning method focusing on edge and topological features provided by the application has the highest VSF score and the best comprehensive segmentation performance, and the PSPNet has the lowest VSF score and the worst comprehensive segmentation performance.

Claims

1. A deep learning method for DSA coronary vessel segmentation focusing on edge and topological features, comprising the following steps: Step 1: Obtain the DSA coronary angiography image dataset and the corresponding coronary artery label dataset for input into the deep learning network. The DSA coronary angiography image dataset contains coronary angiography images from multiple body positions. Step 2: Input the data into the deep learning network. The deep learning network is an end-to-end U-shaped deep learning network, which includes an improved bidirectional Transformer module and a multi-scale semantic fusion module. The deep learning network with the improved bidirectional Transformer module is used to perform feature encoding and feature decoding to generate feature representations at each level and concise semantic maps at each level. The feature encoding part is called the encoder, and the feature decoding part is called the decoder. The encoder and decoder are connected through skip connections and the multi-scale semantic fusion module. Step 3: Train the coronary artery segmentation model in three stages using a composite loss function that includes Dice loss, Focal loss, edge loss, and topology loss. This three-stage training process helps the model achieve optimal convergence while avoiding overfitting. The trained segmentation model can completely segment the coronary arteries in DSA coronary angiography images. The constructed piecewise composite loss function is as follows: Among them, L d For Dice's loss, L f For Focal loss, L b For edge loss, L t Let α be the topology loss, α be the weight of the topology loss, and x1 and x2 be the set boundary points. Step 4: Use composite evaluation metrics to comprehensively evaluate the overall segmentation accuracy, edge segmentation accuracy, and ability to preserve topological consistency of the segmentation model; The encoder consists of four layers for feature encoding of the input data: ① The first layer contains two 3×3 convolutional layers and four downsampling operations to model the local semantic relationships of the input image, reducing the resolution of the input image by 4× to obtain the first-level feature representation. At the same time, the first-level feature representation is mapped into a concise semantic graph through a semantic graph generation module mainly composed of 3×3 convolutions, which is used to reduce the computational complexity when calculating cross attention. ② The second layer consists of an improved bidirectional Transformer module, which takes the first-level feature representation and the first-level concise semantic map as input and outputs the second-level feature representation and the second-level concise semantic map. ③ The third and fourth layers each consist of a batch fusion block that performs downsampling and two improved bidirectional Transformer modules, which generate third and fourth level feature representations and third and fourth level concise semantic graphs, respectively; The encoder and decoder of the U-shaped deep learning network are connected via skip connections and a multi-scale semantic fusion module to reduce the loss of feature information, including: ① Skip connection: The feature representations of each level of the encoder are connected to the results of the corresponding decoder's feature representations at each level through skip connections. The connection method is to add the pixel values ​​of the corresponding pixels. ② Multi-scale semantic fusion module: The second, third and fourth level concise semantic graphs of the encoder are one-dimensionalized and connected into a sequence. Then, the one-dimensional sequence is fed into a standard Transformer module composed of a multi-head self-attention mechanism (MHSA) and a feedforward network (FFN) for multi-scale semantic fusion. After that, it is reshaped into three semantic graphs and fed into the corresponding layers of the decoder. The bidirectional Transformer modules in each layer of the encoder and decoder are used to compute cross-attention, facilitate feature fusion of feature representations and concise semantic graphs, and filter out background noise, including: ① The feature representations output from each layer of the encoder and decoder are downsampled using batch fusion blocks and then mapped to different spaces using a 1×1 convolution operation with a concise semantic map. Finally, a flatten operation is used to transform them into a one-dimensional sequence, i.e.: Q / K / V = flatten(Conv(X,k)) Q' / K' / V' = flatten(Conv(M,k)) Where X represents the feature representation at each level, M represents the two-dimensional concise semantic graph at each level, Conv represents convolution, k represents the size of the convolution kernel, and Q(Q'), K(K'), and V(V') represent the query, key, and value space, respectively. ② Calculate cross-attention: Calculate the similarity score between the query and the key using the dot product and normalize it to obtain the attention weights. Then, multiply the attention weights by the values ​​and sum them to obtain the weighted values ​​X' and M', which are used as the output of the bidirectional Transformer module. Among them, Q'K T =(QK' T ) T .

2. The deep learning method for DSA coronary artery segmentation according to claim 1, characterized in that, The decoder consists of four layers that perform feature decoding on the features extracted by the encoder, with the last layer outputting the segmentation result: ① The first layer contains an upsampling block, which performs the first feature decoding on the fourth-level feature representation and the fourth-level concise semantic map generated by the encoder; ② The second layer contains two improved bidirectional Transformer modules and an upsampling block to perform a second feature decoding on the feature decoding results of the first layer; the third layer contains an improved bidirectional Transformer module to perform a third feature decoding. ③ The fourth layer contains four upsampling operations and reduces the number of channels to 2 through a 1×1 convolution, outputting the final segmentation result.

3. The deep learning method for DSA coronary artery segmentation according to claim 1, characterized in that, The specific formulas for Dice loss, Focal loss, edge loss, and topology loss are as follows: ① Where G represents the blood vessel label image and S represents the model segmentation result image; ②L f =-(1-p t ) γ log(p t ),in P t This is the model's prediction probability function, where p is between 0 and 1, representing the probability that the model predicts a blood vessel, and γ is the modulation factor. ③L b =∫ Ω φ G (q)s θ (q)dq, where Where q∈S indicates that point q is segmented into a vascular region. This represents any point q in the blood vessel contour region of the segmentation result image and the nearest point in the blood vessel contour region of the label image. The distance between them, s θ (q) is the softmax probability output of the network model; ④ Where Dgm(f) is the persistent homology graph of the segmentation result, t is any point on Dgm(f), b(t) and d(t) represent the time when any topological structure t appears or disappears in the persistent homology graph, respectively, and γ * The best match is found by calculating the continuous coherence maps Dgm(f) and Dgm(g) of the segmentation result map and the vessel label map, respectively, and finding the best match γ in Dgm(g) for all points in Dgm(f). * The matching rule is as follows: If k corner points are determined in Dgm(g), then find the k points in Dgm(f) closest to the corner points and match them to their corresponding points in Dgm(g). Match the remaining points in Dgm(f) to the diagonal. Calculate the sum of the squared distances from all points in Dgm(f) to the best match to obtain L. t .

4. The deep learning method for DSA coronary artery segmentation according to claim 3, characterized in that, α = 0.1, modulation factor γ = 2.

5. The deep learning method for DSA coronary artery segmentation according to claim 3, characterized in that, In step three, the method for training the segmentation model in three stages is as follows: Five-fold cross-validation is used for training. An initial learning rate is set, and the model is trained for n epochs. After training, the best model with the minimum loss on the validation set is saved. x1 and x2 are set as follows: ① First stage: The first x1 epochs use L=L d +L f The loss function is used to train the network, enabling the network to reach a preliminary convergence state. ② Second stage: For epochs greater than x1 and less than x2, use L=L d +L f +L b Using it as a loss function for network training preserves more edge features during training, enabling the network to reach a more convergent state. ③ Third stage: The rest use L=L d +L f +L b +αL t The loss function is used to train the network, and topological consistency is considered during the training process to enable the network to eventually converge.

6. The deep learning method for DSA coronary artery segmentation according to claim 1, characterized in that, In step four, composite evaluation indicators are used. include: 1) The composite evaluation index VSF is a weighted sum of four basic indicators: Dice, IoU, BIoU, and Betti error; where ε, φ, ... γ and γ are the weights of the four basic indicators, respectively. 2) Dice and IoU measure the overall similarity between the segmentation result and the labeled image, quantitatively evaluating the pixel-by-pixel accuracy of the coronary artery segmentation model. The larger the Dice and IoU, the higher the overall segmentation accuracy of the coronary artery segmentation model. The calculation method is as follows: ① Calculate the confusion matrix of the segmentation results: TP is where both the predicted result and the label value are blood vessels, TN is where both the predicted result and the label value are background, FP is where the predicted result is blood vessels and the label value is background, and FN is where the predicted result is background and the label value is blood vessels. ② ③ 3) BIoU measures the similarity between the segmentation result and the labeled image from the perspective of the coronary artery edge, quantitatively evaluating the edge segmentation accuracy of the coronary artery segmentation model. The larger the BIoU, the higher the edge segmentation accuracy of the segmentation model. The calculation method is as follows: ①Ablate d pixels from the edge of the blood vessel inwards to obtain S from the predicted result image S and the labeled image G. d and G d The boundary region is obtained by subtracting the original image from the ablation image. The BIoU is represented by calculating the intersection-union ratio of the boundary regions of the predicted image and the label image. ② 4) Betti error is the reciprocal of the Betti number error between the segmentation result and the labeled image. It measures the similarity between the segmentation result and the labeled image from a topological perspective, quantitatively evaluating the ability of the coronary artery segmentation model to preserve topological consistency. The larger the Betti error, the stronger the model's ability to preserve topological consistency. The calculation method is as follows: ① Calculate the Betty number in the segmentation result image and the blood vessel label image: Among them, B n Represents an n-dimensional Betty number. This represents the boundary mapping of an n-dimensional simplex in topology, which can map an n-dimensional simplex to an n-1-dimensional simplex; The kernel representing the boundary mapping is the set of n-dimensional simplexes mapped to the zero vector; The image represents the boundary mapping, which is the set of n+1 dimensional simplexes mapped to the n-dimensional simplex; dim represents the dimension of the vector space. ② Calculate B1 and B2 of the segmentation result image and the vessel label image, and sum them to obtain the Betti error of the segmentation result and the vessel label image. Here, B1 represents the number of connected components, and B2 represents the number of holes. The formula for calculating the Betti error is: Among them, B S B represents the number of Betty's in the segmentation result. G The number of Betty's in the labeled image; The larger the composite evaluation index VSF, the better the coronary artery segmentation model performs in terms of overall segmentation accuracy, edge segmentation accuracy, and preservation of topological consistency, and the fewer island problems and breaks appear in the segmentation results.

Images