Coronary artery segmentation method, system, terminal and storage medium
By combining the two-dimensional convolutional network and three-dimensional convolutional network, the automatic segmentation of coronary artery is achieved by using skeleton extraction and multiple seed point growth, solving the problems of inaccurate segmentation and disconnection in the existing technology, and improving the segmentation effect.
Patent Information
- Application Number
- CN202111394930.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-23
- Publication Date
- 2025-08-29
- Estimated Expiration
- 2041-11-23
AI Technical Summary
The prior art is prone to false positives or false negatives in coronary artery segmentation, and deep learning segmentation networks are prone to cause vascular disconnection and vascular deletion.
The two-dimensional convolution network is used for preliminary segmentation, combining three-dimensional reconstruction and skeleton extraction, and intercepting the region of interest is used as the center, and a variety of seed points are grown through the three-dimensional convolution network to achieve automatic segmentation of the coronary artery.
It reduces interference in non-vascular areas, avoids vascular disconnection and vascular deletion, improves segmentation effect, and can identify areas that cannot be recognized by other networks.
Smart Images
Figure CN114298971B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of medical image processing technology, and in particular relates to a coronary artery segmentation method, system, terminal and storage medium. Background Art
[0002] Blood vessel enhancement and segmentation are long-standing tasks in medical image analysis. Traditional coronary artery segmentation methods include region growing, active contour models, statistical models, shape models, particle filters, and path tracing. These traditional coronary artery segmentation methods are all interactive and require a seed point. Non-interactive coronary artery segmentation methods include level set methods and graph cuts. However, because coronary artery segmentation is considered a voxel-based classification problem, these methods may ultimately generate a large number of false positives or false negatives.
[0003] Since 2016, the application of deep learning in medical imaging has made significant progress. Various deep learning segmentation networks have been proposed, such as FCN (Fully Convolution Network), UNet, 3D Unet, and Res Unet, all of which have achieved superior segmentation results compared to traditional methods. However, when using these deep learning segmentation networks for coronary artery segmentation, due to the extremely small coronary structures of coronary arteries and the uneven characteristics of the segmented regions caused by vascular stenosis, the segmented regions are prone to vessel disconnection and loss. Summary of the Invention
[0004] The present application provides a coronary artery segmentation method, system, terminal, and storage medium, which aim to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.
[0005] In order to solve the above problems, this application provides the following technical solutions:
[0006] A coronary artery segmentation method, comprising:
[0007] Performing preliminary segmentation of the coronary artery image using a two-dimensional convolutional network to obtain a rough coronary artery segmentation result, and performing three-dimensional reconstruction on the rough coronary artery segmentation result;
[0008] Performing skeleton extraction on the three-dimensional reconstructed image using a skeleton refinement algorithm, and cutting out a region of interest on the coronary artery image with the extracted skeleton points as the center;
[0009] The region of interest is segmented using a three-dimensional convolutional network, and multiple seed point growth is performed on the segmentation results to obtain a coronary artery segmentation result.
[0010] The technical solution adopted in the embodiment of the present application also includes: the coronary artery image is a CTA image.
[0011] The technical solution adopted in the embodiment of the present application also includes: the two-dimensional convolutional network uses Deeplab v3+ as the basic network, and the Deeplab v3+ includes a dilated spatial convolution pooling pyramid, and the dilated spatial convolution pooling pyramid includes a dilated convolution layer, a convolution layer, and a global average pooling layer.
[0012] The technical solution adopted in the embodiment of the present application also includes: the training process of the two-dimensional convolutional network is specifically as follows:
[0013] The input of the two-dimensional convolutional network is the original cross-section of the CTA image. The original cross-section is used as the central cross-section. A set number of connected original cross-sections are taken as the channels of the two-dimensional convolutional network for supplementary input, and the central cross-section is used as the label for training.
[0014] The technical solution adopted in the embodiment of the present application further includes: the skeleton extraction of the 3D reconstructed image using the skeleton refinement algorithm is specifically as follows:
[0015] Make a list of candidate pixels to be removed;
[0016] Scanning the image, and deleting pixels in the candidate pixel list;
[0017] The pixels in the candidate pixel list are rechecked with the goal of maintaining the connectivity of the image, and the image is iteratively scanned until the image stops changing.
[0018] The technical solution adopted in the embodiment of the present application also includes: using a three-dimensional convolutional network to segment the region of interest, and performing multiple seed point growth on the segmentation results, specifically:
[0019] Inputting the region of interest into a three-dimensional convolutional network for segmentation, and performing three-dimensional reconstruction on the segmentation result of the three-dimensional convolutional network;
[0020] removing small connected regions of the three-dimensionally reconstructed image according to the extracted skeleton;
[0021] The skeleton points after removing the small connected areas are used as starting seed points of multiple seed points, multiple seed point growth is performed on the segmentation result of the three-dimensional convolutional network, and three-dimensional spatial mapping is performed according to the multiple seed point growth result to obtain a coronary artery segmentation result.
[0022] The technical solution adopted in the embodiment of the present application also includes: performing multiple seed point growth on the segmentation result of the three-dimensional convolutional network is specifically as follows:
[0023] Taking multiple seed points as a set, the segmentation results of the 26 neighborhoods of the seed points are added to the set of blood vessels;
[0024] The segmentation results of the 26 neighborhoods are repeatedly added to the blood vessel points using the updated set until no blood vessel points are added, and the coronary artery segmentation results are obtained.
[0025] Another technical solution adopted in the embodiment of the present application is: a coronary artery segmentation system, comprising:
[0026] Two-dimensional segmentation module: used to perform preliminary segmentation of the coronary artery image using a two-dimensional convolutional network, obtain a rough coronary artery segmentation result, and perform three-dimensional reconstruction of the rough coronary artery segmentation result;
[0027] Skeleton extraction module: used for performing skeleton extraction on the three-dimensional reconstructed image using a skeleton refinement algorithm, and cutting out a region of interest on the coronary artery image with the extracted skeleton points as the center;
[0028] Three-dimensional segmentation module: used to segment the region of interest using a three-dimensional convolutional network, and perform multiple seed point growth on the segmentation results to obtain coronary artery segmentation results.
[0029] Another technical solution adopted by the embodiment of the present application is: a terminal, the terminal including a processor and a memory coupled to the processor, wherein:
[0030] The memory stores program instructions for implementing the coronary artery segmentation method;
[0031] The processor is configured to execute the program instructions stored in the memory to control coronary artery segmentation.
[0032] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the coronary artery segmentation method.
[0033] Compared to existing technologies, the present invention provides advantageous effects in that the coronary artery segmentation method, system, terminal, and storage medium employ a combination of a two-dimensional convolutional network and a three-dimensional convolutional neural network for coronary artery segmentation. First, a two-dimensional convolutional network is used for coarse segmentation, and skeletons are extracted from the segmentation results, focusing the segmentation on vascular regions and reducing non-vascular interference. Subsequently, a three-dimensional convolutional network is used to perform coronary artery segmentation centered on the skeleton points. The present invention achieves automatic coronary artery segmentation, identifying regions that other networks cannot identify, avoiding vessel disconnections and missing vessels, and thus achieving better segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of a coronary artery segmentation method according to an embodiment of the present application;
[0035] Figure 2 FIG. 1 is a schematic diagram of a coronary artery segmentation network structure according to an embodiment of the present application;
[0036] Figure 3 This is a graph showing how the Loss and DSC (Dice Similarity Coefficient) of the 3D Unet in the training set and test set change with the number of training rounds;
[0037] Figure 4 Schematic diagram of the segmentation effect of labels, 3D Unet, Res Unet and the segmentation results of the embodiment of the present application on the CTA cross section
[0038] Figure 5 Schematic diagram of 3D reconstruction results of labels, 3D Unet, Res Unet, and segmentation results of an embodiment of the present application;
[0039] Figure 6 Schematic diagram of the coronary artery segmentation system according to an embodiment of the present application;
[0040] Figure 7 This is a schematic diagram of the terminal structure of an embodiment of the present application;
[0041] Figure 8 A schematic diagram of the structure of the storage medium of an embodiment of the present application. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0043] To address the shortcomings of the existing technology, the coronary artery segmentation method of the embodiment of the present application uses a method that combines a two-dimensional convolutional network and a three-dimensional convolutional neural network to perform coronary artery segmentation. First, a rough segmentation result is obtained by combining the two-dimensional convolutional network, and then skeleton extraction is performed on the rough segmentation result. Finally, the region is divided with the skeleton point as the center and the coronary artery segmentation is performed using the three-dimensional convolutional network, thereby reducing the interference of non-vascular areas and directly extracting more vascular areas.
[0044] Specifically, see Figure 1 , is a flow chart of a coronary artery segmentation method according to an embodiment of the present application. The coronary artery segmentation method according to an embodiment of the present application comprises the following steps:
[0045] S1: Acquire coronary artery images;
[0046] In this step, the coronary artery image is a CTA (CT angiography, CT vascular image) image.
[0047] S2: Use a two-dimensional convolutional network to perform preliminary segmentation on the coronary artery image, obtain rough coronary artery segmentation results, and perform three-dimensional reconstruction on the rough coronary artery segmentation results;
[0048] In this step, the input of the 2D convolutional network is the original cross-section of the CTA image, and the data size of the input image is [512 512]. Figure 2 As shown, this is a schematic diagram of the coronary artery segmentation network structure implemented in this application. In order to make the most of the two-dimensional plane information, the two-dimensional convolutional network in the embodiment of this application uses Deeplab v3+ with a deep network depth and a large receptive field and a resnet framework as the backbone as the basic network. At the same time, Deeplab v3+ also includes an ASPP (Atrous Spatial Pyramid Pooling) module that is optimized and improved for multi-scale information. The improved ASPP module includes a 3×3 dilated convolution layer, a 1×1 convolution layer, and a global average pooling layer, where the dilated convolution layer has a dilated rate of 12, 24, and 38, respectively. Five output results are obtained through three dilated convolution layers, one 1×1 convolution layer, and a global average pooling layer. The five output results are connected, and finally a 1×1 convolution layer is used to reduce the number of channels.
[0049] Bilinear interpolation is used on the feature map obtained by the ASPP module. After a 3×3 convolution, the result is concatenated with the result of the first block of ResNet. Another 3×3 convolution layer is then performed. Finally, the result is bilinearly interpolated to the shape of the input image and convolved to obtain the prediction result. Assuming the coordinates of the insertion point (x, y) are known, the value f(x, y) of the interpolation point needs to be calculated. Given the values of the four coordinates of the nearest points to the insertion point (x1, y1), (x2, y2), (x3, y3), and (x4, y4), the value of the interpolation point can be calculated using the following formula:
[0050]
[0051] In the embodiment of the present application, the purpose of the two-dimensional convolutional network is to identify the coronary arteries as accurately as possible, so the weighted cross entropy loss is used as the loss function of the two-dimensional convolutional network, and the weighted ratio is the ratio of the background to the foreground. In order to make up for the information loss of the two-dimensional convolutional network in three-dimensional space, the present application implements the original cross-section of the CTA image as the central cross-section, takes a set number of connected CT cross-sections as the channel of the two-dimensional convolutional network for supplementary input, and uses the central cross-section as the label for model training. Since too much supplementary information will cause a large gap with the predicted area and hinder segmentation, the embodiment of the present application takes three original cross-sections as channel supplements. The sensitivity of the original Deeplab v3+ network is 0.725. After the improvement of the embodiment of the present application, the sensitivity is 0.937, and more blood vessels are identified.
[0052] In this step, the three-dimensional reconstruction method is specifically: combining the segmentation results of each CT cross-section frame by frame to obtain a three-dimensional reconstructed image.
[0053] S3: Use the skeleton refinement algorithm to extract the skeleton of the 3D reconstructed image, and cut out a region of a set size on the CTA image with the skeleton point as the center as the region of interest;
[0054] In this step, the size of the region of interest is set to [32,64,64]. The maximum cube occupied by the blood vessels in each region of interest is an average of 38.3% of the entire region of interest, the maximum occupancy ratio is 87.98%, and the minimum is 0%, so that each region of interest includes certain background information to facilitate identification, while at the same time, there is no phenomenon that some blood vessels are not segmented. The skeleton extraction process is as follows: first, a list of candidate pixels to be removed is listed, the image is scanned, and the pixels in the candidate pixel list are deleted. Then, the pixels in the candidate pixel list are rechecked with the connectivity of the image as an indicator, and the image is scanned iteratively until the image stops changing. The embodiment of the present application uses skeleton extraction to focus the segmentation on the vascular area, reduce the interference of non-vascular areas, and help identify vascular areas that are difficult to identify.
[0055] S4: Input the region of interest into the 3D convolutional network for segmentation, perform 3D reconstruction on the segmentation result, and remove small connected areas of the reconstructed image based on the extracted skeleton;
[0056] In this step, during the network training phase, a skeleton thinning algorithm is used to extract the skeleton labels of the input image. Skeleton points are extracted from the skeleton point set with a spacing of 1. A block of size [32 6464] is cropped from the input image with the skeleton point as the cropping center, and the cropped block is added to the segmentation region for coronary artery segmentation. In this embodiment of the present application, the loss function Dice of the three-dimensional convolutional network is the loss function.
[0057] Since increasing the size of the cropping block can increase the effective perception of feature points far away from the cropping center, and using a cropping block size of [32 64 64] will affect the prediction effect, the size of the cropping block is set to [62 128 128] during the model testing phase. Figure 3 As shown in FIG, the curves of Loss and DSC (Dice Similarity Coefficient) of 3D Unet on the training set and test set as the training rounds change.
[0058] In the embodiment of the present application, the standard for removing small connected areas is: skeleton objects with less than 280 connected skeleton pixels in a 26-neighborhood connectivity mode in the skeleton area.
[0059] S5: Using the skeleton points after removing small connected areas as the starting seed points for multiple seed points, the segmentation results of the 3D convolutional network are subjected to multiple seed point growth. Three-dimensional spatial mapping is performed based on the multiple seed point growth results to obtain the final coronary artery segmentation results.
[0060] In this step, the multi-seed point growth method is as follows: using multiple seed points as a set, the segmentation results of the 26-neighborhood of the seed point are added to the vessel set. The updated set is then repeatedly added with the segmentation results of the 26-neighborhood as vessel points until no more vessel points are added. The final set is the final coronary artery segmentation result.
[0061] To further verify the feasibility and effectiveness of the embodiments of the present application, experiments were conducted in the following examples. To evaluate the performance of the method, the experiments used model evaluation indicators including Dice Similarity Coefficient (DSC), sensitivity, area under the ROC curve (AUC), and Hausdorff Distance (HD). The calculation formulas for each evaluation indicator are as follows:
[0062]
[0063]
[0064] HD(A,B)=max(h(A,B),h(B,A) (4)
[0065]
[0066]
[0067] Here, TP, FP, and FN are the numbers of true positive voxels (marked as foreground but predicted as background), false positive voxels (marked as background but predicted as foreground), and false negative voxels (marked as foreground but predicted as background). HD describes a measure of similarity between two point sets and is a definition of the distance between them. Consider two point sets A and B. h(A,B) and h(B,A) describe the one-way Hausdorff distance from point set A to point set B and from point set B to point set A, respectively. |||| represents the distance from point set A to point set B, using the Euclidean distance in this paper. HD describes the maximum value of the minimum distance from each point set to the other. To prevent extreme cases, HD95 is generally used as the metric. This refers to the minimum distance at the 0.95th position after sorting the minimum distances, not the maximum value of the minimum distance.
[0068] like Figure 4 The figure shows the segmentation effects of the label, 3D Unet, Res Unet and the segmentation results of the embodiment of the present application on the CTA cross section. Each row is a CT cross section, (a) is the segmentation effect of the label, (b) is the segmentation effect after 3D Unet post-processing to remove impurities, (c) is the segmentation effect after Res Unet post-processing to remove impurities, and (d) is the segmentation effect of the embodiment of the present application.
[0069] Figure 5 Shown are schematic diagrams of 3D reconstruction results of the label, 3D Unet, Res Unet, and the segmentation results of the embodiment of the present application, wherein (a) is the 3D reconstruction result of the label, (b) is the 3D reconstruction result of the segmentation result after 3D Unet post-processing to remove impurities, (c) is the 3D reconstruction result of the segmentation result after Res Unet post-processing to remove impurities, and (d) is the 3D reconstruction result using the segmentation result of the embodiment of the present application. The arrows point to the coronary arteries that are recognized more by the embodiment of the present application than by 3D Unet and Res Unet. Experimental results show that, compared with 3D Unet and Res Unet, the embodiment of the present application can identify many coronary arteries that other networks cannot identify, and to a certain extent solves the problems of disconnection and missing blood vessels.
[0070] Based on the above, the coronary artery segmentation method of the present embodiment utilizes a combined approach of a two-dimensional convolutional network and a three-dimensional convolutional neural network to perform coronary artery segmentation. First, a two-dimensional convolutional network is used for coarse segmentation, and skeletons are extracted from the segmentation results, focusing the segmentation on the vascular region and reducing non-vascular interference. The three-dimensional convolutional network then performs coronary artery segmentation centered on the skeleton points. This embodiment of the present application achieves automatic coronary artery segmentation and can identify regions that other networks cannot, thus avoiding vessel disconnections and missing vessels, thereby achieving better segmentation results.
[0071] See also Figure 6 , is a schematic diagram of the structure of a coronary artery segmentation system according to an embodiment of the present application. The coronary artery segmentation system 40 according to an embodiment of the present application comprises:
[0072] A two-dimensional segmentation module 41 is configured to perform preliminary segmentation of the coronary artery image using a two-dimensional convolutional network, obtain a rough coronary artery segmentation result, and perform three-dimensional reconstruction on the rough coronary artery segmentation result;
[0073] Skeleton extraction module 42: for performing skeleton extraction on the three-dimensional reconstructed image using a skeleton refinement algorithm, and cutting out a region of interest on the coronary artery image with the extracted skeleton points as the center;
[0074] The three-dimensional segmentation module 43 is used to segment the region of interest using a three-dimensional convolutional network, and perform multiple seed point growth on the segmentation results to obtain a coronary artery segmentation result.
[0075] See also Figure 7 , is a schematic diagram of the terminal structure of an embodiment of the present application. The terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51.
[0076] The memory 52 stores program instructions for implementing the above-mentioned coronary artery segmentation method.
[0077] The processor 51 is configured to execute program instructions stored in the memory 52 to control coronary artery segmentation.
[0078] The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.
[0079] See also Figure 8, which is a structural diagram of the storage medium of an embodiment of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.
[0080] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to encompass the broadest scope consistent with the principles and novel features disclosed herein.
Claims
1. A coronary artery segmentation method, characterized in that: include: Performing preliminary segmentation of the coronary artery image using a two-dimensional convolutional network to obtain a rough coronary artery segmentation result, and performing three-dimensional reconstruction on the rough coronary artery segmentation result; Performing skeleton extraction on the three-dimensional reconstructed image using a skeleton refinement algorithm, and cutting out a region of interest on the coronary artery image with the extracted skeleton points as the center; Segmenting the region of interest using a three-dimensional convolutional network, and performing multiple seed point growth on the segmentation results to obtain a coronary artery segmentation result; The coronary artery image is a CTA image; The training process of the two-dimensional convolutional network is specifically as follows: The input of the two-dimensional convolutional network is the original cross-section of the CTA image, the original cross-section is used as the central cross-section, a set number of connected original cross-sections are taken as the channels of the two-dimensional convolutional network for supplementary input, and the central cross-section is used as the label for training; The method of segmenting the region of interest using a three-dimensional convolutional network and performing multiple seed point growth on the segmentation results is as follows: Inputting the region of interest into a three-dimensional convolutional network for segmentation, and performing three-dimensional reconstruction on the segmentation result of the three-dimensional convolutional network; removing small connected regions of the three-dimensionally reconstructed image according to the extracted skeleton; The skeleton points after removing small connected areas are used as starting seed points of multiple seed points, and multiple seed points are grown on the segmentation result of the three-dimensional convolutional network. Three-dimensional spatial mapping is performed according to the multiple seed point growth result to obtain the coronary artery segmentation result.
2. The coronary artery segmentation method according to claim 1, characterized in that: The two-dimensional convolutional network uses Deeplab v3+ as the basic network, and the Deeplab v3+ includes a dilated spatial convolutional pooling pyramid, which includes a dilated convolutional layer, a convolutional layer, and a global average pooling layer.
3. The coronary artery segmentation method according to any one of claims 1 to 2, characterized in that: The skeleton extraction of the three-dimensional reconstructed image using the skeleton refinement algorithm is specifically as follows: Make a list of candidate pixels to be removed; Scanning the image, and deleting pixels in the candidate pixel list; The pixels in the candidate pixel list are rechecked with the goal of maintaining the connectivity of the image, and the image is iteratively scanned until the image stops changing.
4. The coronary artery segmentation method according to claim 3, characterized in that: The multi-point growth on the segmentation result of the three-dimensional convolutional network is specifically performed as follows: Taking multiple seed points as a set, the segmentation results of the 26 neighborhoods of the seed points are added to the set of blood vessels; The segmentation results of the 26 neighborhoods are repeatedly added to the blood vessel points using the updated set until no blood vessel points are added, and the coronary artery segmentation results are obtained.
5. A coronary artery segmentation system using the coronary artery segmentation method according to claim 1, characterized in that: include: Two-dimensional segmentation module: used to perform preliminary segmentation of the coronary artery image using a two-dimensional convolutional network, obtain a rough coronary artery segmentation result, and perform three-dimensional reconstruction of the rough coronary artery segmentation result; Skeleton extraction module: used for performing skeleton extraction on the three-dimensional reconstructed image using a skeleton refinement algorithm, and cutting out a region of interest on the coronary artery image with the extracted skeleton points as the center; Three-dimensional segmentation module: used to segment the region of interest using a three-dimensional convolutional network, and perform multiple seed point growth on the segmentation results to obtain coronary artery segmentation results.
6. A terminal, characterized in that: The terminal includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the coronary artery segmentation method according to any one of claims 1 to 4; The processor is configured to execute the program instructions stored in the memory to control coronary artery segmentation.
7. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the coronary artery segmentation method according to any one of claims 1 to 4.