Cascade refined coronary artery segmentation method based on geometric deformation enhancement

Abstract

The invention relates to the cross technical field of deep learning and medical image segmentation, in particular to a cascade refinement coronary artery segmentation method based on geometric deformation enhancement, which comprises the following steps: S1, image acquisition; s2, preprocessing the image; step S3, constructing a grid; step S4, constructing a model; s5, performing model training; aiming at the problem that the coronary artery has a complex anatomical structure, the invention designs a cascade network for segmentation of the coronary artery and vectorization of a result by integrating a geometric deformation network, and the network can generate continuous and accurate coronary artery grids, adapts to a complex coronary artery structure, avoids fragmentation of the segmentation result, and improves the segmentation efficiency. Different from a grid result generated by a traditional cube method based on voxels, the algorithm provided by the invention can reconstruct a finer vectorized coronary artery grid with a regular form, and avoids the problems of bifurcation adhesion and point cloud dispersion in complex branches.

Description

Technical Field

[0001] The present invention relates to the interdisciplinary technical field of deep learning and medical image segmentation, and specifically to a cascaded refinement coronary artery segmentation method based on geometric deformation enhancement. Background Art

[0002] Vascular disease is a common and serious health problem in the world today. It involves heart and vascular diseases caused by a variety of factors, such as hyperlipidemia, arteriosclerosis and hypertension, which often lead to cardiac ischemia or hemorrhagic diseases, posing a huge threat to human health. In this context, studying how to use deep learning algorithms to achieve intelligent segmentation of coronary angiography images will not only help improve the efficiency and accuracy of clinical diagnosis and treatment, but also help reduce the risk of misdiagnosis and missed diagnosis through standardized processes. It has great practical significance and broad application prospects for promoting early detection and timely intervention of cardiovascular diseases.

[0003] In recent years, with the development of deep learning technology, there have been many research works on the challenge of coronary vessel segmentation. Park et al. proposed a rank-based selective integration method, which aims to improve the performance of deep learning segmentation of coronary arteries by combining weight integration strategy with image-by-image quality estimation, and reduce the morphological errors that affect fully automated quantitative analysis; Nasr-Esfahani et al. used convolutional neural networks, vascular edge detection technology and threshold post-processing technology to achieve binary segmentation of coronary arteries; Samuel et al. proposed a method VSSCNet for effective segmentation of blood vessels in coronary angiography using deep learning technology. The network performs two segmentations on the preprocessed images. A vessel extraction layer with additional supervision and a unique feature map summation mechanism are used to achieve vessel positioning; Tmenova applies the CycleGAN model to simulate and reconstruct angiography images to increase the diversity of angiography datasets; Ma uses deep learning and Bayesian filters to propose a new method for dynamic coronary artery route mapping, using electrocardiogram alignment and catheter tip tracking in X-ray perspective to compensate for vascular motion caused by heart and breathing; Mulay proposes a deep learning-based edge adaptive instance normalization transfer technology for coronary artery segmentation, which combines adaptive instance normalization style transfer with dense extreme inception network and convolutional block attention module to obtain the best coronary vessel segmentation performance.

[0004] Coronary artery segmentation is the most critical step in the coronary angiography image processing process. The segmentation accuracy directly determines the subsequent diagnostic effect. Even though there are many research works based on deep learning algorithms, coronary artery segmentation still has the following technical difficulties:

[0005] ① Complex anatomical structure: The coronary artery has a unique tree-like structure, which requires the segmentation algorithm to capture the entire vascular network from the trunk to the small branches. The distribution of the tree-like coronary artery in three-dimensional space is uneven, and the diameter and direction of each branch are different, which increases the difficulty of segmentation. At the same time, the coronary artery has a tortuous path with many sharp turns and narrow areas. The image features of these areas may be similar to those of the surrounding tissues, making it difficult for image-based segmentation methods to distinguish between blood vessels and surrounding tissues;

[0006] ② Grayscale images and low contrast: Coronary angiography images use grayscale values ​​to express blood vessels and surrounding backgrounds. Although contrast agents injected into blood vessels can improve their visibility in images, the grayscale difference between the blood vessel wall and the image background is not significant, especially in some complex situations, such as coronary artery stenosis, calcification, or hemodynamic changes, the blood vessel boundary may become blurred, making it difficult for the segmentation algorithm to accurately identify the blood vessel contour;

[0007] ③. Coronary artery overlap: The coronary artery system has a complex three-dimensional structure. Different vascular branches overlap and cross each other, which appears as overlapping areas in the two-dimensional projection angiography images, increasing the difficulty of segmentation algorithm processing. Since it is difficult to intuitively distinguish the various vascular layers at the overlap, segmentation technology is required to effectively solve the problem of missing depth information and ensure accurate segmentation of each independent vascular segment.

[0008] ④. Limitations of mesh deformation methods: Existing mesh deformation methods mainly target large and regularly shaped organs, such as the liver and hippocampus. These methods are relatively effective in processing these organs because the geometric shapes of these organs are relatively simple, regular, and large in size. However, the coronary arteries have a complex tree-like structure and many small branches, which makes it difficult to directly apply traditional mesh deformation methods to the segmentation of coronary arteries. Summary of the invention

[0009] The purpose of the present invention is to provide a cascaded refinement coronary artery segmentation method based on geometric deformation enhancement. The method uses the DTR algorithm to extract the key points of the coronary artery, regards the line connecting the starting point to each branch endpoint as the center line of the corresponding coronary artery branch, and improves the geometric accuracy and details of the segmentation by generating fine grid annotations. A cascade network is constructed to combine UNet and GCN. The UNet network is used for image segmentation to process the vectorized segmentation of blood vessels.

[0010] To achieve the above object, the present invention provides the following technical solution: a cascade thinning coronary artery segmentation method based on geometric deformation enhancement, the method comprising the following steps:

[0011] Step S1, image acquisition: using coronary artery computed tomography angiography image data as the image data source, the image data is collected from public datasets and private datasets, the public dataset uses the ASOCA dataset, and the private dataset is a partial CTA image dataset collected;

[0012] Step S2, image preprocessing: preprocessing is performed before training the image, including four steps: image denoising, image cropping, image normalization and data enhancement;

[0013] Step S3, mesh construction: the overall framework of generating the coronary artery fine mesh results includes three processes, namely skeletonization, reconstruction and integration. Through skeletonization, the key points of the coronary artery tree are extracted and segmented into individual branches. These key points and coronary artery annotations are used to reconstruct the smooth surface of each branch.

[0014] Step S4, model construction: a cascade neural network based on geometric enhancement, the method includes jointly training UNet and graph convolutional network, using the image features extracted by UNet to guide the mesh deformation of GCN, and adopting a cascade network strategy to achieve mesh refinement segmentation from coarse to fine, which is composed of two Unet-GCN connection networks, performing mesh deformation and mesh refinement respectively;

[0015] Step S5, model training: Use multiple loss functions for joint optimization, use Dice loss and Focal loss for joint optimization, and the GCN network uses grid loss for optimization, including Chamfer distance loss, Laplace smoothing loss, normal consistency loss, and edge loss.

[0016] Preferably, in step S1, the ASOCA dataset contains a total of 40 training data and 20 test data, the images of the dataset have anisotropic resolution, the plane resolution is 0.3-0.4 mm, and the layer spacing is 0.625 mm. The private dataset package collects 300 CTA image data, and the images have isotropic resolution of 0.5 mm.

[0017] Preferably, in step S2, the specific steps for the image preprocessing part are as follows:

[0018] Step S2.1, image denoising: reduce random noise in the image, including detectors and electronic components, and use Gaussian filtering to denoise the image;

[0019] Step S2.1, image cropping: crop the image to remove the part of the image that does not contain the coronary artery, and crop it to a uniform size of [128, 128, 128];

[0020] Step S2.1, normalization processing: Perform Z-score normalization on the cropped CTA image. The Z-score normalization formula is as follows:

[0021] Z = (x - μ) / σ;

[0022] Where x is the HU value of the pixel in the heart image, u is the mean HU value of all pixels, and σ is the standard deviation of all pixels;

[0023] Step S2.4, data enhancement: random contrast enhancement and random mirror flipping, random horizontal flipping and random rotation are used on the data.

[0024] Preferably, in step S3, the specific steps for skeletonization, reconstruction and integration of the overall framework are as follows:

[0025] Step S3.1, skeletonization: Use the deep reinforcement tree traversal proxy algorithm to extract the key points of the coronary artery and establish its tree structure, extract the centerline of the coronary artery, capture the key of the coronary artery anatomical structure, and then perform vascular enhancement on the image. By analyzing the properties of the eigenvalues ​​of the Hessian matrix, a linear structure similarity response function is constructed to enhance the blood vessels in the two-dimensional and three-dimensional images. Then, the deep reinforcement tree traversal proxy algorithm is initialized. Finally, the key points of the coronary artery are extracted and its tree structure is established through the traversal results of the deep reinforcement model. According to the key points of the coronary artery tree identified by the DRT algorithm, the straight line connecting the starting point and the end point of each branch is regarded as the centerline of the corresponding coronary artery branch, and the centerline of each branch is interpolated using a cubic B-spline curve to make the result smooth;

[0026] Step S3.2, reconstruction: Reconstruction is performed using the key points of each branch of the coronary artery. The tangent of the key point is calculated and used as the normal vector to form a cross section of the coronary artery. On each cross section, ray sampling is performed every 15 degrees in the counterclockwise direction from the key point and intersects with the annotation of the coronary artery to form a grid layer of the coronary artery boundary. A filter is applied to smooth the ray from the key point P Ki To each boundary point Where j represents the angle of the ray, the smoothed radius is projected onto the cross section, and the boundary of the coronary artery is formed on the cross section. After the smooth boundary of the coronary artery is achieved on each cross section, a triangular patch is created using the boundary points of two adjacent cross sections to form a mesh of the coronary artery.

[0027] Step S3.3, integration: Use mesh Boolean operations to merge the coronary artery meshes of each branch to complete the entire coronary artery mesh. Assuming that the first input mesh is X and the second input mesh is Y, the union of the two manifold meshes X∪Y is a mesh that exists in X, in Y, or in both X and Y. The combined result will be a closed manifold mesh.

[0028] Preferably, in step S3.1, initializing the deep reinforcement tree traversal agent algorithm includes defining a state space, an action space, and a reward function, wherein the state space includes local features of the blood vessel, the action space includes forward, backward, and turning operations along the blood vessel path, and the reward function is designed based on the connectivity and integrity of the blood vessel structure.

[0029] Preferably, in step S3.2, the filter used is a one-dimensional Gaussian filter.

[0030] Preferably, in step 4, the first network generates a coarse grid result as the input of the second network, and the second network performs refinement deformation to generate a fine coronary artery grid representation. The specific network model structure and construction steps are as follows:

[0031] Step S4.1: In the first stage of the network, use the preprocessed cropped 3D patch block The UNet network is trained to extract the image features of the coronary arteries. Under the guidance of the UNet projected image features, the GCN is used to deform the grid to achieve vectorization of the segmentation results. The UNet and GCN are trained together;

[0032] Step S4.2, in the second stage of the network, the network parameters of the previous UNet are fixed and used directly to extract the image features of the coronary arteries. The coarse mesh results of the coronary arteries are input into the new GCN without pooling operation. The two steps are cascaded to generate the refined mesh results of the coronary arteries.

[0033] Step S4.3: Initialize the spherical grid As the input of the GCN network, Represents the vertex set, ε represents the edge set, and the entire network is replaced by a graph convolution residual block, which consists of two graph convolution layers in a residual manner. The dimensions of the input, hidden, and output features are listed in the brackets of GraphResBlock.

[0034] Preferably, in step S4.3, the spherical mesh of the ball is initialized to have 162 vertices and 480 edges.

[0035] Preferably, in step S5, the specific formula for jointly optimizing the Dice loss and the Focal loss is as follows:

[0036]

[0037] in It is the Dice loss, a loss function used to measure the overlap between the predicted results and the true annotations, which is used for image segmentation tasks; It is the Focal loss, which is used to deal with the problem of class imbalance. a and b represent the weights of Dice loss and Focal loss respectively;

[0038] Furthermore, the Dice loss formula is as follows:

[0039]

[0040] Where X is the set of pixels in the segmentation result, and Y is the set of pixels in the real result; the specific formula of Focal loss is as follows:

[0041]

[0042] Among them, α is the loss weight of the foreground. By adjusting α, the contribution of the foreground to the loss value during training can be adjusted. β is the adjustment factor. is the predicted value of the pixel in the target image sample, and p is the true value of the pixel in the target image sample.

[0043] Preferably, in step S5, the overall grid loss formula is as follows:

[0044]

[0045] Among them, λ 1 , 2 , 3 and λ 4 are the weights of the loss terms, which are 0.7, 0.1, 0.1, and 0.1 respectively. It is the Chamfer distance loss, which is used to measure the distance between the predicted grid and the true grid and guide the deformation of the grid. The Chamfer distance consists of two parts. One part is the sum of the squares of the distances from each point in the predicted grid to the nearest point of the true grid, and the other part is the sum of the squares of the distances from each point in the true grid to the nearest point of the predicted grid. The specific formula is shown below:

[0046]

[0047] Used to smooth the mesh and calculate the uniform weight of all edges connected to a vertex to maintain the smoothness of the mesh. The specific formula is as follows:

[0048]

[0049] Where N is the number of vertices in the mesh, is the vertex v i The set of neighbor vertices, v i and v j are the positions of vertex i and vertex j respectively;

[0050] Used to calculate the angle between each pair of adjacent face normals to maintain the smoothness and consistency of the mesh. The specific formula is as follows:

[0051]

[0052] Where ε represents the set of edges, e is the edge, n 0 and n 1 is the adjacent normal vector;

[0053] Used to calculate the length of each edge to avoid abnormal vertices. The specific formula is as follows:

[0054]

[0055] where M is the number of edges in the mesh, e i is the length of the ith edge, is the average length of all sides.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. In view of the problem that coronary arteries have complex anatomical structures, the present invention integrates a geometric deformation network and designs a cascade network for coronary artery segmentation and vectorization of results. The network can generate continuous and accurate coronary artery grids, adapt to complex coronary artery structures, and avoid fragmentation of segmentation results. Different from the grid results generated by the traditional voxel-based cube method, the algorithm proposed in the present invention can reconstruct a finer vectorized coronary artery grid with a regular morphology, avoiding the problems of bifurcation adhesion and point cloud dispersion in complex branches.

[0058] 2. To address the problems of grayscale images and low contrast, the present invention extracts image features through a deep learning nnUNet network, and then uses a graph convolutional network GCN to deform the initial spherical mesh to adapt to the complex geometric structure of the coronary arteries. This method not only relies on the grayscale information of the image, but also uses geometric information to enhance the accuracy of segmentation. The network can effectively extract and fuse features to improve the segmentation accuracy of the coronary arteries, and maintain a high segmentation quality even when the vascular boundaries are unclear.

[0059] 3. To address the problem of coronary artery overlap, the present invention uses a variety of loss functions to optimize network training and adopts Chamfer distance loss to measure the distance between the predicted grid and the real grid, thereby guiding the deformation of the grid. In addition, the present invention also uses Laplacian smoothing loss and normal consistency loss to ensure the smoothness and geometric consistency of the grid, which is very effective for processing vascular overlapping areas. Through the joint optimization of these loss functions, the network of the present invention can learn more accurate geometric features, thereby distinguishing different vascular layers in the vascular overlapping area and providing clear and accurate segmentation results.

[0060] 4. In view of the limitations of the mesh deformation method, the present invention first extracts the key points of the coronary arteries through a deep enhanced tree traversal agent and establishes its tree structure. Then, each branch is reconstructed using these key points and coronary artery annotations to form a smooth surface. When processing the complex multi-bifurcation of the coronary arteries, each branch is integrated to form a more realistic vascular shape. The network of the present invention can directly generate a detailed vectorized mesh from the voxel-based segmentation results without the need for complex mesh deformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is the reconstruction result of the coronary artery branches;

[0062] Figure 2 The following is a diagram of the cascade neural network architecture based on He enhancement;

[0063] Figure 3 This is the specific architecture diagram of the 3DUNet network;

[0064] Figure 4 Schematic diagram of the two-stage segmentation results of the cascade network;

[0065] Figure 5 Coronary artery segmentation results of different models. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] A cascaded thinning coronary artery segmentation method based on geometric deformation enhancement, the method comprising the following steps:

[0068] Step S1, image acquisition: The research content of the present invention is coronary artery segmentation, so the image data used is derived from coronary artery computed tomography angiography (CTA) images. The image data are collected from public data sets and private data sets. The public data set uses the ASOCA data set, and the private data set is a partial CTA image data set collected. The ASOCA data set contains 40 training data and 20 test data. The images in the data set have anisotropic resolution, a plane resolution of 0.3-0.4mm, and a layer spacing of 0.625mm; the private data set package collects 300 CTA image data, and the images have an isotropic resolution of 0.5mm. The above data sets are all annotated by professional radiologists for coronary arteries, and contain 2 labels in total, label 0 and label 1 are background label and coronary artery label respectively;

[0069] Step S2, image preprocessing. Before training the image, preprocessing is required, including four steps: image denoising, image cropping, image normalization and data enhancement. The specific steps for image preprocessing are as follows:

[0070] Step S2.1, image denoising: reduce random noise in the image, including detectors, electronic components, etc., and use Gaussian filtering to denoise the image;

[0071] Step S2.1, image cropping: crop the image to remove the part of the image that does not contain the coronary artery, and crop it to a uniform size of [128, 128, 128];

[0072] Step S2.1, normalization processing: Perform Z-score normalization on the cropped CTA image. The Z-score normalization formula is as follows:

[0073] Z = (x - μ) / σ;

[0074] Among them, x is the HU value of the pixel in the heart image, u is the mean HU value of all pixels, and σ is the standard deviation of all pixels.

[0075] Step S2.4, data enhancement: random contrast enhancement, random mirror flipping, random horizontal flipping and random rotation are used on the data;

[0076] Step S3, mesh construction. The overall framework for generating the coronary artery fine mesh results in the present invention includes three processes: skeletonization, reconstruction and integration. Through skeletonization, the key points of the coronary artery tree are extracted and divided into individual branches. The smooth surface of each branch is reconstructed using these key points and coronary artery annotations. When processing complex coronary artery bifurcations, the various branches are integrated together to form a more realistic blood vessel shape.

[0077] Step S3.1, skeletonization: The present invention uses a deep reinforced-tree-traversal agent (DRT) algorithm to extract the key points of the coronary artery and establish its tree structure. Specifically, first, the center line of the coronary artery is extracted to capture the key of the coronary artery anatomical structure. Secondly, the image is enhanced by blood vessels. The linear structure similarity response function is constructed by analyzing the properties of the eigenvalues ​​of the Hessian matrix, thereby enhancing the blood vessels in the two-dimensional and three-dimensional images. Then, the deep reinforced tree traversal agent algorithm is initialized, including defining a state space, an action space, and a reward function. The state space includes local features of the blood vessel, the action space includes operations such as forward, backward, and turning along the blood vessel path, and the reward function is designed according to the connectivity and integrity of the blood vessel structure. Finally, the key points of the coronary artery are extracted and its tree structure is established through the traversal results of the deep reinforcement model. According to the key points of the coronary artery tree identified by the DRT algorithm, the straight line connecting the starting point and the end point of each branch is regarded as the center line of the corresponding coronary artery branch, and the center line of each branch is interpolated using a cubic B-spline curve to make the result smooth.

[0078] Step S3.2, reconstruction: Reconstruction is performed using the key points of each branch of the coronary artery, such as Figure 1 As shown in (a), the tangent of the key point is calculated and used as the normal vector to form a cross section of the coronary artery. On each cross section, ray sampling is performed every 15 degrees in the counterclockwise direction from the key point and intersects with the annotation of the coronary artery to form a grid layer of the coronary artery boundary, as shown in Figure 1 As shown in (b), since the voxel annotation is composed of discrete voxels, its sparsity causes the sampling of the coronary artery boundary on the cross section to appear rough and lack complete smoothness. In order to restore the original shape of the coronary artery as much as possible, a one-dimensional Gaussian filter is applied to smooth the boundary from the key point P Ki To each border point Where j represents the angle of the ray, and the smoothed radius is projected onto the cross section to form the boundary of the coronary artery on the cross section, such as Figure 1 As shown in (c), after achieving the smooth boundary of the coronary artery on each cross section, the boundary points of two adjacent cross sections are used to create triangular patches to form the mesh of the coronary artery;

[0079] Although each boundary on the cross section is flat, the generated coronary artery mesh is coarse due to the voxel-based discrete annotation. Figure 1(d) shows that the boundaries of adjacent coronary arteries vary greatly and lack seamless transitions, which is far from the actual coronary arteries. Therefore, in addition to smoothing the boundaries on the cross section, the coronary artery mesh also needs to be flattened along the direction of the key points. According to the tubular structure of the coronary artery, the lines composed of boundary points sampled at the same angle are smoothed, as shown in Figure 2. When smoothing, it is very important to determine the direction of the starting point so that the polar coordinate system directions of adjacent cross sections can be as consistent as possible. Therefore, the starting ray of the previous cross section is projected to the next cross section to determine the direction of the starting point. The coronary artery mesh after smoothing along the key points is as follows: Figure 1 As shown in (e), the generated coronary artery mesh not only retains the geometric shape but is also softer and closer to the real coronary artery;

[0080] Step S3.3, integration: using mesh Boolean operation to merge the coronary mesh of each branch, so as to complete the whole coronary mesh. The mesh Boolean operation is performed on the closed manifold mesh. Assuming that the first input mesh is X and the second input mesh is Y, the union X∪Y of the two manifold meshes is a mesh that exists in X, in Y, or in both X and Y. The combination result will be a closed manifold mesh, which has a closed solid shape and contains the volume of all input meshes. Different from the previous methods, the method of the present invention reconstructs and merges each branch mesh separately, rather than generating the whole coronary mesh as a whole. This method helps to avoid complex modeling and greatly reduces the computational burden associated with coronary bifurcation, especially in the case of trifurcation. In addition, since each branch of the bifurcation shares the same trunk, the transition between the branches is smoother and closer to the actual structure of the coronary blood vessels.

[0081] Step S4, model construction: In order to directly generate a vectorized grid of the coronary artery, the present invention proposes a cascade neural network based on geometric enhancement. The method includes jointly training UNet and graph convolutional network, and using the image features extracted by UNet to guide the grid deformation of GCN. In addition, the present invention adopts a cascade network strategy to achieve grid refinement segmentation from coarse to fine. The strategy consists of two Unet-GCN connection networks, which perform grid deformation and grid refinement respectively. The first network generates a coarse grid result as the input of the second network. The second network performs refinement deformation to generate a fine coronary artery grid representation. The specific network model structure is as follows Figure 2 As shown, the specific steps are as follows:

[0082] Step S4.1: In the first stage of the network, use the preprocessed cropped 3D patch block The UNet network is trained to extract the image features of the coronary arteries. Under the guidance of the UNet projected image features, the GCN is used to deform the grid to achieve vectorization of the segmentation results. The UNet and GCN are trained together;

[0083] Step S4.2, in the second stage of the network, the network parameters of the previous UNet are fixed and directly used to extract the image features of the coronary arteries. The coarse grid results of the coronary arteries are input into the new GCN without pooling operation. The two steps are cascaded to generate the refined grid results of the coronary arteries. The specific parameters of the 3DUNet network structure are shown in Table 1. ConvBlock consists of two convolutional layers. The maximum pooling is performed after encoder-0, encoder-1, encoder-2 and encoder-3, with a step size of 2. In addition, jump connections are used between encoder-0 and decoder-0, encoder-1 and decoder-1, and encoder-2 and decoder-2. The overall structure of 3DUNet is basically consistent with that of UNet.

[0084] Table 13 DUNet network specific architecture diagram

[0085] layer Output size Network structure Encoder-0 32×32×32 conv(3×3×3.32) Encoder-1 16×16×16 ConvBlock(3×3×3.64) Encoder-2 8×8×8 ConvBlock(3×3×3.128) Encoder-3 4×4×4 ConvBlock(3×3×3.256) Decoder-3 4×4×4 ConvBlock(3×3×3.256) Decoder-2 8×8×8 ConvBlock(3×3×3.128) Decoder-1 16×16×16 ConvBIock(3×3×3.64) Decoder-0 32×32×32 ConvBlock(3×3×3.32) Classifier 32×32×32 conv(1×1×1, 1)

[0086] Step S4.3: The specific structure of the graph convolutional network is shown in Table 2. Initialize a spherical grid with 162 vertices and 480 edges. As the input of the GCN network, Represents the vertex set, ε represents the edge set, and the entire network is replaced by a graph convolution residual block, which consists of two graph convolution layers in a residual manner. The dimensions of the input, hidden, and output features are listed in the brackets of GraphResBlock. Graph-0 and Graph-1 are followed by a graph pooling operation to divide each triangle into four triangles;

[0087] Table 2 shows the specific architecture of the graph convolutional network.

[0088]

[0089] Step S5: Model training:

[0090] In order to train 3DUNet and GCN networks together, the present invention uses multiple loss functions for joint optimization. The image loss is mainly based on voxel segmentation UNet, so Dice loss and Focal loss are used for joint optimization. The specific formula is as follows:

[0091]

[0092] in It is the Dice loss, a loss function used to measure the overlap between the predicted results and the true annotations, which is used for image segmentation tasks; It is the Focal loss, which is used to deal with the problem of class imbalance. a and b represent the weights of Dice loss and Focal loss respectively;

[0093] Furthermore, the Dice loss formula is as follows:

[0094]

[0095] Where X is the set of pixels in the segmentation result, and Y is the set of pixels in the real result; the specific formula of Focal loss is as follows:

[0096]

[0097] Among them, α is the loss weight of the foreground. By adjusting α, the contribution of the foreground to the loss value during training can be adjusted. β is the adjustment factor. is the predicted value of the pixel in the target image sample, and p is the true value of the pixel in the target image sample;

[0098] The GCN network is optimized using grid loss, which includes Chamfer distance loss, Laplace smoothing loss, normal consistency loss, and edge loss. The overall grid loss formula is as follows:

[0099]

[0100] Among them, λ 1 , 2 , 3 and λ 4 are the weights of the loss terms, which are 0.7, 0.1, 0.1, and 0.1 respectively. It is the Chamfer distance loss, which is used to measure the distance between the predicted grid and the true grid and guide the deformation of the grid. The Chamfer distance consists of two parts. One part is the sum of the squares of the distances from each point in the predicted grid to the nearest point of the true grid, and the other part is the sum of the squares of the distances from each point in the true grid to the nearest point of the predicted grid. The specific formula is shown below:

[0101]

[0102] Used to smooth the mesh and calculate the uniform weight of all edges connected to a vertex to maintain the smoothness of the mesh. The specific formula is as follows:

[0103]

[0104] Where N is the number of vertices in the mesh, is the vertex v i The set of neighbor vertices, vi and v j are the positions of vertex i and vertex j respectively;

[0105] Used to calculate the angle between each pair of adjacent face normals to maintain the smoothness and consistency of the mesh. The specific formula is as follows:

[0106]

[0107] Where ε represents the set of edges, e is the edge, n 0 and n 1 is the adjacent normal vector;

[0108] Used to calculate the length of each edge to avoid abnormal vertices. The specific formula is as follows:

[0109]

[0110] where M is the number of edges in the mesh, e i is the length of the ith edge, is the average length of all sides.

[0111] Experimental verification:

[0112] Based on the above method, the present invention carried out the following experiments. The first experiment was a vectorized grid generation experiment, as follows Figure 3 As shown, the above result is the reconstruction result obtained using the marching-cube method, and the result below is the mesh generated by the algorithm of the present invention. The method of the present invention successfully reconstructs the smooth surface of the coronary artery and captures the rich details of small and narrow branches. In addition, the method of the present invention is designed to handle various complex coronary artery structures, including trifurcations and even quadrifurcations, and the transition at the junction of multiple bifurcations appears natural and realistic. In addition, the reconstruction of the present method accurately retains the tubular morphology of the coronary artery, even when the blood vessel is compressed at the plaque, which ensures that the reconstruction of the present method is very similar to the real anatomical structure of the coronary artery.

[0113] The second experiment is to test the output results of the cascade network, as follows Figure 4 As shown in the figure, mesh deformation is the output of the first stage, and mesh refinement is the output of the second stage. From the experimental results, it can be seen that the mesh deformation in stage one cannot eliminate the interference between blood vessels, causing the blood vessels to look stuck together. On the contrary, the mesh refinement process effectively masks the redundant blood vessels, retaining only a central, larger blood vessel in the image input. This method ensures that adjacent parallel blood vessels do not stick to each other, thereby preventing mesh overlap.

[0114] The third experiment is to segment the coronary artery using different model methods. The experimental results are as follows Figure 5 As shown, the experimental results show that voxel-based segmentation usually leads to vascular fragmentation, especially for coronary arteries with complex and twisted structures. In contrast, the geometry-based method of the present invention retains the complete and complex coronary artery structure and effectively avoids the problem of vascular breakage. In addition, the method of the present invention accurately depicts the tiny and narrow branches of the coronary arteries, overcomes the limitations of the sparsity and low resolution of CTA images, and the overall segmentation result of the coronary arteries maintains the smoothness of the blood vessels, thereby producing a more realistic morphology.

[0115] The present invention uses the DTR algorithm to extract the key points of the coronary arteries. Based on these key points, the line connecting the starting point to the endpoint of each branch is regarded as the center line of the corresponding coronary artery branch. Morphological regularization is proposed for coronary artery segmentation. The geometric accuracy and details of the segmentation are improved by generating fine grid annotations. A cascade network is constructed to combine UNet and GCN. The UNet network is used for image segmentation, and the graph convolution network is used to process the vectorized segmentation of blood vessels.

[0116] In the prior art, the centerline of the coronary artery is generally extracted using a skeletonization algorithm. This method requires image segmentation or distance map generation first, which increases the computational cost. At the same time, when processing complex anatomical structures such as coronary arteries, it is impossible to capture the entire vascular network from the trunk to the small branches. The present invention uses a DRT algorithm, a multi-task discriminator and a dynamic reward mechanism, which can effectively process the bifurcation points of the coronary arteries and determine the tracking order without manual intervention. The multi-task discriminator is trained independently of the DRT agent and can quickly calculate the distance from the point to the nearest bifurcation point and the reference point, thereby improving the accuracy of bifurcation point and terminal point detection.

[0117] In the prior art, a complete coronary artery mesh is generated as a whole. This method increases the computational burden of coronary artery bifurcation, especially in the case of trifurcation, where the generated mesh cannot fit the coronary artery blood vessel situation. The present invention is divided into three steps: skeletonization, reconstruction, and merging. Reconstructing and merging each branch mesh effectively avoids complex modeling, and because each branch of the bifurcation shares the same trunk, the transition between branches is smoother, closer to the actual structure of the coronary artery blood vessel, and can capture more rich details of small and narrow branches.

[0118] In the prior art, most coronary artery segmentation uses a single UNet network or a GCN network. This method cannot handle the situation where different vascular branches overlap and cross each other, and cannot generate a coronary artery grid with continuity and integrity, resulting in the problem of vascular fragmentation. The present invention constructs a cascade network, combines UNet and a graph convolutional network, and realizes accurate segmentation and vectorization of coronary arteries. Through a coarse-to-fine segmentation strategy, a rough grid result is first generated, and then refined as the input of the second network, gradually improving the accuracy of segmentation. UNet shares weights in two stages, ensuring the consistency and accuracy of image feature extraction, speeding up the network training speed, and adding the use of GCN, the network can better utilize geometric features, improve the accuracy of segmentation and the smoothness of the grid.

[0119] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0120] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cascaded thinning coronary artery segmentation method based on geometric deformation enhancement, characterized in that: The method comprises the following steps: Step S1, image acquisition: using coronary artery computed tomography angiography image data as the image data source, the image data is collected from public datasets and private datasets, the public dataset uses the ASOCA dataset, and the private dataset is a partial CTA image dataset collected; Step S2, image preprocessing: preprocessing is performed before training the image, including four steps: image denoising, image cropping, image normalization and data enhancement; Step S3, mesh construction: the overall framework of generating the coronary artery fine mesh results includes three processes, namely skeletonization, reconstruction and integration. Through skeletonization, the key points of the coronary artery tree are extracted and segmented into individual branches. These key points and coronary artery annotations are used to reconstruct the smooth surface of each branch. Step S4, model construction: a cascade neural network based on geometric enhancement, the method includes jointly training UNet and graph convolutional network, using the image features extracted by UNet to guide the mesh deformation of GCN, and adopting a cascade network strategy to achieve mesh refinement segmentation from coarse to fine, which is composed of two Unet-GCN connection networks, performing mesh deformation and mesh refinement respectively; Step S5, model training: Use multiple loss functions for joint optimization, use Dice loss and Focal loss for joint optimization, and the GCN network uses grid loss for optimization, including Chamfer distance loss, Laplace smoothing loss, normal consistency loss, and edge loss.

2. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 1, characterized in that: In step S1, the ASOCA dataset contains 40 training data and 20 test data. The images in the dataset have anisotropic resolution, a plane resolution of 0.3-0.4 mm, and a layer spacing of 0.625 mm. The private dataset package collects 300 CTA image data, and the images have an isotropic resolution of 0.5 mm.

3. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 1, characterized in that: In step S2, the specific steps for image preprocessing are as follows: Step S2.1, image denoising: reduce random noise in the image, including detectors and electronic components, and use Gaussian filtering to denoise the image; Step S2.1, image cropping: crop the image to remove the part of the image that does not contain the coronary artery, and crop it to a uniform size of [128, 128, 128]; Step S2.1, normalization processing: Perform Z-score normalization on the cropped CTA image. The Z-score normalization formula is as follows: Z = (x - μ) / σ; Where x is the HU value of the pixel in the heart image, u is the mean HU value of all pixels, and σ is the standard deviation of all pixels; Step S2.4, data enhancement: random contrast enhancement and random mirror flipping, random horizontal flipping and random rotation are used on the data.

4. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 1, characterized in that: In step S3, the specific steps for skeletonization, reconstruction and integration of the overall framework are as follows: Step S3.1, skeletonization: Use the deep reinforcement tree traversal proxy algorithm to extract the key points of the coronary artery and establish its tree structure, extract the centerline of the coronary artery, capture the key of the coronary artery anatomical structure, and then perform vascular enhancement on the image. By analyzing the properties of the eigenvalues ​​of the Hessian matrix, a linear structure similarity response function is constructed to enhance the blood vessels in the two-dimensional and three-dimensional images. Then, the deep reinforcement tree traversal proxy algorithm is initialized. Finally, the key points of the coronary artery are extracted and its tree structure is established through the traversal results of the deep reinforcement model. According to the key points of the coronary artery tree identified by the DRT algorithm, the straight line connecting the starting point and the end point of each branch is regarded as the centerline of the corresponding coronary artery branch, and the centerline of each branch is interpolated using a cubic B-spline curve to make the result smooth; Step S3.2, reconstruction: Reconstruction is performed using the key points of each branch of the coronary artery. The tangent of the key point is calculated and used as the normal vector to form a cross section of the coronary artery. On each cross section, ray sampling is performed every 15 degrees in the counterclockwise direction from the key point and intersects with the annotation of the coronary artery to form a grid layer of the coronary artery boundary. A filter is applied to smooth the ray from the key point P Ki To each boundary point Where j represents the angle of the ray, the smoothed radius is projected onto the cross section, and the boundary of the coronary artery is formed on the cross section. After the smooth boundary of the coronary artery is achieved on each cross section, a triangular patch is created using the boundary points of two adjacent cross sections to form a mesh of the coronary artery. Step S3.3, integration: Use mesh Boolean operations to merge the coronary artery meshes of each branch to complete the entire coronary artery mesh. Assuming that the first input mesh is X and the second input mesh is Y, the union of the two manifold meshes X∪Y is a mesh that exists in X, in Y, or in both X and Y. The combined result will be a closed manifold mesh.

5. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 4, characterized in that: In step S3.1, initializing the deep reinforcement tree traversal agent algorithm includes defining a state space, an action space, and a reward function. The state space includes local features of the blood vessel, the action space includes forward, backward, and turning operations along the blood vessel path, and the reward function is designed based on the connectivity and integrity of the blood vessel structure.

6. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 4, characterized in that: In step S3.2, the filter used is a one-dimensional Gaussian filter.

7. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 1, characterized in that: In step 4, the first network generates a coarse grid result as the input of the second network, and the second network performs refinement deformation to generate a fine coronary artery grid representation. The specific network model structure and construction steps are as follows: Step S4.1: In the first stage of the network, use the preprocessed cropped 3D patch block The UNet network is trained to extract the image features of the coronary arteries. Under the guidance of the UNet projected image features, the GCN is used to deform the grid to achieve vectorization of the segmentation results. The UNet and GCN are trained together; Step S4.2, in the second stage of the network, the network parameters of the previous UNet are fixed and used directly to extract the image features of the coronary arteries. The coarse mesh results of the coronary arteries are input into the new GCN without pooling operation. The two steps are cascaded to generate the refined mesh results of the coronary arteries. Step S4.3: Initialize the spherical grid As the input of the GCN network, Represents the vertex set, ε represents the edge set, and the entire network is replaced by a graph convolution residual block, which consists of two graph convolution layers in a residual manner. The dimensions of the input, hidden, and output features are listed in the brackets of GraphResBlock.

8. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 7, characterized in that: In step S4.3, the spherical mesh of the ball is initialized with 162 vertices and 480 edges.

9. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 1, characterized in that: In step S5, the specific formula for jointly optimizing the Dice loss and the Focal loss is as follows: in It is the Dice loss, a loss function used to measure the overlap between the predicted results and the true annotations, which is used for image segmentation tasks; It is the Focal loss, which is used to deal with the problem of class imbalance. a and b represent the weights of Dice loss and Focal loss respectively; Furthermore, the Dice loss formula is as follows: Where X is the set of pixels in the segmentation result, and Y is the set of pixels in the real result; the specific formula of Focal loss is as follows: Among them, α is the loss weight of the foreground. By adjusting α, the loss value contribution of the foreground during training can be adjusted. β is the adjustment factor. p is the predicted value of the pixel in the target image sample, and p is the true value of the pixel in the target image sample.

10. The method for segmenting coronary arteries by cascading thinning based on geometric deformation enhancement according to claim 1, characterized in that: In step S5, the overall grid loss formula is as follows: Among them, λ1, λ2, λ3 and λ4 are the weights of the loss term, which are 0.7, 0.1, 0.1 and 0.1 respectively. It is the Chamfer distance loss, which is used to measure the distance between the predicted grid and the true grid and guide the deformation of the grid. The Chamfer distance consists of two parts. One part is the sum of the squares of the distances from each point in the predicted grid to the nearest point of the true grid, and the other part is the sum of the squares of the distances from each point in the true grid to the nearest point of the predicted grid. The specific formula is shown below: Used to smooth the mesh and calculate the uniform weight of all edges connected to a vertex to maintain the smoothness of the mesh. The specific formula is as follows: Where N is the number of vertices in the mesh, is the vertex v i The set of neighbor vertices, v i and v j are the positions of vertex i and vertex j respectively; Used to calculate the angle between each pair of adjacent face normals to maintain the smoothness and consistency of the mesh. The specific formula is as follows: Where ε represents the set of edges, e is the edge, n0 and n1 are adjacent normal vectors; Used to calculate the length of each edge to avoid abnormal vertices. The specific formula is as follows: where M is the number of edges in the mesh, e i is the length of the ith edge, is the average length of all sides.

Images