A two-stage 3D coronary artery segmentation reconstruction method and system

Abstract

The application provides a two-stage 3D coronary artery segmentation and reconstruction method and system, which comprises the following steps: obtaining a real label of a coronary artery and performing inflation by using an inflation operator to obtain an inflation label of the coronary artery; taking the inflation label of the coronary artery as supervision information, obtaining a 3D cardiovascular image and inputting the 3D cardiovascular image into a trained coarse segmentation model to perform coarse segmentation, obtaining a position feature map of the coronary artery and performing feature fusion with the 3D cardiovascular image; performing segmentation slicing and slice combination on the fused 3D cardiovascular image to obtain a plurality of 2D cardiovascular image combination slices and inputting the 2D cardiovascular image combination slices into a trained fine segmentation model to perform fine segmentation processing; and finally stacking all the fine segmented 2D cardiovascular image slices to obtain a reconstructed 3D coronary artery image, thereby completing the segmentation and reconstruction of the 3D coronary artery; the two-stage segmentation and reconstruction of the coronary artery from coarse to fine can effectively improve the accuracy of the coronary artery reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence medical and medical image diagnosis, more particularly, to a two-stage 3D coronary artery segmentation reconstruction method and system. BACKGROUND

[0002] Coronary heart disease is one of the primary diseases that endanger human health, and coronary angiography is one of the main bases for clinical diagnosis and treatment of coronary heart disease, and is called the "gold standard" for diagnosing coronary heart disease. The three-dimensional reconstruction technology of blood vessels not only provides doctors with visual three-dimensional blood vessel shape images, but also assists in measuring relevant parameters of blood vessels (such as diameter, blood vessel length and cross-sectional area, etc.), thereby helping the diagnosis and treatment of coronary heart disease. Therefore, segmentation and reconstruction of the coronary artery can help subsequent artificial intelligence assisted clinical diagnosis research, and have good clinical significance and high application value.

[0003] The prior art discloses a semi-supervised coronary artery segmentation system and method combining multiple networks, two-dimensional image slices are extracted along the coordinate axis as samples on the original image to construct a training data set and a test data set; a convolutional neural network is constructed for identification of two-dimensional blood vessel slices; a multi-scale feature decomposition network is constructed for segmentation of coronary artery blood vessels in two-dimensional blood vessel slices; a loss function combining supervised learning and unsupervised learning is designed; the multi-scale feature decomposition network is trained with the slices containing blood vessels as input, and the blood vessel segmentation task is completed on the test image. The method in the prior art identifies two-dimensional blood vessel slices by constructing a convolutional neural network, segments coronary artery blood vessels in two-dimensional blood vessel slices by constructing a multi-scale feature decomposition network, and performs semi-supervised learning combined with unlabeled data. Although the method can reduce the difficulty of obtaining data labels and improve the segmentation accuracy, the method is a single-stage reconstruction scheme based on deep learning. The defect of the single-stage scheme is that it focuses on the global structure of the coronary artery and ignores the local details of the coronary artery, and it is not suitable for the sparsity of the coronary artery in the CT image. In addition, the single-stage scheme may have problems of fracture and branch loss when facing complex coronary artery reconstruction. SUMMARY

[0004] To overcome the defect of the single-stage scheme in the prior art that may have problems of fracture and branch loss when facing complex coronary artery reconstruction, the present application provides a two-stage 3D coronary artery segmentation reconstruction method and system, which segments and reconstructs the coronary artery from coarse to fine, and can improve the accuracy of coronary artery reconstruction.

[0005] To solve the above technical problems, the technical scheme of the present application is as follows:

[0006] The present application provides a two-stage 3D coronary artery segmentation reconstruction method, comprising the following steps:

[0007] S1: obtaining a real label of the coronary artery and performing inflation using an inflation operator to obtain an inflation label of the coronary artery;

[0008] S2: taking the inflation label of the coronary artery as supervision information, obtaining a 3D cardiovascular image and inputting the obtained 3D cardiovascular image into a trained coarse segmentation model to perform coarse segmentation, and obtaining a position feature map of the coronary artery;

[0009] S3: performing feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image to obtain a fused 3D cardiovascular image;

[0010] S4: performing slice segmentation and slice combination on the fused 3D cardiovascular image to obtain a plurality of 2D cardiovascular image combination slices;

[0011] S5: inputting all the 2D cardiovascular image combination slices into a trained fine segmentation model to perform fine segmentation processing, and obtaining 2D cardiovascular image slices after fine segmentation;

[0012] S6: stacking all the 2D cardiovascular image slices after fine segmentation to obtain a reconstructed 3D coronary artery image, and completing segmentation and reconstruction of the 3D coronary artery.

[0013] Preferably, in the step S1, the specific method for obtaining the inflation label of the coronary artery by inflating the real label of the coronary artery using the inflation operator is as follows:

[0014] An inflation coefficient is set, and the real label of the coronary artery is inflated in a spherical manner using the inflation operator to obtain the inflation label of the coronary artery.

[0015] Preferably, the coarse segmentation model in the step S2 is a 3D U-Net neural network model based on CNN.

[0016] Preferably, in the step S3, the position feature map of the coronary artery is specifically a soft-feature map generated by the trained coarse segmentation model in the segmentation process.

[0017] Preferably, in the step S4, the specific method for obtaining the plurality of 2D cardiovascular image combination slices by performing slice segmentation and slice combination on the fused 3D cardiovascular image in the Z-axis direction is as follows:

[0018] Let the size of the fused 3D cardiovascular image be X×Y×Z, then perform slice segmentation on the fused 3D cardiovascular image in the Z-axis direction to obtain Z 2D cardiovascular image slices with a size of X×Y, and the i-th 2D cardiovascular image slice is denoted as I i (i=1, 2…Z), and the corresponding inflation label is denoted as L i ;

[0019] combining adjacent n 2D cardiovascular image slices to obtain a 2D cardiovascular image combined slice with a size of X*Y*n.

[0020] Preferably, the fine segmentation model in the step S5 is specifically a 3D P2T neural network model based on a Transformer, comprising an encoder and a decoder connected in sequence.

[0021] The encoder comprises a first embedding layer, a first pyramid pooling Transformer layer, a second embedding layer, a second pyramid pooling Transformer layer, a third embedding layer, a third pyramid pooling Transformer layer, a fourth embedding layer and a fourth pyramid pooling Transformer layer connected in sequence.

[0022] The decoder comprises a first upsampling layer, a first connection layer, a second upsampling layer, a second connection layer, a third upsampling layer, a third connection layer, a fourth upsampling layer and a convolution layer connected in sequence.

[0023] The output end of the first pyramid pooling Transformer layer is further connected to the input end of the third connection layer, the output end of the second pyramid pooling Transformer layer is further connected to the input end of the second connection layer, and the output end of the third pyramid pooling Transformer layer is further connected to the input end of the first connection layer.

[0024] Preferably, the coarse segmentation model and the fine segmentation model are both provided with a skip-connection.

[0025] The application further provides a two-stage 3D coronary artery segmentation reconstruction system applying the two-stage 3D coronary artery segmentation reconstruction method.

[0026] The data acquisition unit is used to acquire a real label of the coronary artery, and perform inflation by using an inflation operator to acquire an inflation label of the coronary artery.

[0027] The coarse segmentation unit is used to acquire a 3D cardiovascular image and input the 3D cardiovascular image into the trained coarse segmentation model for coarse segmentation by taking the inflation label of the coronary artery as supervision information to acquire a position feature map of the coronary artery.

[0028] The feature fusion unit is used to perform feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image to acquire a fused 3D cardiovascular image.

[0029] The segmentation and slicing unit is used to perform segmentation and slicing on the fused 3D cardiovascular image and combine the sliced images to obtain a plurality of 2D cardiovascular image combined slices.

[0030] fine segmentation unit: used for combining all 2D coronary image slices into a trained fine segmentation model for fine segmentation processing, to obtain 2D coronary image slices after fine segmentation;

[0031] reconstruction unit: used for stacking all 2D coronary image slices after fine segmentation, to obtain reconstructed 3D coronary image, to complete segmentation and reconstruction of 3D coronary artery.

[0032] The application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps in the above method.

[0033] The application also provides an electronic device, which comprises a processor and a memory, and the memory stores computer readable instructions, and the computer readable instructions are executed by the processor to run the steps in the above method.

[0034] Compared with the prior art, the beneficial effects of the technical scheme of the application are:

[0035] The application provides a two-stage 3D coronary artery segmentation and reconstruction method and system, which first obtains a real label of a coronary artery, and uses an inflation operator to perform inflation to obtain an inflation label of the coronary artery; takes the inflation label of the coronary artery as supervision information, obtains a 3D cardiovascular image, and inputs the 3D cardiovascular image into a trained coarse segmentation model to perform coarse segmentation, to obtain a position feature map of the coronary artery; performs feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image, to obtain a fused 3D cardiovascular image; performs segmentation slicing and slice combination on the fused 3D cardiovascular image in the Z-axis direction, to obtain a plurality of 2D cardiovascular image combination slices; inputs all the 2D cardiovascular image combination slices into a trained fine segmentation model to perform fine segmentation processing, to obtain 2D coronary image slices after fine segmentation; and finally stacks all the 2D coronary image slices after fine segmentation along the Z-axis direction, to obtain a reconstructed 3D coronary image, to complete segmentation and reconstruction of the 3D coronary artery.

[0036] The present application is based on deep learning, and the two-stage coarse-to-fine 3D coronary artery reconstruction method can reduce the workload of clinical experts, and the reconstructed high-quality coronary artery can also provide assistance for cardiovascular disease diagnosis; at the same time, the two-stage method is used to reconstruct the 3D coronary artery, which can effectively combine the global and local information, and gradually refine the 3D coronary artery; in addition, the present application also uses the inflation label supervision strategy to ensure the integrity of the global structure of the coronary artery as much as possible, and solves the problems of fracture and branch missing caused by the complex structure of the coronary artery during single model reconstruction; in addition, the present application also introduces a feature fusion strategy to emphasize the position of the coronary artery, and provides more comprehensive and effective guidance for the second stage of fine segmentation; in the fine segmentation stage, the multi-slice model is used, which can ensure that the model learns the details of the coronary artery while not losing the connectivity of the blood vessels, and the reconstruction effect is better. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A two-stage 3D coronary artery segmentation and reconstruction method flowchart provided for example 1.

[0038] Figure 2 A two-stage 3D coronary artery segmentation and reconstruction method flowchart provided for example 2.

[0039] Figure 3 A coarse segmentation model structure diagram provided for example 2.

[0040] Figure 4 A feature fusion diagram of the position feature map of the coronary artery and the 3D cardiovascular image provided for example 2.

[0041] Figure 5 A 2D cardiovascular image combination slice flowchart provided for example 2.

[0042] Figure 6 A fine segmentation model structure diagram provided for example 2.

[0043] Figure 7 A two-stage 3D coronary artery segmentation and reconstruction system structure diagram provided for example 3. DETAILED DESCRIPTION

[0044] The drawings are only used for illustrative description, and cannot be understood as a limitation of the present patent;

[0045] In order to better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0046] For those skilled in the art, it is understandable that some well-known structures in the drawings and their descriptions may be omitted.

[0047] The technical solutions of the present application are further described below in combination with the drawings and examples.

[0048] Example 1

[0049] As Figure 1 shown, the present embodiment provides a two-stage 3D coronary artery segmentation and reconstruction method, including the following steps:

[0050] S1: Obtain the real label of the coronary artery, and use the dilation operator to dilate to obtain the dilated label of the coronary artery;

[0051] S2: Take the dilated label of the coronary artery as the supervision information, obtain the 3D cardiovascular image and input it into the trained coarse segmentation model for coarse segmentation to obtain the position feature map of the coronary artery;

[0052] S3: Fuse the position feature map of the coronary artery with the 3D cardiovascular image to obtain the fused 3D cardiovascular image;

[0053] S4: Segment and combine the fused 3D cardiovascular image to obtain a plurality of 2D cardiovascular image combination slices;

[0054] S5: Input all the 2D cardiovascular image combination slices into the trained fine segmentation model for fine segmentation processing to obtain the fine segmented 2D cardiovascular image slices;

[0055] S6: Stack all the fine segmented 2D cardiovascular image slices to obtain the reconstructed 3D coronary artery image, completing the segmentation and reconstruction of the 3D coronary artery.

[0056] In the specific implementation process, first, the 3D cardiovascular image and the real label of the coronary artery are obtained, and the real label is dilated using the dilation operator to obtain the dilated label of the coronary artery; the dilated label of the coronary artery is taken as the supervision information, and the 3D cardiovascular image is input into the trained coarse segmentation model for coarse segmentation to obtain the position feature map of the coronary artery; the position feature map of the coronary artery is fused with the 3D cardiovascular image to obtain the fused 3D cardiovascular image; the fused 3D cardiovascular image is segmented and combined in the Z-axis direction to obtain a plurality of 2D cardiovascular image combination slices; all the 2D cardiovascular image combination slices are input into the trained fine segmentation model for fine segmentation processing to obtain the fine segmented 2D cardiovascular image slices; finally, all the fine segmented 2D cardiovascular image slices are stacked along the Z-axis direction to obtain the reconstructed 3D coronary artery image, completing the segmentation and reconstruction of the 3D coronary artery.

[0057] The method is based on deep learning, and a two-stage coarse-to-fine 3D coronary artery reconstruction method can reduce the workload of clinical experts, and the reconstructed high-quality coronary artery can provide assistance for cardiovascular disease diagnosis; meanwhile, the two-stage method can effectively combine global and local information to gradually refine the 3D coronary artery; in addition, the method also uses an inflation label supervision strategy to ensure the integrity of the global structure of the coronary artery as much as possible, and solves the problems of fracture and branch missing caused by the complex structure of the coronary artery during single model reconstruction; in addition, the method also introduces a feature fusion strategy to emphasize the position of the coronary artery, and provides more comprehensive and effective guidance for the second stage of fine segmentation; in the fine segmentation stage, a multi-slice model is used to ensure that the model learns the details of the coronary artery while not losing the connectivity of the blood vessels, and the reconstruction effect is better.

[0058] Embodiment 2

[0059] As shown in Figure 2 The application provides a two-stage 3D coronary artery segmentation and reconstruction method, comprising the following steps:

[0060] S1: obtaining the real label of the coronary artery, and using an inflation operator to perform inflation to obtain the inflation label of the coronary artery, and the specific method is:

[0061] setting an inflation coefficient, performing spherical inflation on the real label of the coronary artery by using the inflation operator to obtain the inflation label of the coronary artery;

[0062] S2: using the inflation label of the coronary artery as supervision information, obtaining a 3D cardiovascular image and inputting the 3D cardiovascular image into a trained coarse segmentation model to perform coarse segmentation, and obtaining a position feature map of the coronary artery;

[0063] S3: performing feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image to obtain a fused 3D cardiovascular image;

[0064] S4: performing segmentation slicing and slice combination on the fused 3D cardiovascular image to obtain a plurality of 2D cardiovascular image combination slices;

[0065] S5: inputting all the 2D cardiovascular image combination slices into a trained fine segmentation model to perform fine segmentation processing, and obtaining 2D cardiovascular image slices after fine segmentation;

[0066] S6: stacking all the 2D cardiovascular image slices after fine segmentation to obtain a reconstructed 3D coronary artery image, and completing the segmentation and reconstruction of the 3D coronary artery;

[0067] The coarse segmentation model in the step S2 is specifically a 3D U-Net neural network model based on CNN;

[0068] The position feature map of the coronary artery in the step S3 is specifically a soft-feature map generated in a segmentation process by the trained coarse segmentation model.

[0069] In the step S4, the fused 3D cardiovascular image is segmented and combined in the Z-axis direction to obtain a plurality of 2D cardiovascular image combined slices, and the specific method is as follows:

[0070] The size of the fused 3D cardiovascular image is denoted as X×Y×Z, the fused 3D cardiovascular image is segmented in the Z-axis direction to obtain Z 2D cardiovascular image slices with the size of X×Y, and the i-th 2D cardiovascular image slice is denoted as I i (i=1, 2…Z), and the corresponding dilated label is denoted as L i ;

[0071] The adjacent n 2D cardiovascular image slices are combined to obtain a 2D cardiovascular image combined slice with the size of X×Y×n;

[0072] The fine segmentation model in the step S5 is specifically a 3D P2T neural network model based on a Transformer, which includes an encoder and a decoder connected in sequence.

[0073] The encoder includes a first embedding layer, a first pyramid pooling Transformer layer, a second embedding layer, a second pyramid pooling Transformer layer, a third embedding layer, a third pyramid pooling Transformer layer, a fourth embedding layer, and a fourth pyramid pooling Transformer layer connected in sequence.

[0074] The decoder includes a first upsampling layer, a first connection layer, a second upsampling layer, a second connection layer, a third upsampling layer, a third connection layer, a fourth upsampling layer, and a convolution layer connected in sequence.

[0075] The output end of the first pyramid pooling Transformer layer is further connected to the input end of the third connection layer, the output end of the second pyramid pooling Transformer layer is further connected to the input end of the second connection layer, and the output end of the third pyramid pooling Transformer layer is further connected to the input end of the first connection layer.

[0076] The coarse segmentation model and the fine segmentation model are both provided with a skip-connection.

[0077] In the specific implementation process, first, a 3D cardiovascular image is obtained, and a real label of the coronary artery is dilated by using a dilating operator to obtain a dilated label of the coronary artery.

[0078] The label inflation is to inflate the real label by using an inflation operator, so that the inflated label contains more voxels than the real label; in this embodiment, a spherical inflation is used during inflation, and the inflation coefficient is 7, so that the coronary artery will expand outward;

[0079] The inflated label of the coronary artery is used as the supervision information, and the 3D cardiovascular image is input into the trained coarse segmentation model to perform coarse segmentation to obtain a position feature map of the coronary artery;

[0080] The inflated label is used as the supervision information of the coarse segmentation model to guide the coarse segmentation model to capture as many coronary arteries as possible, so as to ensure that there is no blood vessel rupture and branch missing;

[0081] As shown in Figure 3 , the coarse segmentation model in this embodiment is specifically a 3D U-Net neural network model based on CNN. The 3D U-Net mainly consists of a compression path for capturing context information and an expansion path symmetrical to the compression path for realizing accurate positioning. In order to avoid the loss of shallow information, a skip-connection connection is used to fuse deep semantic information and shallow spatial information. This network can make full use of a small amount of images for end-to-end training, can segment each pixel point, obtain higher segmentation accuracy, and at the same time, this network structure is also conducive to better learning of deep image features;

[0082] The position feature map of the coronary artery is fused with the 3D cardiovascular image to obtain a fused 3D cardiovascular image;

[0083] The coarse-grained segmentation module produces a soft-feature map during inference, and then obtains a final coarse segmentation result according to the soft-feature map. The soft-feature map is fused with the original image to strengthen the pixels of the coronary artery and provide guidance for fine-grained segmentation. The interval including the coronary artery is found through the coarse segmentation result. Considering that some fine branches and tips may be missing, 5 is added before and after the interval, and the interval is a slice interval;

[0084] The initial 3D cardiovascular image is shown in Figure 4 (b), the position feature map of the coronary artery contains the position information of the coronary artery, as shown in Figure 4 (a), and the fusion effect is shown in Figure 4 (c);

[0085] Then, the fused 3D cardiovascular image is segmented and combined in the Z-axis direction to obtain a plurality of 2D cardiovascular image combination slices, specifically:

[0086] As Figure 5 shown, the size of the fused 3D coronary image is recorded as 512x512x275, and the fused 3D coronary image is segmented and sliced in the Z-axis direction to obtain Z 2D coronary image slices with a size of 512x512, and the i-th 2D coronary image slice is recorded as I i (i = 1, 2…275), and the corresponding dilated label is recorded as L i ;

[0087] The adjacent 3 2D coronary image slices are combined to obtain a plurality of 2D coronary image combined slices with a size of 512x512x3;

[0088] Then all the 2D coronary image combined slices are input into the trained fine segmentation model for fine segmentation processing to obtain the fine segmented 2D coronary image slices;

[0089] As Figure 6 shown, the fine segmentation model is a 3D P2T neural network model based on Transformer, which includes an encoder and a decoder connected in turn, wherein the encoder is composed of P2T modules, which can capture strong context information and long-range dependencies while reducing the number of parameters;

[0090] The encoder includes a first embedding layer, a first pyramid pooling Transformer layer, a second embedding layer, a second pyramid pooling Transformer layer, a third embedding layer, a third pyramid pooling Transformer layer, a fourth embedding layer and a fourth pyramid pooling Transformer layer connected in turn;

[0091] The decoder includes a first upsampling layer, a first connection layer, a second upsampling layer, a second connection layer, a third upsampling layer, a third connection layer, a fourth upsampling layer and a convolutional layer connected in turn;

[0092] The output end of the first pyramid pooling Transformer layer is also connected to the input end of the third connection layer, the output end of the second pyramid pooling Transformer layer is also connected to the input end of the second connection layer, and the output end of the third pyramid pooling Transformer layer is also connected to the input end of the first connection layer.

[0093] In addition, the network adds a skip-connection for deep semantic information and shallow spatial information fusion, which is beneficial to obtain global and local information, thereby refining the segmentation result;

[0094] Finally, all the fine segmented 2D coronary image slices are stacked along the Z-axis direction to obtain a reconstructed 3D coronary image, and the segmentation and reconstruction of the 3D coronary artery are completed.

[0095] The method is based on deep learning, and the two-stage coarse-to-fine 3D coronary artery reconstruction method can reduce the workload of clinical experts, and the reconstructed high-quality coronary artery can provide assistance for cardiovascular disease diagnosis; meanwhile, the two-stage method can effectively combine global and local information to gradually refine the 3D coronary artery; in addition, the method also uses an inflation label supervision strategy to ensure the integrity of the global structure of the coronary artery as much as possible, and solves the problems of fracture and branch missing caused by the complex structure of the coronary artery during single model reconstruction; in addition, the method also introduces a feature fusion strategy to emphasize the position of the coronary artery, and provides more comprehensive and effective guidance for the second stage of fine segmentation; in the fine segmentation stage, a multi-slice model is used to ensure that the model learns the details of the coronary artery while not losing the connectivity of the blood vessels, and the reconstruction effect is better.

[0096] Embodiment 3

[0097] As shown in Figure 7 The embodiment provides a two-stage 3D coronary artery segmentation and reconstruction system, which applies the two-stage 3D coronary artery segmentation and reconstruction method described in embodiment 1 or 2, and includes:

[0098] The data acquisition unit 301 is used to acquire the real label of the coronary artery, and use the inflation operator to inflate to acquire the inflation label of the coronary artery;

[0099] The coarse segmentation unit 302 is used to acquire the 3D cardiovascular image and input it into the trained coarse segmentation model for coarse segmentation with the inflation label of the coronary artery as the supervision information, to acquire the position feature map of the coronary artery;

[0100] The feature fusion unit 303 is used to perform feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image to acquire the fused 3D cardiovascular image;

[0101] The segmentation and slicing unit 304 is used to perform segmentation and slicing on the fused 3D cardiovascular image and combine the slices to acquire a plurality of 2D cardiovascular image combination slices;

[0102] The fine segmentation unit 305 is used to input all the 2D cardiovascular image combination slices into the trained fine segmentation model for fine segmentation processing to acquire the fine segmented 2D cardiovascular image slices;

[0103] The reconstruction unit 306 is used to stack all the fine segmented 2D cardiovascular image slices to acquire the reconstructed 3D coronary artery image, and complete the segmentation and reconstruction of the 3D coronary artery.

[0104] In the implementation process, first, the data acquisition unit 301 acquires the real label of the coronary artery, and the real label of the coronary artery is dilated by using a dilating operator to acquire the dilated label of the coronary artery; the rough segmentation unit 302 takes the dilated label of the coronary artery as supervision information, acquires the 3D cardiovascular image and inputs the acquired 3D cardiovascular image into the trained rough segmentation model to perform rough segmentation, and acquires the position feature map of the coronary artery; the feature fusion unit 303 performs feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image to acquire the fused 3D cardiovascular image; the segmentation slice unit 304 performs segmentation and slice combination on the fused 3D cardiovascular image in the Z-axis direction to acquire a plurality of 2D cardiovascular image combination slices; the fine segmentation unit 305 inputs all the 2D cardiovascular image combination slices into the trained fine segmentation model to perform fine segmentation processing, and acquires the 2D cardiovascular image slice after fine segmentation; finally, the reconstruction unit 306 stacks all the 2D cardiovascular image slices after fine segmentation along the Z-axis direction to acquire the reconstructed 3D coronary artery image, and completes the segmentation and reconstruction of the 3D coronary artery;

[0105] The two-stage coarse-to-fine 3D coronary artery reconstruction system based on deep learning can reduce the workload of clinical experts, and the reconstructed high-quality coronary artery can provide assistance for cardiovascular disease diagnosis; meanwhile, the two-stage method is used to reconstruct the 3D coronary artery, which can effectively combine the global and local information and gradually refine the 3D coronary artery; in addition, the inflation label supervision strategy is used to ensure the integrity of the global structure of the coronary artery as much as possible, and the problems of fracture and branch loss caused by the complex structure of the coronary artery during single model reconstruction are solved; in addition, the feature fusion strategy is introduced to emphasize the position of the coronary artery, and more comprehensive and effective guidance is provided for the fine-grained segmentation in the second stage; the multi-slice model is used in the fine segmentation stage, which can ensure that the model learns the details of the coronary artery without losing the connectivity of the blood vessels, and the reconstruction effect is better.

[0106] The same or similar reference signs correspond to the same or similar components;

[0107] The terms describing the positional relationship in the drawings are only used for illustrative description, and should not be understood as a limitation on the patent;

[0108] Obviously, the above embodiments of the application are only examples for clearly illustrating the application, and are not intended to limit the implementation manner of the application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the implementation manners are not required or can not be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the application should be included in the protection scope of the claims of the application.

Claims

1. A two-stage 3D coronary artery segmentation reconstruction method, characterized in that, The method comprises the following steps: S1: obtaining a real label of the coronary artery and performing inflation using an inflation operator to obtain an inflation label of the coronary artery; S2: using the inflation label of the coronary artery as supervision information, obtaining a 3D cardiovascular image and inputting the 3D cardiovascular image into a trained coarse segmentation model to perform coarse segmentation, and obtaining a position feature map of the coronary artery; The coarse segmentation model is specifically a 3D U-Net neural network model based on CNN; S3: performing feature fusion on the position feature map of the coronary artery and the 3D cardiovascular image to obtain a fused 3D cardiovascular image; The position feature map of the coronary artery is specifically a soft-feature map generated by the trained coarse segmentation model during segmentation; and a final coarse segmentation result is obtained according to the soft-feature map; S4: performing segmentation slicing and slice combination on the fused 3D cardiovascular image to obtain a plurality of 2D cardiovascular image combination slices; S5: inputting all the 2D cardiovascular image combination slices into a trained fine segmentation model to perform fine segmentation processing, and obtaining 2D cardiovascular image slices after fine segmentation; The fine segmentation model is specifically a 3D P2T neural network model based on Transformer, and comprises an encoder and a decoder connected in sequence; S6: stacking all the 2D cardiovascular image slices after fine segmentation to obtain a reconstructed 3D coronary artery image, and completing segmentation and reconstruction of the 3D coronary artery.

2. The two-stage 3D coronary artery segmentation reconstruction method according to claim 1, wherein, In the step S1, an inflation coefficient is set, and the real label of the coronary artery is subjected to spherical inflation using the inflation operator to obtain the inflation label of the coronary artery.

3. The two-stage 3D coronary artery segmentation reconstruction method according to claim 1, wherein, In the step S4, the size of the fused 3D cardiovascular image is denoted as X×Y×Z, the fused 3D cardiovascular image is sliced in the Z-axis direction, Z 2D cardiovascular image slices with the size of X×Y are obtained, and the i-th 2D cardiovascular image slice is denoted as The corresponding dilated label is denoted as ; Combining adjacent n 2D cardiovascular image slices to obtain a slice group 2D cardiovascular image combined slice with a size of X×Y×n.

4. The two-stage 3D coronary artery segmentation reconstruction method according to claim 3, characterized in that, The encoder comprises a first embedding layer, a first pyramid pooling Transformer layer, a second embedding layer, a second pyramid pooling Transformer layer, a third embedding layer, a third pyramid pooling Transformer layer, a fourth embedding layer and a fourth pyramid pooling Transformer layer connected in sequence; The decoder comprises a first upsampling layer, a first connection layer, a second upsampling layer, a second connection layer, a third upsampling layer, a third connection layer, a fourth upsampling layer and a convolution layer connected in sequence; The output end of the first pyramid pooling Transformer layer is further connected to the input end of the third connection layer, the output end of the second pyramid pooling Transformer layer is further connected to the input end of the second connection layer, and the output end of the third pyramid pooling Transformer layer is further connected to the input end of the first connection layer.

5. The two-stage 3D coronary artery segmentation reconstruction method according to claim 4, wherein, The coarse segmentation model and the fine segmentation model both have a skip-connection.

6. A two-stage 3D coronary artery segmentation reconstruction system, applying the two-stage 3D coronary artery segmentation reconstruction method of any one of claims 1-5, characterized in that, The method comprises the following steps: A data acquisition unit is configured to obtain a real label of the coronary artery and perform inflation using an inflation operator to obtain an inflation label of the coronary artery; A coarse segmentation unit is configured to use the inflation label of the coronary artery as supervision information, obtain a 3D cardiovascular image and input the 3D cardiovascular image into a trained coarse segmentation model to perform coarse segmentation, and obtain a position feature map of the coronary artery; The feature fusion unit is configured to fuse the position feature map of the coronary artery with the 3D cardiovascular image to obtain a fused 3D cardiovascular image. The segmentation and slicing unit is configured to segment and slice the fused 3D cardiovascular image to obtain a plurality of 2D cardiovascular image combination slices. The fine segmentation unit is configured to input all the 2D cardiovascular image combination slices into a trained fine segmentation model to perform fine segmentation processing to obtain fine segmented 2D cardiovascular image slices. The reconstruction unit is configured to stack all the fine segmented 2D cardiovascular image slices to obtain a reconstructed 3D coronary artery image, thereby completing segmentation and reconstruction of the 3D coronary artery.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the steps in the method of any one of claims 1-5.

8. An electronic device, comprising: The computer program, when executed by a processor, implements the steps in the method of any one of claims 1-5. The computer program, when executed by a processor, implements the steps in the method of any one of claims 1-5.

Images