A coronary artery image segmentation method and system based on MUNet

Abstract

The present invention provides a method and system for coronary artery image segmentation based on MUNet. The method comprises: obtaining a coronary artery image dataset and preprocessing it; constructing a MUNet neural network model based on self-attention and multi-view feature attention; training the MUNet neural network model using the preprocessed coronary artery image dataset; obtaining a coronary artery image to be segmented and inputting it into the trained MUNet neural network model for segmentation to obtain a preliminary segmentation result; and smoothing the preliminary segmentation result using an image morphology algorithm to obtain a final segmentation result. The present invention can more effectively extract global and local information of the coronary artery image, achieve more accurate coronary artery image segmentation, significantly improve segmentation accuracy, and also improve the generalization ability of the model, providing a more reliable reference for subsequent diagnosis and treatment.

Description

Technical Field

[0001] The present invention relates to the technical field of CT vascular medical image segmentation, and more particularly to a coronary artery image segmentation method and system based on MUNet. Background Art

[0002] Currently, traditional and deep learning methods are the primary approaches for segmenting coronary artery 3D CTA images. Among these traditional methods, techniques based on image segmentation and level sets are relatively common. While these methods can automatically extract coronary arteries, they require manual seed point setting or vessel region demarcation, leading to errors and discontinuities in localized regions.

[0003] With the development of deep learning technology, deep learning-based segmentation methods have made significant progress in coronary artery 3D CTA image segmentation. These methods, through architectures such as convolutional neural networks (CNNs), can automatically learn features from data, thereby achieving more accurate segmentation.

[0004] For example, the existing patent CN111739034A discloses a coronary artery region segmentation method and system based on an improved 3D Unet. This method also uses a deep learning method to perform coronary artery segmentation. However, this method optimizes the original network model by using residual connections and deepening the network depth, which may cause the training model to be too complex and produce the risk of overfitting. At the same time, it is lacking in the acquisition of global information. Another existing patent CN111476796A discloses a semi-supervised coronary artery segmentation system and segmentation method combining multiple networks. However, this method constructs a new 2D data training set by slicing the original three-dimensional data. This method will lose the connection between features to a certain extent, which will have a certain impact on the segmentation results.

[0005] In addition, in existing deep learning-based segmentation methods, the use of 2D slice segmentation may lead to the loss of key three-dimensional spatial information, especially in curved and intersecting vascular areas; and the use of three-dimensional convolutional neural networks introduces a lot of computational complexity, limiting its feasibility in practical applications.

[0006] Therefore, existing coronary artery segmentation technologies face challenges such as low accuracy of spatial continuity and three-dimensional morphological information, poor segmentation accuracy, and high computational complexity. Summary of the Invention

[0007] To overcome the shortcomings of the above-mentioned prior art, such as low accuracy of spatial continuity and three-dimensional morphological information, poor segmentation accuracy, and high computational complexity, the present invention provides a coronary artery image segmentation method and system based on MUNet. By constructing a MUNet network model based on self-attention and multi-view feature attention, the model is used to perform global and local segmentation of coronary artery images, significantly improving its segmentation accuracy.

[0008] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0009] A coronary artery image segmentation method based on MUNet includes the following steps:

[0010] S1: Acquire coronary artery image dataset and perform preprocessing;

[0011] The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery;

[0012] S2: Constructing MUNet neural network model based on self-attention and multi-view feature attention;

[0013] The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0014] The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder;

[0015] S3: Inputting the preprocessed coronary artery image dataset into the MUNet neural network model for training to obtain a trained MUNet neural network model;

[0016] S4: Obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results;

[0017] S5: Use image morphology algorithm to smooth the preliminary segmentation result to obtain the final segmentation result and complete the segmentation of the coronary artery.

[0018] Preferably, the preprocessing in step S1 includes any one or more of scale transformation, cropping, random flipping, random rotation, affine transformation and image value normalization.

[0019] Preferably, in step S2, the feature coarse extraction module includes: a 3D convolution block, a Convnext feature extraction block 1 and a Convnext feature extraction block 2 connected in sequence;

[0020] The structure of the 3D convolutional block includes: a 4×4×4 convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer connected in sequence;

[0021] The structure of each Convnext feature extraction block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext feature extraction block also form a residual sum connection through a bypass.

[0022] Preferably, in step S2, the structure of the multi-view feature attention module includes: branches 1, 2, and 3 arranged in parallel, wherein branches 1, 2, and 3 correspond to the coronal view, the sagittal view, and the axial view, respectively;

[0023] The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence;

[0024] The structure of each convolution block is the same, and includes: a 4×4×4 convolution layer, a batch normalization BN layer, a ReLU activation layer, and a 1×1×1 convolution layer connected in sequence;

[0025] The structure of each hybrid attention block is the same, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the spatial domain attention branch includes: a 1×1×1 convolution layer, a 3×3×3 convolution layer, a 3×3×3 convolution layer, a 1×1×1 convolution layer and a batch normalization BN layer connected in sequence; the channel domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence;

[0026] The outputs of the spatial domain attention branch and the channel domain attention branch are weighted added as the output of the hybrid attention block;

[0027] The outputs of branches 1, 2 and 3 are concatenated and fused as the output of the multi-view feature attention module.

[0028] Preferably, in step S2, a transition layer is provided before each of the multi-view feature fusion modules, and the transition layer includes: a 1×1×1 convolution layer, a batch normalization (BN) layer, a ReLU activation layer, a maximum pooling downsampling layer, and a Convnext block connected in sequence;

[0029] The Convnext block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext block also form a residual sum connection through a bypass.

[0030] Preferably, in step S2, the Transformer encoder comprises: an embedding layer and a plurality of Transformer blocks connected in sequence;

[0031] Each of the Transformer blocks has the same structure, and includes the following connected in sequence: layer normalization LN layer 1, multi-head self-attention layer, layer normalization LN layer 2 and multi-layer perceptron layer; the input of layer normalization LN layer 1 also forms a residual sum connection with the output of the multi-head self-attention layer; the input of layer normalization LN layer 2 also forms a residual sum connection with the output of the multi-layer perceptron layer.

[0032] Preferably, in step S2, the Transformer encoder includes four Transformer blocks, and the outputs of the four Transformer blocks are respectively recorded as T1, T2, T3 and T4;

[0033] The number of the multi-view feature attention modules is 3; the output of the feature coarse extraction module is recorded as S1, and the outputs of the three multi-view feature attention modules are recorded as M1, M2 and M3 respectively;

[0034] In the feature fusion network, T1 and S1 are fused and output as F1; T2 and M1 are fused and output as F2; ​​T3 and M2 are fused and output as F3; T4 and M3 are fused and output as F4;

[0035] In the decoder, F4 is fused with F3 after upsampling, and the output is C3; C3 is fused with F2 after upsampling, and the output is C2; C2 is fused with F1 after upsampling, and the output is C1; C1 is upsampled and outputs the segmentation prediction result of the coronary artery image as the output of the MUNet neural network model.

[0036] Preferably, in step S3, the loss function used in the MUNet neural network model training is a loss function L based on the similarity Dice coefficient. dice , specifically:

[0037]

[0038] Among them, P represents the prediction result of the MUNet neural network model, and R represents the true label of the preprocessed coronary artery image.

[0039] Preferably, step S5 includes:

[0040] S5.1: Use morphological opening to remove noise and eliminate errors or noise that may be introduced during the image segmentation process; the morphological opening operation is erosion followed by dilation;

[0041] The erosion operation is used to remove a layer of pixels at the edge of the image, thereby reducing the image. The mathematical formula is expressed as:

[0042]

[0043] Where S1 represents the image after corrosion, B1 represents the corroded structural element, A1 represents the original image before corrosion, and z represents the position of the corroded pixel in the original image. Only when B1 is translated to position z and B1 completely belongs to A1 at this time, does z belong to the corroded image S1 at this time.

[0044] The dilation operation is used to increase the size of an image by adding a layer of pixels to the edge of the image. The mathematical formula is expressed as:

[0045]

[0046] Where S2 represents the expanded image, B2 represents the expanded structuring element, A2 represents the original image before expansion, and z' represents the position of the expanded pixel in the original image. Only when B2 is translated to position z' and B2 has a common point with A2 at this time, does z' belong to the expanded image S2.

[0047] S5.2: Use a morphological closing operation to connect adjacent vascular breaks to enhance vascular continuity; the morphological closing operation is to first dilate and then erode;

[0048] S5.3: Repeat steps S5.1 to S5.2 several times, further smoothing the edges of the blood vessels by continuously applying morphological opening and closing operations to obtain the final segmentation result;

[0049] S5.4: Map the final segmentation result to the medical image space, store the mapped data in nii format, and complete the segmentation of the coronary artery.

[0050] The present invention also provides a coronary artery image segmentation system based on MUNet, which applies the above-mentioned coronary artery image segmentation method based on MUNet, including:

[0051] Preprocessing unit: used to obtain coronary artery image dataset and perform preprocessing;

[0052] The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery;

[0053] Model building unit: used to build the MUNet neural network model based on self-attention and multi-view feature attention;

[0054] The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0055] The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder;

[0056] Model training unit: used to input the preprocessed coronary artery image dataset into the MUNet neural network model for training, and obtain the trained MUNet neural network model;

[0057] Preliminary segmentation unit: used to obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results;

[0058] Post-processing unit: used to smooth the preliminary segmentation results using image morphology algorithms to obtain the final segmentation results and complete the segmentation of the coronary arteries.

[0059] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0060] The present invention provides a coronary artery image segmentation method and system based on MUNet. First, a coronary artery image dataset is obtained and preprocessed. Then, a MUNet neural network model is constructed based on self-attention and multi-view feature attention. Then, the preprocessed coronary artery image dataset is input into the MUNet neural network model for training to obtain a trained MUNet neural network model. Then, a coronary artery image to be segmented is obtained and input into the trained MUNet neural network model for segmentation to obtain a preliminary segmentation result. Finally, an image morphology algorithm is used to smooth the preliminary segmentation result to obtain a final segmentation result, thereby completing the segmentation of the coronary artery.

[0061] The present invention constructs a MUNet neural network model based on self-attention and multi-view feature attention, utilizes the Transformer's self-attention mechanism to extract long-range features, obtains more global information about coronary artery images, and utilizes a multi-view feature attention extraction module to extract feature information from multiple viewpoints of coronary artery images, focusing on feature information in details. Compared with traditional coronary artery image segmentation methods, the present invention can more effectively extract global and local information of coronary artery images, achieve more accurate coronary artery image segmentation, significantly improve segmentation accuracy, and also enhance the generalization ability of the model, providing a more reliable reference for subsequent diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a coronary artery image segmentation method based on MUNet provided in Example 1.

[0063] Figure 2 This is a structural diagram of the MUNet neural network model provided in Example 2.

[0064] Figure 3 This is the Convnext feature extraction block structure diagram provided in Example 2.

[0065] Figure 4 This is a structural diagram of the multi-view feature attention module provided in Example 2.

[0066] Figure 5 This is a structural diagram of the hybrid attention block provided in Example 2.

[0067] Figure 6 This is a structural diagram of a coronary artery image segmentation system based on MUNet provided in Example 3. DETAILED DESCRIPTION

[0068] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0069] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0070] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0071] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0072] Example 1

[0073] like Figure 1As shown, this embodiment provides a coronary artery image segmentation method based on MUNet, comprising the following steps:

[0074] S1: Acquire coronary artery image dataset and perform preprocessing;

[0075] The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery;

[0076] S2: Constructing MUNet neural network model based on self-attention and multi-view feature attention;

[0077] The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0078] The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder;

[0079] S3: Inputting the preprocessed coronary artery image dataset into the MUNet neural network model for training to obtain a trained MUNet neural network model;

[0080] S4: Obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results;

[0081] S5: Use image morphology algorithm to smooth the preliminary segmentation result to obtain the final segmentation result and complete the segmentation of the coronary artery.

[0082] In the specific implementation process, firstly, a coronary artery image dataset is acquired and preprocessed;

[0083] Then, the MUNet neural network model was constructed based on self-attention and multi-view feature attention;

[0084] The MUNet neural network model includes: Transformer encoder, feature extraction network, feature fusion network and decoder; the feature extraction network includes: feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0085] The Transformer encoder and feature extraction network are set up in parallel. The outputs of the Transformer encoder and feature extraction network are connected to the input of the feature fusion network respectively, and the output of the feature fusion network is connected to the decoder.

[0086] Then, the preprocessed coronary artery image dataset is input into the MUNet neural network model for training to obtain a trained MUNet neural network model;

[0087] Then, the coronary artery image to be segmented is obtained and input into the trained MUNet neural network model for segmentation to obtain the preliminary segmentation results;

[0088] Finally, the image morphology algorithm is used to smooth the preliminary segmentation results to obtain the final segmentation results and complete the segmentation of the coronary arteries;

[0089] This method constructs a MUNet network model based on self-attention and multi-view feature attention, and uses this model to perform global and local segmentation of coronary artery images, significantly improving its segmentation accuracy.

[0090] Example 2

[0091] This embodiment provides a coronary artery image segmentation method based on MUNet, comprising the following steps:

[0092] S1: Acquire coronary artery image dataset and perform preprocessing;

[0093] The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery;

[0094] S2: Constructing MUNet neural network model based on self-attention and multi-view feature attention;

[0095] The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0096] The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder;

[0097] S3: Inputting the preprocessed coronary artery image dataset into the MUNet neural network model for training to obtain a trained MUNet neural network model;

[0098] S4: Obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results;

[0099] S5: Use image morphology algorithm to smooth the preliminary segmentation results to obtain the final segmentation results and complete the segmentation of the coronary artery;

[0100] The preprocessing in step S1 includes: any one or more of scale transformation, cropping, random flipping, random rotation, affine transformation and image value normalization;

[0101] In step S2, the feature coarse extraction module includes: a 3D convolution block, a Convnext feature extraction block 1 and a Convnext feature extraction block 2 connected in sequence;

[0102] The structure of the 3D convolutional block includes: a 4×4×4 convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer connected in sequence;

[0103] The structure of each Convnext feature extraction block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer, and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext feature extraction block also form a residual sum connection through a bypass;

[0104] In step S2, the structure of the multi-view feature attention module includes: branches 1, 2, and 3 arranged in parallel, where branches 1, 2, and 3 correspond to the coronal view, the sagittal view, and the axial view, respectively;

[0105] The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence;

[0106] The structure of each convolution block is the same, and includes: a 4×4×4 convolution layer, a batch normalization BN layer, a ReLU activation layer, and a 1×1×1 convolution layer connected in sequence;

[0107] The structure of each hybrid attention block is the same, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the spatial domain attention branch includes: a 1×1×1 convolution layer, a 3×3×3 convolution layer, a 3×3×3 convolution layer, a 1×1×1 convolution layer and a batch normalization BN layer connected in sequence; the channel domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence;

[0108] The outputs of the spatial domain attention branch and the channel domain attention branch are weighted added as the output of the hybrid attention block;

[0109] The outputs of branches 1, 2 and 3 are concatenated and fused as the output of the multi-view feature attention module;

[0110] In step S2, a transition layer is provided before each of the multi-view feature fusion modules, and the transition layer includes: a 1×1×1 convolution layer, a batch normalization (BN) layer, a ReLU activation layer, a maximum pooling downsampling layer, and a Convnext block connected in sequence;

[0111] The Convnext block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer, and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext block also form a residual sum connection through a bypass;

[0112] In step S2, the Transformer encoder includes: an embedding layer and a plurality of Transformer blocks connected in sequence;

[0113] Each of the Transformer blocks has the same structure, including: a layer-normalized LN layer 1, a multi-head self-attention layer, a layer-normalized LN layer 2, and a multi-layer perceptron layer connected in sequence; the input of the layer-normalized LN layer 1 also forms a residual sum connection with the output of the multi-head self-attention layer; the input of the layer-normalized LN layer 2 also forms a residual sum connection with the output of the multi-layer perceptron layer;

[0114] In step S2, the Transformer encoder includes four Transformer blocks, and the outputs of the four Transformer blocks are respectively recorded as T1, T2, T3 and T4;

[0115] The number of the multi-view feature attention modules is 3; the output of the feature coarse extraction module is recorded as S1, and the outputs of the three multi-view feature attention modules are recorded as M1, M2 and M3 respectively;

[0116] In the feature fusion network, T1 and S1 are fused and output as F1; T2 and M1 are fused and output as F2; ​​T3 and M2 are fused and output as F3; T4 and M3 are fused and output as F4;

[0117] In the decoder, F4 is fused with F3 after upsampling, and the output is C3; C3 is fused with F2 after upsampling, and the output is C2; C2 is fused with F1 after upsampling, and the output is C1; C1 is upsampling and outputs the segmentation prediction result of the coronary artery image as the output of the MUNet neural network model;

[0118] In step S3, the loss function used in the MUNet neural network model training is the loss function L based on the similarity Dice coefficient. dice , specifically:

[0119]

[0120] Where P represents the prediction result of the MUNet neural network model, and R represents the true label of the preprocessed coronary artery image;

[0121] The step S5 comprises:

[0122] S5.1: Use morphological opening to remove noise and eliminate errors or noise that may be introduced during the image segmentation process; the morphological opening operation is erosion followed by dilation;

[0123] The erosion operation is used to remove a layer of pixels at the edge of the image, thereby reducing the image. The mathematical formula is expressed as:

[0124]

[0125] Where S1 represents the image after corrosion, B1 represents the corroded structural element, A1 represents the original image before corrosion, and z represents the position of the corroded pixel in the original image. Only when B1 is translated to position z and B1 completely belongs to A1 at this time, does z belong to the corroded image S1 at this time.

[0126] The dilation operation is used to increase the size of an image by adding a layer of pixels to the edge of the image. The mathematical formula is expressed as:

[0127]

[0128] Where S2 represents the expanded image, B2 represents the expanded structuring element, A2 represents the original image before expansion, and z' represents the position of the expanded pixel in the original image. Only when B2 is translated to position z' and B2 has a common point with A2 at this time, does z' belong to the expanded image S2.

[0129] S5.2: Use a morphological closing operation to connect adjacent vascular breaks to enhance vascular continuity; the morphological closing operation is to first dilate and then erode;

[0130] S5.3: Repeat steps S5.1 to S5.2 several times, further smoothing the edges of the blood vessels by continuously applying morphological opening and closing operations to obtain the final segmentation result;

[0131] S5.4: Map the final segmentation result to the medical image space, store the mapped data in nii format, and complete the segmentation of the coronary artery.

[0132] In the specific implementation process, a coronary artery image dataset is first obtained and preprocessed. The preprocessing includes any one or more of scale transformation, cropping, random flipping, random rotation, affine transformation, and image value normalization. The preprocessed dataset is then divided into a training set, a validation set, and a test set to enable the network model to have a better learning effect.

[0133] Then, the MUNet neural network model was constructed based on self-attention and multi-view feature attention;

[0134] like Figure 2 As shown in the figure, the MUNet neural network model includes: Transformer encoder, feature extraction network, feature fusion network and decoder; the feature extraction network includes: feature coarse extraction module and three multi-view feature attention modules connected in sequence;

[0135] The Transformer encoder and feature extraction network are set up in parallel. The outputs of the Transformer encoder and feature extraction network are connected to the input of the feature fusion network respectively, and the output of the feature fusion network is connected to the decoder.

[0136] exist Figure 2 In the figure, Stem is a coarse feature extraction module, which includes the following connected in sequence: 4×4×4 convolution layer, batch normalization BN layer, ReLU activation layer, Convnext feature extraction block 1 and Convnext feature extraction block 2;

[0137] like Figure 3 As shown, the structure of each Convnext feature extraction block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer, and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext feature extraction block also form a residual sum connection through a bypass;

[0138] like Figure 4 As shown in Figure 1, the structure of the Multi-plane Hybrid Attention (MPHA) module includes: parallel branches 1, 2, and 3, where branches 1, 2, and 3 correspond to the coronal, sagittal, and axial views, respectively. The input image is transposed to obtain its coronal, sagittal, and axial views.

[0139] The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence;

[0140] The structure of each convolution block is the same, and includes: a 4×4×4 convolution layer, a batch normalization BN layer, a ReLU activation layer, and a 1×1×1 convolution layer connected in sequence;

[0141] like Figure 5 As shown, the structure of each hybrid attention block is the same, and both include: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the spatial domain attention branch includes: a 1×1×1 convolution layer, a 3×3×3 convolution layer, a 3×3×3 convolution layer, a 1×1×1 convolution layer and a batch normalization BN layer connected in sequence; the channel domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence;

[0142] The outputs of the spatial domain attention branch and the channel domain attention branch are weighted added as the output of the hybrid attention block;

[0143] The outputs of branches 1, 2 and 3 are concatenated and fused as the output of the multi-view feature attention module;

[0144] exist Figure 2 In the example, a transition layer is provided before each of the multi-view feature fusion modules. The transition layer includes: a 1×1×1 convolutional layer, a batch normalization (BN) layer, a ReLU activation layer, a maximum pooling downsampling layer, and a Convnext block connected in sequence; the maximum pooling downsampling layer has a dimension of 2×2×2;

[0145] The Convnext block is Figure 3 The structure is the same as in, including: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer, and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext block also form a residual sum connection through a bypass;

[0146] In this embodiment, the Transformer encoder includes: an embedding layer, and several Transformer blocks connected in sequence;

[0147] Each of the Transformer blocks has the same structure, including: a layer-normalized LN layer 1, a multi-head self-attention layer, a layer-normalized LN layer 2, and a multi-layer perceptron layer connected in sequence; the input of the layer-normalized LN layer 1 also forms a residual sum connection with the output of the multi-head self-attention layer; the input of the layer-normalized LN layer 2 also forms a residual sum connection with the output of the multi-layer perceptron layer;

[0148] In this embodiment, the Transformer encoder includes four Transformer blocks, and the outputs of the four Transformer blocks are denoted as T1, T2, T3, and T4 respectively;

[0149] The output of the feature coarse extraction module is denoted as S1, and the outputs of the three multi-view feature attention modules are denoted as M1, M2 and M3 respectively;

[0150] In the feature fusion network, T1 and S1 are fused and output as F1; T2 and M1 are fused and output as F2; ​​T3 and M2 are fused and output as F3; T4 and M3 are fused and output as F4;

[0151] In the decoder, F4 is fused with F3 after upsampling, and the output is C3; C3 is fused with F2 after upsampling, and the output is C2; C2 is fused with F1 after upsampling, and the output is C1; C1 is upsampling and outputs the segmentation prediction result of the coronary artery image as the output of the MUNet neural network model;

[0152] After building the MUNet neural network model, the training set is used to train the MUNet neural network model, and the loss function L based on the similarity Dice coefficient is used. dice , specifically:

[0153]

[0154] Where P represents the prediction result of the MUNet neural network model, and R represents the true label of the preprocessed coronary artery image;

[0155] The validation set is then used to verify the model performance, and the loss function L is also used. dice For reference, compare each verification result with the previous round of verification results, and automatically save the network model with the best effect in the set epoch;

[0156] Finally, the performance of the model is tested using the test set, and the trained MUNet neural network model is finally obtained;

[0157] Then, the coronary artery image to be segmented is obtained and input into the trained MUNet neural network model for segmentation to obtain the preliminary segmentation results;

[0158] Finally, the image morphological algorithm is used to smooth the preliminary segmentation results, which includes the following steps:

[0159] S5.1: Use morphological opening to remove noise and eliminate errors or noise that may be introduced during the image segmentation process; the morphological opening operation is erosion followed by dilation;

[0160] The erosion operation is used to remove a layer of pixels at the edge of the image, thereby reducing the image. The mathematical formula is expressed as:

[0161]

[0162] Where S1 represents the image after corrosion, B1 represents the corroded structural element, A1 represents the original image before corrosion, and z represents the position of the corroded pixel in the original image. Only when B1 is translated to position z and B1 completely belongs to A1 at this time, does z belong to the corroded image S1 at this time.

[0163] The dilation operation is used to increase the size of an image by adding a layer of pixels to the edge of the image. The mathematical formula is expressed as:

[0164]

[0165] Where S2 represents the expanded image, B2 represents the expanded structuring element, A2 represents the original image before expansion, and z' represents the position of the expanded pixel in the original image. Only when B2 is translated to position z' and B2 has a common point with A2 at this time, does z' belong to the expanded image S2.

[0166] S5.2: Use a morphological closing operation to connect adjacent vascular breaks to enhance vascular continuity; the morphological closing operation is to first dilate and then erode;

[0167] S5.3: Repeat steps S5.1 to S5.2 several times, further smoothing the edges of the blood vessels by continuously applying morphological opening and closing operations to obtain the final segmentation result;

[0168] S5.4: Map the final segmentation result to the medical image space and store the mapped data in nii format to complete the segmentation of the coronary artery;

[0169] To verify the effectiveness of this method, this example also provides a comparative experiment. Specifically, the 480 collected original images were divided into 400 and 80 images. The 400 images were trained using a five-fold method and served as the training and validation sets for model training. The remaining 80 images were used as a test set to test the best model. As shown in Table 1, the coronary artery image segmentation results based on self-attention and multi-view feature attention far outperformed other segmentation models, and the average test results for each fold and the final test were higher than those of existing models.

[0170] Table 1 Dice test index table under the 5-fold training method

[0171] Name Fold_1(%) Fold_2(%) Fold_3(%) Fold_4(%) Fold_5(%) ACC (%) UNETR 76.32±2.87 77.87±2.89 78.07±2.63 76.28±2.72 77.71±3.11 77.29±2.84 UXNET 80.48±2.61 80.45±2.53 80.54±2.75 80.45±2.75 80.45±2.77 80.53±2.81 CANet 80.37±2.84 80.16±2.88 80.25±3.13 80.02±2.88 79.93±2.95 80.15±2.94 CSANet 80.45±2.62 80.31±3.09 80.54±2.84 80.75±2.54 80.58±3.24 80.53±2.87 MUNet 81.79±2.55 82.02±2.92 81.61±2.78 81.68±2.68 81.59±2.78 81.74±2.74

[0172] Therefore, compared with traditional segmentation models, this method proposes a multi-view feature attention module based on the fact that 3D medical images have different texture features when observed from different perspectives. Secondly, a MUNet network model is constructed based on self-attention and multi-view feature attention, which achieves higher segmentation accuracy and efficiency for coronary artery image segmentation. In addition, a post-processing module eliminates small errors or noise that may be introduced by coronary artery image segmentation, connects adjacent broken vessels, and enhances vascular continuity, further improving the segmentation effect and providing assistance for subsequent clinical diagnosis and treatment.

[0173] This method constructs a MUNet network model based on self-attention and multi-view feature attention, and uses this model to perform global and local segmentation of coronary artery images, significantly improving its segmentation accuracy.

[0174] Example 3

[0175] like Figure 6 As shown, this embodiment provides a coronary artery image segmentation system based on MUNet, applying the coronary artery image segmentation method based on MUNet described in Example 1 or 2, including:

[0176] Preprocessing unit 301: used to obtain a coronary artery image dataset and perform preprocessing;

[0177] The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery;

[0178] Model construction unit 302: used to construct a MUNet neural network model based on self-attention and multi-view feature attention;

[0179] The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0180] The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder;

[0181] Model training unit 303: used to input the pre-processed coronary artery image dataset into the MUNet neural network model for training, and obtain a trained MUNet neural network model;

[0182] The preliminary segmentation unit 304 is used to obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results;

[0183] The post-processing unit 305 is used to smooth the preliminary segmentation result using an image morphology algorithm to obtain a final segmentation result and complete the segmentation of the coronary arteries.

[0184] In a specific implementation process, first, the preprocessing unit 301 obtains a coronary artery image dataset and performs preprocessing;

[0185] Then the model construction unit 302 constructs a MUNet neural network model based on self-attention and multi-view feature attention;

[0186] The MUNet neural network model includes: Transformer encoder, feature extraction network, feature fusion network and decoder; the feature extraction network includes: feature coarse extraction module and several multi-view feature attention modules connected in sequence;

[0187] The Transformer encoder and feature extraction network are set up in parallel. The outputs of the Transformer encoder and feature extraction network are connected to the input of the feature fusion network respectively, and the output of the feature fusion network is connected to the decoder.

[0188] Then the model training unit 303 inputs the pre-processed coronary artery image dataset into the MUNet neural network model for training to obtain a trained MUNet neural network model;

[0189] Then, the preliminary segmentation unit 304 obtains the coronary artery image to be segmented and inputs it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results;

[0190] Finally, the post-processing unit 305 uses an image morphology algorithm to smooth the preliminary segmentation result to obtain the final segmentation result and complete the segmentation of the coronary artery.

[0191] This system constructs a MUNet network model based on self-attention and multi-view feature attention, and uses this model to perform global and local segmentation of coronary artery images, significantly improving its segmentation accuracy.

[0192] The same or similar reference numerals correspond to the same or similar components;

[0193] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0194] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A coronary artery image segmentation method based on MUNet, characterized in that: The following steps are involved: S1: Obtain a coronary artery image dataset and perform preprocessing; The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery; S2: Constructing MUNet neural network model based on self-attention and multi-view feature attention; The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence; The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder; The structure of the multi-view feature attention module includes: branches 1, 2 and 3 arranged in parallel, wherein branches 1, 2 and 3 correspond to the coronal view, the sagittal view and the axial view respectively; The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence; Each of the hybrid attention blocks has the same structure, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the outputs of the spatial domain attention branch and the channel domain attention branch are weightedly added as the output of the hybrid attention block; The outputs of branches 1, 2 and 3 are concatenated and fused as the output of the multi-view feature attention module; S3: Inputting the preprocessed coronary artery image dataset into the MUNet neural network model for training to obtain a trained MUNet neural network model; S4: Obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results; S5: Use image morphology algorithm to smooth the preliminary segmentation result to obtain the final segmentation result and complete the segmentation of the coronary artery.

2. The coronary artery image segmentation method based on MUNet according to claim 1, characterized in that: The preprocessing in step S1 includes any one or more of scale transformation, cropping, random flipping, random rotation, affine transformation and image value normalization.

3. The coronary artery image segmentation method based on MUNet according to claim 1, characterized in that: In step S2, the feature coarse extraction module includes: a 3D convolution block, a Convnext feature extraction block 1 and a Convnext feature extraction block 2 connected in sequence; The structure of the 3D convolutional block includes: a 4×4×4 convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer connected in sequence; The structure of each Convnext feature extraction block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext feature extraction block also form a residual sum connection through a bypass.

4. The coronary artery image segmentation method based on MUNet according to claim 1, characterized in that: In step S2, each of the convolution blocks has the same structure, including: a 4×4×4 convolutional layer, a batch normalization (BN) layer, a ReLU activation layer, and a 1×1×1 convolutional layer connected in sequence; The spatial domain attention branch includes: a 1×1×1 convolution layer, a 3×3×3 convolution layer, a 3×3×3 convolution layer, a 1×1×1 convolution layer and a batch normalization BN layer connected in sequence; the channel domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence.

5. The coronary artery image segmentation method based on MUNet according to claim 4, characterized in that: In step S2, a transition layer is provided before each of the multi-view feature attention modules, and the transition layer includes: a 1×1×1 convolutional layer, a batch normalization (BN) layer, a ReLU activation layer, a maximum pooling downsampling layer, and a Convnext block connected in sequence; The Convnext block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext block also form a residual sum connection through a bypass.

6. The coronary artery image segmentation method based on MUNet according to claim 1, characterized in that: In step S2, the Transformer encoder includes: an embedding layer and a plurality of Transformer blocks connected in sequence; Each of the Transformer blocks has the same structure, and includes the following connected in sequence: layer normalization LN layer 1, multi-head self-attention layer, layer normalization LN layer 2 and multi-layer perceptron layer; the input of layer normalization LN layer 1 also forms a residual sum connection with the output of the multi-head self-attention layer; the input of layer normalization LN layer 2 also forms a residual sum connection with the output of the multi-layer perceptron layer.

7. A coronary artery image segmentation method based on MUNet according to any one of claims 1 to 6, characterized in that: In step S2, the Transformer encoder includes four Transformer blocks, and the outputs of the four Transformer blocks are respectively recorded as T1, T2, T3 and T4; The number of the multi-view feature attention modules is 3; the output of the feature coarse extraction module is recorded as S1, and the outputs of the three multi-view feature attention modules are recorded as M1, M2 and M3 respectively; In the feature fusion network, T1 and S1 are fused and output as F1; T2 and M1 are fused and output as F2; ​​T3 and M2 are fused and output as F3; Fuse the features of T4 and M3 and output F4; In the decoder, F4 is fused with F3 after upsampling, and the output is C3; C3 is fused with F2 after upsampling, and the output is C2; C2 is fused with F1 after upsampling, and the output is C1; C1 is upsampled and outputs the segmentation prediction result of the coronary artery image as the output of the MUNet neural network model.

8. The coronary artery image segmentation method based on MUNet according to claim 7, characterized in that: In step S3, the loss function used in MUNet neural network model training is a loss function based on the similarity Dice coefficient. , specifically: Among them, P represents the prediction result of the MUNet neural network model, and R represents the true label of the preprocessed coronary artery image.

9. The coronary artery image segmentation method based on MUNet according to claim 1, characterized in that: The step S5 comprises: S5.1: Use morphological opening to remove noise and eliminate errors or noise that may be introduced during the image segmentation process; the morphological opening operation is erosion followed by dilation; The erosion operation is used to remove a layer of pixels at the edge of the image, thereby reducing the image. The mathematical formula is expressed as: Where S1 represents the image after corrosion, B1 represents the corroded structural element, A1 represents the original image before corrosion, and z represents the position of the corroded pixel in the original image. Only when B1 is translated to position z and B1 completely belongs to A1 at this time, does z belong to the corroded image S1 at this time. The dilation operation is used to increase the size of an image by adding a layer of pixels to the edge of the image. The mathematical formula is expressed as: Among them, S2 represents the image after expansion, B2 represents the expanded structural element, and A2 represents the original image before expansion. Indicates the position of the expanded pixel in the original image. Only when B2 is translated to position , and B2 has something in common with A2 at this time, It belongs to the expanded image S2; S5.2: Use a morphological closing operation to connect adjacent vascular breaks to enhance vascular continuity; the morphological closing operation is to first dilate and then erode; S5.3: Repeat steps S5.1 to S5.2 several times, further smoothing the edges of the blood vessels by continuously applying morphological opening and closing operations to obtain the final segmentation result; S5.4: Map the final segmentation result to the medical image space, store the mapped data in nii format, and complete the segmentation of the coronary artery.

10. A coronary artery image segmentation system based on MUNet, applying the coronary artery image segmentation method based on MUNet according to any one of claims 1 to 9, characterized in that: include: Preprocessing unit: used to obtain coronary artery image dataset and perform preprocessing; The coronary artery image dataset includes a plurality of 3D CTA images of the coronary artery; Model building unit: used to build the MUNet neural network model based on self-attention and multi-view feature attention; The MUNet neural network model includes: a Transformer encoder, a feature extraction network, a feature fusion network and a decoder; the feature extraction network includes: a feature coarse extraction module and several multi-view feature attention modules connected in sequence; The Transformer encoder and the feature extraction network are set in parallel, the outputs of the Transformer encoder and the feature extraction network are respectively connected to the input of the feature fusion network, and the output of the feature fusion network is connected to the decoder; The structure of the multi-view feature attention module includes: branches 1, 2 and 3 arranged in parallel, wherein branches 1, 2 and 3 correspond to the coronal view, the sagittal view and the axial view respectively; The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence; Each of the hybrid attention blocks has the same structure, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the outputs of the spatial domain attention branch and the channel domain attention branch are weightedly added as the output of the hybrid attention block; The outputs of branches 1, 2 and 3 are concatenated and fused as the output of the multi-view feature attention module; Model training unit: used to input the preprocessed coronary artery image dataset into the MUNet neural network model for training, and obtain the trained MUNet neural network model; Preliminary segmentation unit: used to obtain the coronary artery image to be segmented and input it into the trained MUNet neural network model for segmentation to obtain preliminary segmentation results; Post-processing unit: used to smooth the preliminary segmentation results using image morphology algorithms to obtain the final segmentation results and complete the segmentation of the coronary arteries.

Images