Pulmonary artery segmentation method and system for CT images based on contextual attention and feature fusion

Abstract

The present invention discloses a CT image pulmonary artery segmentation method and system based on contextual attention and feature fusion. The method comprises the following steps: obtaining an original CT image and performing image preprocessing in a training stage, extracting image blocks, constructing and training a 3D U-Net network based on contextual attention and feature fusion, obtaining a CT image to be inferred and performing image preprocessing in an inference stage, extracting image blocks, recording the CT image to which the blocks belong and the position in the CT image, obtaining a pulmonary artery segmentation prediction value based on the trained 3D U-Net network, combining the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT to obtain the pulmonary artery segmentation prediction value of the entire CT image, extracting the maximum connected component of the pulmonary artery after hard segmentation processing to obtain a segmentation result. The present invention can better extract key features of the pulmonary artery, alleviate the problem of class imbalance, and improve segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a CT image pulmonary artery segmentation method and system based on contextual attention and feature fusion. Background Art

[0002] Medical image segmentation refers to separating tissues, regions, structures, etc. of interest in a medical image from the entire image, such as pulmonary artery segmentation, tumor segmentation, skin lesion segmentation, etc.

[0003] In recent years, with the progress in the field of deep learning, a large number of deep learning-based medical image segmentation methods have been proposed. The most classic one is the U-Net structure, which consists of an encoder, a decoder, and a jump connection. The encoder captures the features and context of the input image through a series of convolutions and downsampling operations, and gradually reduces the size of the feature map; the decoder, on the contrary, gradually enlarges the feature map to its original size through a series of convolutions and upsampling operations to restore spatial information; the jump connection directly connects the corresponding layers of the decoder and the encoder, allowing the network to retain high-resolution information from the encoder feature map, which helps to retain details during the segmentation process. The U-Net structure is still the mainstream structure of medical image segmentation models, and the 3D U-Net for three-dimensional images such as CT images has also become an indispensable baseline model in various three-dimensional medical image segmentation tasks.

[0004] The current mainstream CT image pulmonary artery segmentation method is also based on the 3D U-Net model. However, the complex structure of the pulmonary artery, low imaging contrast, and thin ends have caused the following problems and challenges for its accurate segmentation:

[0005] 1. Prediction of fracture problem: The segmented pulmonary artery has discontinuous segments, which may have a great impact on subsequent clinical applications;

[0006] 2. Class imbalance problem: including inter-class imbalance problem and intra-class imbalance problem. The former refers to the fact that the number of voxels in the pulmonary artery is much smaller than the number of background voxels, and the latter refers to the fact that branch arteries are more difficult to segment than main arteries.

[0007] 3. Robustness issue: Lung diseases and abnormalities can cause significant changes in CT images, which in turn have a negative impact on pulmonary artery segmentation. Summary of the invention

[0008] In order to overcome the defects and shortcomings of the prior art, the present invention provides a CT image pulmonary artery segmentation method and system based on contextual attention and feature fusion. The present invention better extracts the key features of the pulmonary artery through contextual attention, alleviates the category imbalance problem through a hybrid loss function of feature fusion and group supervision, and thus improves the segmentation accuracy.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] The present invention provides a CT image pulmonary artery segmentation method based on context attention and feature fusion, comprising the following steps:

[0011] Training phase:

[0012] Obtain the original CT image, identify the lung area in the original CT image, crop out the part outside the lung area, perform linear normalization on the CT image, and obtain the true pulmonary artery label of the CT image;

[0013] Sliding and extracting image blocks in the lung area of ​​the CT image with a set step length and window size;

[0014] Construct a 3D U-Net network based on contextual attention and feature fusion, input the image block into the 3DU-Net network for network training, and obtain the predicted value of pulmonary artery segmentation corresponding to the image block;

[0015] During network training, the predicted values ​​of the pulmonary artery segmentation of the image blocks and the actual pulmonary artery labels are used to calculate the loss based on group supervision and hybrid loss function, and the network parameters are updated using the back propagation method. The iterative training is repeated until the convergence condition is reached and the training is completed.

[0016] Reasoning stage:

[0017] Acquire a CT image to be inferred, identify a lung region in the CT image to be inferred, and perform linear normalization on the CT image;

[0018] Sliding and extracting image blocks in the lung area in the CT image with a set step length and window size, and recording the CT image to which the block belongs and the position in the CT image;

[0019] The image blocks are input into the trained 3D U-Net network to obtain the pulmonary artery segmentation prediction value;

[0020] The CT image to which each block belongs and its position in the CT image are obtained, and the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT are combined to obtain the pulmonary artery segmentation prediction value of the entire CT image. After hard segmentation processing, the binary segmentation result is obtained, and the maximum connected component of the pulmonary artery is extracted as the final CT image pulmonary artery segmentation result.

[0021] As a preferred technical solution, the real pulmonary artery label of the CT image is obtained by using a binary label, where the pulmonary artery voxel of the CT image is marked as 1 and the background of the CT image is marked as 0.

[0022] As a preferred technical solution, it also includes a data enhancement step, which performs data enhancement on the extracted image blocks, including horizontal flipping, vertical flipping, Gaussian smoothing and voxel grayscale jittering.

[0023] As a preferred technical solution, the 3D U-Net network based on contextual attention and feature fusion includes: an encoder, a decoder, and a skip connection structure, wherein the encoder is connected to the decoder;

[0024] The encoder includes a context attention convolution block and a downsampling operation module, which extracts abstract features from the input image block through convolution and downsampling operations, reduces the size of the feature map, increases the number of feature map channels, and obtains the feature map;

[0025] The decoder includes a context attention convolution block and an upsampling operation module, which restores the feature map to the same size as the input image block through convolution and upsampling;

[0026] The skip connection structure connects the low-level feature map in the encoder to the high-level feature map in the decoder;

[0027] The encoder and the decoder are both connected to a feature fusion module, and the corresponding feature maps are subjected to feature fusion by the feature fusion module to obtain a pulmonary artery segmentation prediction value of the encoder group and a pulmonary artery segmentation prediction value of the decoder group.

[0028] As a preferred technical solution, the context attention convolution block includes a convolution, a first instance normalization module, a first ReLU activation function, a context attention module, a second instance normalization module, and a second ReLU activation function connected in sequence;

[0029] The context attention convolution block uses linear interpolation to upsample the output feature map to the input image block size, and outputs it to the feature fusion module for feature fusion.

[0030] As a preferred technical solution, the contextual attention module performs k×k×k convolution on the input to generate a Key. The Key itself is used as a static context, including local context information from neighbors. At the same time, the input itself is used as a Query, and a 1×1×1 convolution is performed on the input to generate a Value. After the Key and the Query are concatenated at the channel level, a 1×1×1 convolution is performed to interact at the channel level. A SoftMax operation is performed to generate a spatial attention score matrix. The attention matrix and each element of the Value are multiplied as a dynamic context, and the module output is obtained by adding each element to mix the dynamic and static context.

[0031] As a preferred technical solution, the feature fusion module concatenates the feature maps output by each context attention convolution block in this group at the channel level, and uses them as the output of this group after 1x1x1 convolution and sigmoid function, which is specifically expressed as:

[0032] p i =Sigmoid(Conv1(Concate(f i,1 ,f i,2 ,…,f i,n )))

[0033] Among them, f i,1 ,f i,2 ,…,f i,n Represents the feature maps for fusion output by each context attention convolution block in this group, Conv1 represents 1x1x1 convolution, Concate represents the concatenation operation at the channel level, and p i It represents the output of each feature map of the i-th group after concatenation, convolution, and Sigmoid activation function, as the pulmonary artery segmentation prediction value of the i-th group for the current input image block.

[0034] As a preferred technical solution, the predicted value of the pulmonary artery segmentation of the image block and the actual pulmonary artery label are used to calculate the loss based on group supervision and a mixed loss function, and the specific steps include:

[0035] Obtain the pulmonary artery segmentation prediction value p0 of the encoder group and the pulmonary artery segmentation prediction value p1 of the decoder group output by the network during the training phase, calculate the loss using the mixed loss function and sum them up, expressed as:

[0036] Loss=HyperLoss(p0,gt)+HyperLoss(p1,gt)

[0037] HyperLoss(p i ,gt)=Focal(p i ,gt)+λTversky(p i ,gt)

[0038] Among them, gt represents the corresponding true pulmonary artery label, HyperLoss uses a mixed loss function of Root Tversky Loss and Focal Loss as the loss function during network training, and λ is a coefficient used to balance the two losses;

[0039] The expression of Focal Loss is:

[0040] Focal(pred,gt)=∑ i -η t,i (1-p t,i )γ log(p t,i )

[0041]

[0042]

[0043] Among them, g i represents the true label of voxel i, p i represents the predicted value of voxel i, η represents the hyperparameter used to adjust the weights of samples of different categories, and γ represents the hyperparameter used to adjust the weights of difficult and easy samples;

[0044] The expression of Root Tversky Loss is:

[0045]

[0046] Among them, α and β represent the hyperparameters used to balance accuracy and recall, r represents the coefficient used to adjust the attention to difficult-to-predict voxels, and 0 <r<1。

[0047] As a preferred technical solution, a binary segmentation result is obtained after hard segmentation processing, and the specific steps include:

[0048] The pulmonary artery segmentation prediction value is a floating point value in the interval [0,1]. A threshold is set and a binary segmentation result is obtained after hard segmentation processing. Values ​​less than the set threshold are divided into 0 and used as the background, and values ​​greater than the threshold are divided into 1 and used as the segmented pulmonary artery voxels.

[0049] The present invention also provides a CT image pulmonary artery segmentation system based on contextual attention and feature fusion, comprising: an image acquisition module, an image preprocessing module, a sampling module, a position recording module, a 3D U-Net network construction module, a 3DU-Net network training module, a pulmonary artery segmentation prediction value calculation module, and a segmentation result acquisition module;

[0050] In the training stage, the image acquisition module is used to acquire the original CT image;

[0051] The image preprocessing module is used to identify the lung area in the original CT image, crop the part outside the lung area, perform linear normalization on the CT image, and obtain the real pulmonary artery label of the CT image;

[0052] The sampling module is used to slide and extract image blocks in the lung area in the CT image with a set step length and window size;

[0053] The 3D U-Net network building module is used to build a 3DU-Net network based on contextual attention and feature fusion;

[0054] The 3D U-Net network training module is used to input the image block into the 3D U-Net network for network training, use the predicted value of the pulmonary artery segmentation of the image block and the real pulmonary artery label, calculate the loss based on the group supervision and hybrid loss function, use the back propagation method to update the network parameters, repeat the iterative training until the convergence condition is reached, and complete the 3D U-Net network training;

[0055] In the inference stage, the image acquisition module is used to acquire the CT image to be inferred;

[0056] The image preprocessing module is used to identify the lung area in the CT image to be inferred and perform linear normalization on the CT image;

[0057] The sampling module is used to slide and extract image blocks in the lung area in the CT image with a set step length and window size;

[0058] The position recording module is used to record the CT image to which the block belongs and the position in the CT image;

[0059] The pulmonary artery segmentation prediction value calculation module is used to input the image block into the trained 3DU-Net network to obtain the pulmonary artery segmentation prediction value;

[0060] The segmentation result acquisition module is used to obtain the CT image to which each block belongs and its position in the CT image, combine the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT to obtain the pulmonary artery segmentation prediction value of the entire CT image, obtain a binary segmentation result after hard segmentation processing, and extract the maximum connected component of the pulmonary artery as the final CT image pulmonary artery segmentation result.

[0061] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0062] (1) The present invention introduces a contextual attention operation, which can adjust the weights of each position of the feature map according to the context information, so as to focus more on the learning of key features of the pulmonary artery;

[0063] (2) The present invention introduces a feature fusion module to fuse feature maps of multiple sizes to fully utilize the complementarity between features at different levels;

[0064] (3) The present invention introduces a group supervision mechanism. The additional supervision signal enables the gradient to be better transmitted to the shallow layer of the neural network, alleviating the difficulty of gradient backpropagation caused by the category imbalance problem.

[0065] (4) The present invention uses a hybrid loss function of Root Tversky Loss and Focal Loss to assign higher weights to difficult samples, so that the model pays more attention to the pulmonary artery voxels that are originally difficult to identify, thereby improving the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of a CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to the present invention;

[0067] Figure 2 The structural diagram of the 3D U-Net network based on contextual attention and feature fusion of the present invention;

[0068] Figure 3 It is a structural schematic diagram of the feature fusion module of the present invention;

[0069] Figure 4 Schematic diagram of the structure of the context attention convolution block of the present invention;

[0070] Figure 5 Schematic diagram of the structure of the context attention module of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0072] Example 1

[0073] like Figure 1 As shown, this embodiment provides a CT image pulmonary artery segmentation method based on contextual attention and feature fusion, including two stages: training and reasoning. The training stage includes the following steps:

[0074] S1: CT image preprocessing: Identify the lung area in the original CT image and crop out the part outside the lung area. Then, perform linear normalization on the CT image. For the label corresponding to the CT image, the lung area should also be cropped out.

[0075] In this embodiment, the label corresponding to the CT image in step S1 is a binary label, which is used to indicate whether each voxel in the CT image belongs to the pulmonary artery (including the intrapulmonary and extrapulmonary parts) or the background (other lung tissue structures and extrapulmonary parts). The pulmonary artery voxel is marked as 1 in the label, and the background voxel is marked as 0 in the label.

[0076] S2: Sliding window sampling: extract image blocks in the extracted lung area with a step size of 128×128×128 and a window size of 128×128×128;

[0077] In this embodiment, the sliding window in step S2 moves along the three dimensions of the CT image with a specified step length to extract an image block of a specified size;

[0078] In this embodiment, data enhancement is further performed on the extracted image blocks, and specific data enhancement methods include: horizontal flipping, vertical flipping, Gaussian smoothing, and voxel grayscale jittering.

[0079] S3: Pulmonary artery segmentation: A series of image blocks obtained by the above sliding window sampling are input into the 3D U-Net network based on context attention and feature fusion, and the predicted value of the pulmonary artery segmentation of the image block is obtained based on the forward process of the neural network;

[0080] In this embodiment, the pulmonary artery segmentation prediction value of the image block includes one channel, which represents the probability value of each voxel in the block belonging to the pulmonary artery, and the value is in the interval [0, 1].

[0081] like Figure 2 As shown in the figure, the 3D U-Net network based on contextual attention and feature fusion specifically includes:

[0082] Encoder: It consists of contextual attention convolution blocks and downsampling operations. It gradually extracts high-level abstract features from the input image blocks through convolution and downsampling operations, gradually reduces the size of the feature map, and increases the number of feature map channels.

[0083] Decoder: It consists of a context attention convolution block and an upsampling operation, which restores the feature map output by the encoder to the same size as the input image block through convolution and upsampling;

[0084] Skip connection: directly connects the feature maps of the shallower layers in the encoder to the feature maps of the higher layers in the decoder. On the one hand, it can accelerate the back propagation of gradients, and on the other hand, it introduces high-resolution spatial information in the encoder to the decoder, enhancing the positioning ability.

[0085] Feature fusion module: The encoder and decoder each have a feature fusion module to fuse the feature maps of their respective layers; the fused feature maps make full use of the complementarity of feature map information of different sizes and stages, and will be used for group supervision during model training;

[0086] like Figure 3 As shown in the figure, the feature fusion module concatenates the feature maps output by each context attention convolution block in this group for fusion at the channel level, and then uses them as the output of this group after 1x1x1 convolution and sigmoid function, that is:

[0087] p i =Sigmoid(Conv1(Concate(f i,1 ,f i,2 ,…,f i,n )))

[0088] Among them, f i,1 ,f i,2 ,…,f i,n It is the feature map for fusion output by each context attention convolution block in this group. Conv1 refers to 1x1x1 convolution, and Concate refers to the concatenation operation at the channel level. The output p of each feature map of the i-th group after concatenation, convolution, and Sigmoid activation function i is regarded as the ith group of pulmonary artery segmentation prediction values ​​for the current input image block.

[0089] In the embodiment, the feature maps for fusion output by each context attention convolution block of the encoder are classified into an encoder group, and after feature fusion, the pulmonary artery segmentation prediction value p0 of the encoder group is obtained. Similarly, the pulmonary artery segmentation prediction value p1 of the decoder group is obtained.

[0090] like Figure 4 As shown in the figure, the context attention convolution block of the 3D U-Net network based on context attention and feature fusion includes a 3x3x3 convolution, a first instance normalization module, a first ReLU activation function, a context attention module, a second instance normalization module, and a second ReLU activation function connected in sequence; at the same time, the context attention convolution block also uses linear interpolation to upsample the original output feature map to the input image block size as another output for feature fusion;

[0091] like Figure 5As shown, the contextual attention module generates Key by performing k×k×k convolution on the input (k is 3 in the embodiment). The Key itself is used as a static context, containing local context information from neighbors; at the same time, the input itself is used as a query, and a 1×1×1 convolution is performed on the input to generate a Value. After the Key and the Query are concatenated at the channel level, they interact at the channel level through a 1×1×1 convolution, which is similar to the additive attention mechanism; then a spatial attention score matrix is ​​generated through a SoftMax operation, which contains the relationship information from each pair of Key and Value. Then the attention matrix and each element of the Value are multiplied as a dynamic context. Finally, the module output is obtained by adding each element to mix the dynamic and static context. The general attention module uses the attention mechanism to assign different weights to different positions of the feature map, guiding the model to pay attention to more important positions; however, the Key generation in the attention mechanism is generally point-to-point, which ignores the context information. On this basis, the contextual attention improves the Key generation process, uses k×k×k convolution to generate keys, and introduces context information through convolution, so that the model can consider the semantic information of the local area rather than just the semantic information of a single position.

[0092] S4: Calculate loss and update parameters: When training the network, use the segmentation results of the image blocks and the true pulmonary artery labels, calculate the loss based on group supervision and mixed loss function, use the back propagation method to update the network parameters, and repeat the iterative training until the model converges and the training is completed;

[0093] In this embodiment, the network outputs the pulmonary artery segmentation prediction value p0 of the encoder group and the pulmonary artery segmentation prediction value p1 of the decoder group during the training phase, and the group supervision is embodied in that the loss of the two groups is calculated using a mixed loss function with reference to the pulmonary artery labels, and then summed, that is:

[0094] loss=HyperLoss(p0,gt)+HyperLoss(p1,gt)

[0095] Among them, gt is the corresponding pulmonary artery label, and HyperLoss uses a mixed loss function of Root Tversky Loss and FocalLoss as the loss function during network training, which is specifically expressed as:

[0096] HyperLoss(p i ,gt)=Focal(p i ,gt)+λTversky(p i ,gt)

[0097] Among them, λ is a coefficient used to balance the two losses. Focal Loss is used to measure the difference between the predicted probability distribution and the true probability distribution; Root Tversky Loss is used to measure the overlap degree between the predicted region and the true region.

[0098] In this embodiment, the expression of Focal Loss is:

[0099] Focal(pred,gt) = ∑ i -η t,i (1 - p t,i ) γ log(p t,i )

[0100] Among them, γ is an adjustable hyperparameter, and η t,i and p t,i are defined as follows:

[0101]

[0102]

[0103] Among them, g i represents the category of voxel i, and p i represents the predicted value of voxel i. η is an adjustable hyperparameter. The coefficient η t,i assigns different weights to different category samples to alleviate the problem of inter-class imbalance, and changes the specific weights of different category samples by adjusting the hyperparameter η. Another coefficient (1 - p t,i ) γ takes the prediction confidence as part of the coefficient, increases the weight of difficult samples, makes the model pay more attention to the pulmonary artery voxels that are originally difficult to segment, alleviates the problem of intra-class imbalance, and the specific weights of difficult and easy samples can be further adjusted by the hyperparameter γ.

[0104] In this embodiment, the expression of Root Tversky Loss is:

[0105]

[0106] Among them, p i , g i are the prediction and true label for a single voxel respectively. α, β are hyperparameters used to balance accuracy and recall rate, and 0 < r < 1 is used as a coefficient to adjust the attention to difficult-to-predict voxels;

[0107] By adjusting the coefficients α and β, the model's emphasis on accuracy and recall can be changed, thereby alleviating the problem of inter-class imbalance. The coefficient r increases the gradient value of voxels with smaller prediction confidence, that is, it gives higher weights to difficult samples, making the model pay more attention to the pulmonary artery voxels that are originally difficult to segment, alleviating the problem of intra-class imbalance.

[0108] The inference phase consists of the following steps:

[0109] S1: CT image preprocessing: The inference stage does not require CT image labels, nor does it require label cropping. The remaining operations are the same as those in the training stage.

[0110] S2: Sliding window sampling: Compared with the training stage, when extracting blocks, it is necessary to additionally record the CT image to which the blocks belong and their positions in the CT image, which are used to combine the segmentation results of each block during post-processing. Data enhancement is not used in the inference stage, and the rest of the operations are the same as in the training stage.

[0111] S3: Pulmonary artery segmentation: The image block extracted in step S2 is input into the trained 3D U-Net network based on contextual attention and feature fusion, and only the pulmonary artery segmentation prediction value p1 of the decoder group is used as the pulmonary artery segmentation prediction value of the network.

[0112] S4: post-processing: combining the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT image and the positions of the blocks in the CT image recorded in step S2 to obtain the pulmonary artery segmentation prediction value of the entire CT image;

[0113] The predicted value of pulmonary artery segmentation is a floating point value in the interval [0,1]. After hard segmentation processing with a threshold of 0.5, it becomes a binary segmentation result of 0 or 1. A value less than 0.5 is classified as 0 (background), and a value greater than 0.5 is classified as 1. The voxel marked as 1 is the segmented pulmonary artery voxel. Then, the maximum connected component of the pulmonary artery is extracted and used as the final segmentation result.

[0114] Example 2

[0115] This embodiment provides a CT image pulmonary artery segmentation system based on contextual attention and feature fusion, including: an image acquisition module, an image preprocessing module, a sampling module, a position recording module, a 3D U-Net network construction module, a 3DU-Net network training module, a pulmonary artery segmentation prediction value calculation module, and a segmentation result acquisition module;

[0116] In the training phase, the image acquisition module is used to obtain the original CT images;

[0117] The image preprocessing module is used to identify the lung area in the original CT image, crop the part outside the lung area, perform linear normalization on the CT image, and obtain the real pulmonary artery label of the CT image;

[0118] The sampling module is used to slide and extract image blocks in the lung area of ​​the CT image with a set step length and window size;

[0119] The 3D U-Net network building module is used to build a 3D U-Net network based on contextual attention and feature fusion;

[0120] The 3D U-Net network training module is used to input image blocks into the 3D U-Net network for network training, use the predicted value of the pulmonary artery segmentation of the image block and the actual pulmonary artery label, calculate the loss based on group supervision and mixed loss function, use the back propagation method to update the network parameters, repeat iterative training until the convergence condition is reached, and complete the 3D U-Net network training;

[0121] In the inference stage, the image acquisition module is used to obtain the CT image to be inferred;

[0122] The image preprocessing module is used to identify the lung area in the CT image to be inferred and perform linear normalization on the CT image;

[0123] The sampling module is used to slide and extract image blocks in the lung area of ​​the CT image with a set step length and window size;

[0124] The position recording module is used to record the CT image to which the block belongs and the position in the CT image;

[0125] The prediction value calculation module of pulmonary artery segmentation is used to input the image block into the trained 3D U-Net network to obtain the prediction value of pulmonary artery segmentation;

[0126] The segmentation result acquisition module is used to obtain the CT image to which each block belongs and its position in the CT image, and to combine the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT to obtain the pulmonary artery segmentation prediction value of the entire CT image. After hard segmentation processing, a binary segmentation result is obtained, and the maximum connected component of the pulmonary artery is extracted as the final CT image pulmonary artery segmentation result.

[0127] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A CT image pulmonary artery segmentation method based on contextual attention and feature fusion, characterized in that: The steps include: Training phase: Obtain the original CT image, identify the lung area in the original CT image, crop out the part outside the lung area, perform linear normalization on the CT image, and obtain the true pulmonary artery label of the CT image; Sliding and extracting image blocks in the lung area of ​​the CT image with a set step length and window size; Construct a 3D U-Net network based on contextual attention and feature fusion, input the image block into the 3DU-Net network for network training, and obtain the predicted value of pulmonary artery segmentation corresponding to the image block; During network training, the predicted values ​​of the pulmonary artery segmentation of the image blocks and the actual pulmonary artery labels are used to calculate the loss based on group supervision and hybrid loss function, and the network parameters are updated using the back propagation method. The iterative training is repeated until the convergence condition is reached and the training is completed. The predicted value of the pulmonary artery segmentation of the image block and the actual pulmonary artery label are used to calculate the loss based on group supervision and a mixed loss function. The specific steps include: Obtain the pulmonary artery segmentation prediction value p0 of the encoder group and the pulmonary artery segmentation prediction value p1 of the decoder group output by the network during the training phase, calculate the loss using the mixed loss function and sum them up, expressed as: Loss=HyperLoss(p0,gt)+HyperLoss(p1,gt) HyperLoss(p i ,gt)=Focal(p i ,gt)+λTversky(p i ,gt) Among them, gt represents the corresponding true pulmonary artery label, HyperLoss uses the mixed loss function of Root Tversky Loss and FocalLoss as the loss function during network training, and λ is the coefficient used to balance the two losses; The expression of FocalLoss is: Focal(pred,gt)=∑ i -or t,i (1-p t,i ) γ log(p t,i ) Among them, g i represents the true label of voxel i, p i represents the predicted value of voxel i, η represents the hyperparameter used to adjust the weights of samples of different categories, and γ represents the hyperparameter used to adjust the weights of difficult and easy samples; The expression of Root Tversky Loss is: Among them, α and β represent the hyperparameters used to balance accuracy and recall, r represents the coefficient used to adjust the attention to difficult-to-predict voxels, and 0 <r<1; Reasoning phase: Acquire a CT image to be inferred, identify a lung region in the CT image to be inferred, and perform linear normalization on the CT image; Sliding and extracting image blocks in the lung area in the CT image with a set step length and window size, and recording the CT image to which the block belongs and the position in the CT image; The image blocks are input into the trained 3D U-Net network to obtain the pulmonary artery segmentation prediction value; The CT image to which each block belongs and its position in the CT image are obtained, and the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT are combined to obtain the pulmonary artery segmentation prediction value of the entire CT image. After hard segmentation processing, the binary segmentation result is obtained, and the maximum connected component of the pulmonary artery is extracted as the final CT image pulmonary artery segmentation result.

2. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 1, characterized in that: The real pulmonary artery label of the CT image is obtained by using a binary label, where the pulmonary artery voxel of the CT image is labeled as 1 and the background of the CT image is labeled as 0.

3. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 1, characterized in that: The method also includes a data enhancement step, in which the extracted image blocks are enhanced, including horizontal flipping, vertical flipping, Gaussian smoothing, and voxel grayscale jittering.

4. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 1, characterized in that: The 3D U-Net network based on contextual attention and feature fusion includes: an encoder, a decoder, and a skip connection structure, wherein the encoder is connected to the decoder; The encoder includes a context attention convolution block and a downsampling operation module, which extracts abstract features from the input image block through convolution and downsampling operations, reduces the size of the feature map, increases the number of feature map channels, and obtains a feature map; The decoder includes a context attention convolution block and an upsampling operation module, which restores the feature map to the same size as the input image block through convolution and upsampling; The skip connection structure connects the low-level feature map in the encoder to the high-level feature map in the decoder; The encoder and the decoder are both connected to a feature fusion module, and the corresponding feature maps are subjected to feature fusion by the feature fusion module to obtain a pulmonary artery segmentation prediction value of the encoder group and a pulmonary artery segmentation prediction value of the decoder group.

5. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 4, characterized in that: The context attention convolution block includes a convolution, a first instance normalization module, a first ReLU activation function, a context attention module, a second instance normalization module, and a second ReLU activation function connected in sequence; The context attention convolution block uses linear interpolation to upsample the output feature map to the input image block size, and outputs it to the feature fusion module for feature fusion.

6. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 5, characterized in that: The contextual attention module performs k×k×k convolution on the input to generate Key. The Key itself is used as static context, including local context information from neighbors. At the same time, the input itself is used as Query, and 1×1×1 convolution is performed on the input to generate Value. After concatenating Key and Query at the channel level, they interact at the channel level through 1×1×1 convolution. After the SoftMax operation, the spatial attention score matrix is ​​generated. The attention matrix and each element of Value are multiplied as dynamic context, and the module output is obtained by adding each element to mix the dynamic and static context.

7. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 4, characterized in that: The feature fusion module concatenates the feature maps output by each context attention convolution block in this group at the channel level, and uses them as the output of this group after 1x1x1 convolution and sigmoid function, which is specifically expressed as: p i =Sigmoid(Conv1(Concate(f i,1 ,f i,2 ,…,f i,n ))) Among them, f i,1 ,f i,2 ,…,f i,n Represents the feature maps for fusion output by each context attention convolution block in this group, Conv1 represents 1x1x1 convolution, Concate represents the concatenation operation at the channel level, and p i It represents the output of each feature map of the i-th group after concatenation, convolution, and Sigmoid activation function, as the pulmonary artery segmentation prediction value of the i-th group for the current input image block.

8. The CT image pulmonary artery segmentation method based on contextual attention and feature fusion according to claim 1, characterized in that: After hard segmentation, the binary segmentation result is obtained. The specific steps include: The pulmonary artery segmentation prediction value is a floating point value in the interval [0,1]. A threshold is set and a binary segmentation result is obtained after hard segmentation processing. Values ​​less than the set threshold are divided into 0 and used as the background, and values ​​greater than the threshold are divided into 1 and used as the segmented pulmonary artery voxels.

9. A CT image pulmonary artery segmentation system based on contextual attention and feature fusion, characterized in that: include: Image acquisition module, image preprocessing module, sampling module, position recording module, 3D U-Net network construction module, 3D U-Net network training module, pulmonary artery segmentation prediction value calculation module, segmentation result acquisition module; In the training stage, the image acquisition module is used to acquire the original CT image; The image preprocessing module is used to identify the lung area in the original CT image, crop the part outside the lung area, perform linear normalization on the CT image, and obtain the real pulmonary artery label of the CT image; The sampling module is used to slide and extract image blocks in the lung area in the CT image with a set step length and window size; The 3D U-Net network building module is used to build a 3DU-Net network based on contextual attention and feature fusion; The 3D U-Net network training module is used to input the image block into the 3D U-Net network for network training, use the predicted value of the pulmonary artery segmentation of the image block and the real pulmonary artery label, calculate the loss based on the group supervision and hybrid loss function, use the back propagation method to update the network parameters, repeat the iterative training until the convergence condition is reached, and complete the 3D U-Net network training; The predicted value of the pulmonary artery segmentation of the image block and the actual pulmonary artery label are used to calculate the loss based on group supervision and a mixed loss function, specifically including: Obtain the pulmonary artery segmentation prediction value p0 of the encoder group and the pulmonary artery segmentation prediction value p1 of the decoder group output by the network during the training phase, calculate the loss using the mixed loss function and sum them up, expressed as: Loss=HyperLoss(p0,gt)+HyperLoss(p1,gt) HyperLoss(p i ,gt)=Focal(p i ,gt)+λTversky(p i ,gt) Among them, gt represents the corresponding true pulmonary artery label, HyperLoss uses the mixed loss function of Root Tversky Loss and FocalLoss as the loss function during network training, and λ is the coefficient used to balance the two losses; The expression of Focal Loss is: Focal(pred,gt)=∑ i -or t,i (1-p t,i ) γ log(p t,i ) Among them, g i represents the true label of voxel i, p i represents the predicted value of voxel i, η represents the hyperparameter used to adjust the weights of samples of different categories, and γ represents the hyperparameter used to adjust the weights of difficult and easy samples; The expression of Root Tversky Loss is: Among them, α and β represent the hyperparameters used to balance accuracy and recall, r represents the coefficient used to adjust the attention to difficult-to-predict voxels, and 0 <r<1; In the inference stage, the image acquisition module is used to acquire the CT image to be inferred; The image preprocessing module is used to identify the lung area in the CT image to be inferred and perform linear normalization on the CT image; The sampling module is used to slide and extract image blocks in the lung area in the CT image with a set step length and window size; The position recording module is used to record the CT image to which the block belongs and the position in the CT image; The pulmonary artery segmentation prediction value calculation module is used to input the image block into the trained 3DU-Net network to obtain the pulmonary artery segmentation prediction value; The segmentation result acquisition module is used to obtain the CT image to which each block belongs and its position in the CT image, combine the pulmonary artery segmentation prediction values ​​of all image blocks belonging to the same CT to obtain the pulmonary artery segmentation prediction value of the entire CT image, obtain a binary segmentation result after hard segmentation processing, and extract the maximum connected component of the pulmonary artery as the final CT image pulmonary artery segmentation result.

Images