A fetal brain tissue segmentation method based on a spatial channel attention gate network

Abstract

The application discloses a fetal brain tissue segmentation method based on a spatial channel attention gate network, obtains a FeTA data set, and pre-processes the FeTA data set to obtain a plurality of 2D images; the 2D images are divided into a training set and a test set according to a set proportion; a segmentation network based on the spatial channel attention gate network is constructed; the segmentation network based on the spatial channel attention gate network is trained by using the training set; fetal brain MR images in the test set are input into the segmentation model to output a predicted image of the fetal brain; evaluation results of the predicted image are obtained by using a plurality of evaluation functions; the segmentation network is evaluated based on the evaluation results; the test is ended after the evaluation is qualified; and a to-be-segmented image is input into the trained segmentation network to obtain a target fetal brain tissue image. The application reduces the interference of redundant information on a segmentation result, so that the segmentation precision of the network is improved.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a method for segmenting fetal brain tissue based on spatial channel attention gating networks. Background Technology

[0002] In recent years, deep learning methods have achieved great success in medical image segmentation and computer-aided diagnosis. Deep learning networks can automatically learn the contextual information of images through convolutional operations, thereby obtaining better image feature representations.

[0003] The existing MTSE U-Net technology can perform multiple tasks, including fetal brain tissue segmentation and prediction of fetal brain type and gestational age. Its fetal brain tissue segmentation is achieved through an extension of U-Net, simply adding a convolutional block to the skip connections of U-Net. However, MTSE U-Net exhibits significant inaccuracies in segmenting the details of fetal brain tissue.

[0004] The existing Deep U-Net improves the network by deepening the convolutional layers of U-Net and introducing more robust activation and optimization functions to obtain fetal brain tissue segmentation results. Deep U-Net simply deepens the convolutional layers, treating all information in the input image equally, which consumes more computing power but does not necessarily improve the network's segmentation accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a method for segmenting fetal brain tissue based on spatial channel attention gating networks.

[0006] The technical solution adopted in this invention is:

[0007] A method for segmenting fetal brain tissue based on spatial channel attention gating networks, comprising the following steps:

[0008] Step S1: After obtaining the FeTA dataset and preprocessing it to obtain several 2D images, divide them into training set and test set according to a set ratio;

[0009] Step S2: Construct a segmentation network based on a spatial channel attention gate network, including a RaSE module, a multi-scale fusion (MSF) module, and a classification module arranged sequentially. The RaSE module, i.e., the residual and squeeze excitation module, adopts an encoder-decoder structure classification network. The RaSE module includes a residual block (ResBlock) and an attention block (Attention Block) arranged sequentially. The residual block performs preliminary feature extraction to reduce feature loss. The output of the residual block is connected to the attention block. The attention block learns the importance of each feature channel, promotes useful features, and suppresses useless features based on their importance. The multi-scale feature fusion module fuses feature maps of different scales from the decoder of the residual block. The classification module divides the result of the feature fusion part into N classes through a 1×1 convolution, where the value of N depends on the number of categories of the label.

[0010] Step S3: Train the segmentation network based on the spatial channel attention gate network using the training set. The specific steps of step S3 are as follows:

[0011] Step S3-1: Randomly flip the input 2D image vertically or horizontally and randomly rotate it from 0 to 360°.

[0012] Step S3-2: Input the 2D image into the encoder of the segmentation network constructed based on the spatial channel attention gate network, and use the RaSE module and downsampling to extract high-level information; use the residual part of the 2D image to extract feature information λ' at different scales, and input the feature information λ' into the channel attention mechanism SE Attention to obtain feature information λ with channel weights at different scales.

[0013] Step S3-3: Input the high-level information into the encoder of FBTS Net, and pass the feature information λ with channel weights from the encoder of the same layer to the decoder by skipping the connection, and restore the image size by upsampling.

[0014] In steps S3-4, the three-layer decoder obtains features ω1, ω2, and ω3 respectively. ω1, ω2, and ω3 are simultaneously input into the multi-scale fusion module to recover the image size through upsampling, resulting in ω′1, ω′2, and ω′3 with a size of 256×256.

[0015] In steps S3-5, redundant information in ω′1, ω′2, and ω′3 is suppressed by spatial attention blocks (SAM+) to obtain spatial attention features ω″1, ω″2, and ω″3.

[0016] Steps S3-6: Initialize the three weights α, γ, and λ for training. Add ω”1, ω”2, and ω”3 according to their weights, and then perform a 1×1 convolution to obtain the fused feature y, as shown in the following formula:

[0017] y=α×ω”1+γ×ω”2+λ×ω”3 (3);

[0018] Step S3-7: The fused feature y is fed into the classifier to perform multi-classification of the feature information, and a predicted map of the fetal brain tissue is obtained.

[0019] Step S4: Input the fetal brain MR images from the test set into the segmentation model to obtain the predicted map of the fetal brain;

[0020] Step S5: Obtain the evaluation results of the predicted image through multiple evaluation functions. Evaluate the segmentation network based on the evaluation results. If the evaluation is unsatisfactory, update the training parameters of the segmentation network and execute step S3. If the evaluation is satisfactory, the test ends.

[0021] Step S6: Input the image to be segmented into the trained segmentation network to obtain the target fetal brain tissue image.

[0022] Furthermore, in step S1, the FeTA dataset is preprocessed by slicing along the axial plane to obtain several 2D images, which are then divided into training and testing sets at a set ratio of 8:2.

[0023] Specifically, the FeTA dataset was first preprocessed by slicing it along the axial plane to obtain 11,183 2D images of size 256×256, of which 8,946 were used for training and 2,237 were used for testing.

[0024] Furthermore, the encoder of the RaSE module is connected to the decoder through the underlying module.

[0025] Furthermore, the RaSE module satisfies the following formula:

[0026] RS(X)=R(X)×S(R(X)) (1)

[0027] Where X is the input, RS(X) is the final result of the RaSE module, R(X) is the result of the residual block, and S(R(X)) is the channel feature result extracted by the spatial channel attention block with the result of the residual block as input.

[0028] Furthermore, in step S2, SE Net is used as the spatial channel attention block (SE Attention Block) to construct the attention block. SE Net first performs a squeezing operation (i.e., global average pooling operation) on the features to generate channel descriptors, which represent the distribution of channel features. Then, it learns the specific sample activation of each channel through adaptive activation operation (i.e., two fully connected operations).

[0029] Furthermore, the MSF module includes an upsampling block, a spatial attention part, and a feature fusion part. The decoder generates features with the same high resolution as the input image by upsampling the feature maps of semantic information at different levels; and fully represents spatial information by fusing the feature maps obtained by operations such as average pooling, max pooling, and 1×1 convolution.

[0030] Furthermore, SAM+ adds a 1×1 convolution to SAM, and fully represents spatial information by fusing the feature maps obtained from average pooling, max pooling and 1×1 convolution operations.

[0031] Furthermore, the evaluation functions in step S5 include the Dice coefficient, IoU coefficient, Recall (i.e., recall rate), and Precision (i.e., precision rate).

[0032] This invention employs the above technical solutions. Regarding the feature extraction part, Deep U-Net extracts more image information without distinguishing all information, resulting in insufficient feature representation capability of the network. The RaSE module, composed of ResNet and SENet, not only extracts more image information but also focuses the network on more effective information, improving the encoder's feature extraction capability. To address the problem of inaccurate detail segmentation, the MSF module is used to reduce interference from redundant information and combine feature information at multiple scales, improving the accuracy of detail segmentation. Attached Figure Description

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0034] Figure 1 This is a schematic diagram of the structure of a fetal brain tissue segmentation network based on a spatial channel attention gating network according to the present invention;

[0035] Figure 2 This is a schematic diagram of the channel attention residual convolution module of the present invention;

[0036] Figure 3 This is a schematic diagram of the SE Net network structure of the present invention;

[0037] Figure 4This is a schematic diagram of the spatial attention module of the present invention;

[0038] Figure 5 This is a schematic diagram of the structure of the multi-scale fusion module of the present invention. Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0040] like Figures 1 to 5 As shown in the figure, this invention discloses a method for segmenting fetal brain tissue based on spatial channel attention gating networks, which includes the following steps:

[0041] Step S1: After obtaining the FeTA dataset and preprocessing it to obtain several 2D images, divide them into training set and test set according to a set ratio;

[0042] Step S2: Construct a segmentation network based on a spatial channel attention gate network, including a RaSE module, a multi-scale fusion (MSF) module, and a classification module arranged sequentially. The RaSE module, i.e., the residual and squeeze excitation module, adopts an encoder-decoder structure classification network. The RaSE module includes a residual block (ResBlock) and an attention block (Attention Block) arranged sequentially. The residual block performs preliminary feature extraction to reduce feature loss. The output of the residual block is connected to the attention block. The attention block learns the importance of each feature channel, promotes useful features, and suppresses useless features based on their importance. The multi-scale feature fusion module fuses feature maps of different scales from the decoder of the residual block. The classification module divides the result of the feature fusion part into N classes through a 1×1 convolution, where the value of N depends on the number of categories of the label.

[0043] Step S3: Train the segmentation network based on the spatial channel attention gate network using the training set. The specific steps of step S3 are as follows:

[0044] Step S3-1: Randomly flip the input 2D image vertically or horizontally and randomly rotate it from 0 to 360°.

[0045] Step S3-2: Input the 2D image into the encoder of the segmentation network constructed based on the spatial channel attention gate network, and use the RaSE module and downsampling to extract high-level information; use the residual part of the 2D image to extract feature information λ' at different scales, and input the feature information λ' into the channel attention mechanism SE Attention to obtain feature information λ with channel weights at different scales.

[0046] Step S3-3: Input the high-level information into the encoder of FBTS Net, and pass the feature information λ with channel weights from the encoder of the same layer to the decoder by skipping the connection, and restore the image size by upsampling.

[0047] In steps S3-4, the three-layer decoder obtains features ω1, ω2, and ω3 respectively. ω1, ω2, and ω3 are simultaneously input into the multi-scale fusion module to recover the image size through upsampling, resulting in ω′1, ω′2, and ω′3 with a size of 256×256.

[0048] In steps S3-5, redundant information in ω′1, ω′2, and ω′3 is suppressed by spatial attention blocks (SAM+) to obtain spatial attention features ω′′1, ω″2, and ω″3.

[0049] Steps S3-6: Initialize the three weights α, γ, and λ for training. Add ω”1, ω”2, and ω”3 according to their weights, and then perform a 1×1 convolution to obtain the fused feature y, as shown in the following formula:

[0050] y=α×ω”1+γ×ω”2+λ×ω”3 (3);

[0051] Step S3-7: The fused feature y is fed into the classifier to perform multi-classification of the feature information, and a predicted map of the fetal brain tissue is obtained.

[0052] Step S4: Input the fetal brain MR images from the test set into the segmentation model to obtain the predicted map of the fetal brain;

[0053] Step S5: Obtain the evaluation results of the predicted image through multiple evaluation functions. Evaluate the segmentation network based on the evaluation results. If the evaluation is unsatisfactory, update the training parameters of the segmentation network and execute step S3. If the evaluation is satisfactory, the test ends.

[0054] Step S6: Input the image to be segmented into the trained segmentation network to obtain the target fetal brain tissue image.

[0055] Furthermore, in step S1, the FeTA dataset is preprocessed by slicing along the axial plane to obtain several 2D images, which are then divided into training and testing sets at a set ratio of 8:2.

[0056] Specifically, the FeTA dataset was first preprocessed by slicing it along the axial plane to obtain 11,183 2D images of size 256×256, of which 8,946 were used for training and 2,237 were used for testing.

[0057] Furthermore, the encoder of the RaSE module is connected to the decoder through the underlying module.

[0058] Furthermore, the RaSE module satisfies the following formula:

[0059] RS(X)=R(X)×S(R(X)) (1)

[0060] Where X is the input, RS(X) is the final result of the RaSE module, R(X) is the result of the residual block, and S(R(X)) is the channel feature result extracted by the spatial channel attention block with the result of the residual block as input.

[0061] Furthermore, in step S2, SE Net is used as the spatial channel attention block (SE Attention Block) to construct the attention block. SE Net first performs a squeezing operation (i.e., global average pooling operation) on the features to generate channel descriptors, which represent the distribution of channel features. Then, it learns the specific sample activation of each channel through adaptive activation operation (i.e., two fully connected operations).

[0062] Furthermore, the MSF module includes an upsampling block, a spatial attention part, and a feature fusion part. The decoder generates features with the same high resolution as the input image by upsampling the feature maps of semantic information at different levels; and fully represents spatial information by fusing the feature maps obtained by operations such as average pooling, max pooling, and 1×1 convolution.

[0063] Furthermore, SAM+ adds a 1×1 convolution to SAM, and fully represents spatial information by fusing the feature maps obtained from average pooling, max pooling and 1×1 convolution operations.

[0064] Furthermore, the evaluation functions in step S5 include the Dice coefficient, IoU coefficient, Recall (i.e., recall rate), and Precision (i.e., precision rate).

[0065] The specific principles of this invention will be explained in detail below:

[0066] This invention proposes a spatial channel attention-gating network—FBTSNet (Fetal Brain Tissue Segmentation Network)—for the automatic segmentation of fetal brain tissue. Figure 1 As shown, this network consists of feature extraction, feature fusion, and classification components.

[0067] (1) Considering that the encoder-decoder structure will pay more attention to spatial information, in order to enable the network to pay attention to both spatial and channel information, a residual convolution module based on channel attention—the Residual and Squeeze-and-Excitation (RaSE) module—was designed and used for feature extraction by the encoder.

[0068] (2) To enrich the feature information, residual blocks are first used to restore the spatial information of the decoder, thereby reducing information loss. Then, a multi-scale feature fusion (MSF) module based on spatial attention (SAM) is designed to fuse feature maps of different scales from the decoder. MSF restores spatial details more accurately by combining contextual information.

[0069] (3) After the input image undergoes feature extraction and feature fusion, it acquires rich feature information. The classification module uses a 1×1 convolution to divide the results obtained from the feature fusion part into N classes, where the value of N depends on the number of categories of the label.

[0070] Channel attention-based residual convolution modules: Conventional convolutions involve a sequence of two 3×3 convolution operations before each pooling, up-convolution, and over-convolution operation, thus limiting the channel information obtained and potentially leading to feature loss. In contrast, residual structures use skip links to reduce feature loss. SE-Net (Squeeze-and-Excitation) mainly consists of squeezing and excitation operations, such as... Figure 3 As shown, SE Net first performs a squeezing operation (i.e., global average pooling) on ​​the features to generate channel descriptors, which represent the distribution of channel features. Then, it learns specific sample activations for each channel through adaptive activation operations (i.e., two fully connected operations). Furthermore, SE Net's flexibility allows it to be directly combined with other network models, enhancing the network model's ability to select channel weights and thus improving the network model's segmentation performance. Therefore, to fully extract image channel features and enrich semantic information, it is crucial to clarify "what features" are meaningful for a given input image. Figure 2 As shown, this invention proposes the RaSE (Residual and Squeeze-and-Excitation) module. The RaSE module is implemented by using ResBlock (Residual Block, such as...) Figure 2 After the left side, insert an Attention Block (e.g., ... Figure 2The ResBlock is constructed using the right-hand side. It performs initial feature extraction to reduce feature loss. The SE Net is used as the SE Attention Block to construct the Attention Block. The AttentionBlock can further learn the importance of each feature channel, improving useful features and suppressing features useless to the task at hand based on their importance. The RaSE module follows the formula:

[0071] RS(X)=R(X)×S(R(X)) (1)

[0072] Where X is the input, RS(X) is the final result of the RaSE module, R(X) is the result of the ResBlock, and S(R(X)) is the result of the SE Attention Block further extracting channel features with the result of the ResBlock as input.

[0073] Multi-scale feature fusion module: By fusing high-level and low-level semantic information from the decoder, richer information can be used for classification, thereby improving the segmentation performance of the network. However, the low-level semantic features generated by the encoder-decoder structure of the classification network are relatively weak and unsuitable for dense prediction tasks. To address this issue, this invention introduces the MSF model to appropriately combine high-level and low-level features, such as... Figure 5 As shown, N represents the number of categories. The MSF module of this invention is constructed including an upsampling block, a spatial attention part, and a feature fusion part, as described below. First, the decoding process upsamples the feature maps of semantic information at different levels to generate features with the same high resolution as the input image. High-level semantic information is spatially coarse but semantically stronger. To overcome the spatial dimensionality loss caused by upsampling, this invention proposes SAM+ (Spatial attention module+). SAM+ adds a 1×1 convolution to SAM, and fully represents spatial information by fusing feature maps obtained from operations such as average pooling, max pooling, and 1×1 convolution. See details... Figure 4 Finally, feature fusion is achieved through concatenation. All feature maps are summed using trainable weights (initially 0), followed by a 1x1 convolution. The concatenation follows the equation below:

[0074] C(S1,S2,...,S n )=S1×α+S2×γ+...+S n ×λ (2)

[0075] In the formula, C is the cascaded output, α, γ, λ, etc. are Greek numerals that can be trained, and S1 to S2 are the weights. n This refers to the feature mapping after upsampling of feature maps from different decoding layers.

[0076] This invention employs the above technical solutions, using a RaSE module composed of ResNet and SENet. This module not only extracts more image information but also allows the network to focus on more effective information, resulting in better image information extraction. A multi-scale fusion (MSF) module based on spatial attention mechanism enhances the feature fusion capability of the decoder, reduces the interference of redundant information on the segmentation results, and improves the segmentation accuracy of details by combining feature information from multiple scales.

[0077] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A method for segmenting fetal brain tissue based on spatial channel attention gating networks, characterized in that: It includes the following steps: Step S1: After obtaining the FeTA dataset and preprocessing it to obtain several 2D images, divide them into training set and test set according to a set ratio; Step S2: Construct a segmentation network based on a spatial channel attention gate network, including a RaSE module, a multi-scale feature fusion module (MSF), and a classification module arranged sequentially. The RaSE module adopts an encoder-decoder structure classification network and includes a residual block (ResBlock) and an attention block arranged sequentially. The residual block performs preliminary feature extraction to reduce feature information loss. SE Net is used as the attention block to learn the importance of each feature channel, promote useful features based on their importance, and suppress useless features. The multi-scale feature fusion module fuses feature maps of different scales from the decoder of the residual block; The multi-scale feature fusion module includes an upsampling block, a spatial attention part, and a feature fusion part. The spatial attention part adopts the SAM+ module, which adds a 1×1 convolution to the spatial attention module SAM. The classification module divides the result of the feature fusion part into N classes through a 1×1 convolution, where the value of N depends on the number of classes of the label; The RaSE module satisfies the following formula: （1）； in, As input, The final result of the RaSE module, The result for the residual block. The channel feature results are extracted from the spatial channel attention block using the results of the residual block as input. Step S3: Train the segmentation network based on the spatial channel attention gate network using the training set. The specific steps are as follows: Step S3-1: Randomly flip the input 2D image vertically or horizontally and randomly rotate it from 0 to 360°. Step S3-2: Input the 2D image into the encoder of the segmentation network constructed based on the spatial channel attention gate network, and use the RaSE module and downsampling to extract high-level information; use the residual part of the RaSE module to extract feature information at different scales from the 2D image. , feature information The input is fed into the channel attention mechanism SE Attention to obtain feature information with channel weights at different scales. ; Step S3-3: Input the high-level information into the encoder of the FBTS Net, and skip connections to transfer the feature information with channel weights from the encoder at the same layer. The image is passed to the decoder and upsampled to restore its original size. In steps S3-4, the three-layer decoder obtains the features respectively. ,Will Simultaneously, the image size is obtained by upsampling and then inputting it into the multi-scale fusion module. The size is ; Steps S3-5, then... Redundant information in the data is suppressed using spatial attention blocks to obtain spatial attention features. ; Step S3-6: Initialize the three weights to be used in training. ,Will Add them according to their weights, and then... After convolution, the fused features are obtained. The formula is as follows: （3）； Step S3-7: Merge the features The data is fed into a classifier to perform multi-classification of the feature information, resulting in a predicted map of the fetal brain tissue. Step S4: Input the fetal brain MR images from the test set into the segmentation model to obtain the predicted map of the fetal brain; Step S5: Obtain the evaluation results of the predicted image through multiple evaluation functions. Evaluate the segmentation network based on the evaluation results. If the evaluation is unsatisfactory, update the training parameters of the segmentation network and execute step S3. If the evaluation is satisfactory, the test ends. Step S6: Input the image to be segmented into the trained segmentation network to obtain the target fetal brain tissue image.

2. The method for segmenting fetal brain tissue based on spatial channel attention gating networks according to claim 1, characterized in that: In step S1, the FeTA dataset is preprocessed by slicing along the axial plane to obtain several 2D images. These 2D images are then divided into a training set and a test set at a ratio of 8:

2.

3. The method for segmenting fetal brain tissue based on spatial channel attention gating networks according to claim 1, characterized in that: The encoder of the RaSE module is connected to the decoder through the underlying module.

4. The method for segmenting fetal brain tissue based on spatial channel attention gating networks according to claim 1, characterized in that: In step S2, SE Net first performs a squeezing operation on the features to generate channel descriptors, which represent the distribution of channel features; then, through an adaptive activation operation, it learns the specific sample activation for each channel.

5. The method for segmenting fetal brain tissue based on spatial channel attention gating networks according to claim 1, characterized in that: The decoder generates features with the same high resolution as the input image by upsampling the feature maps of semantic information at different levels: the SAM+ module fully represents spatial information by fusing feature maps obtained from operations such as average pooling, max pooling and 1×1 convolution.

6. The method for segmenting fetal brain tissue based on spatial channel attention gating networks according to claim 1, characterized in that: Spatial information is fully represented by feature maps obtained by fusing average pooling, max pooling, and 1×1 convolution operations.

7. The method for segmenting fetal brain tissue based on spatial channel attention gating networks according to claim 1, characterized in that: The evaluation functions in step S5 include Dice coefficient, IoU coefficient, recall, and precision.

Images