A cascade medical image segmentation method based on dual-branch feature fusion

Abstract

The present invention belongs to the field of medical image segmentation technology, specifically a cascade medical image segmentation method based on dual-branch feature fusion, which is used to improve the feature utilization of medical images while reducing network parameters, thereby improving the segmentation quality of medical images. The present invention constructs a cascade medical image segmentation network with dual-branch feature fusion, which mainly includes a dual-branch feature encoder, a feature interaction fusion module (EA-CBAIM) and a decoder. The CNN branch and the VSSD branch in the dual-branch feature encoder respectively obtain multi-resolution features, and the EA-CBAIM feature interaction fusion module realizes important feature enhancement. Finally, the decoder captures and combines feature information of multiple scales, gradually refines the results, and improves the segmentation quality. In summary, the present invention can effectively extract pathological features in medical images, realize automatic segmentation of lesions, and enhance the generalization ability of medical image segmentation models.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and specifically relates to a cascade medical image segmentation method based on dual-branch feature fusion. Background Art

[0002] Medical images are used to study the internal structure, function or pathological state of the human body and some of its organs. They can provide doctors with information such as lesion areas and human organs, thereby supporting clinical decisions such as diagnosis and treatment. However, in the process of medical image analysis, doctors need to spend a lot of time and energy to find the lesion area, which is easily affected by subjective factors and inevitably misses the mark. Therefore, it is very necessary to achieve objective and effective automatic segmentation of medical images.

[0003] Deep learning has been introduced into the field of medical image segmentation due to its powerful learning ability. At present, U-Net network and its variants are widely used in the field of medical image segmentation. It is based on the encoder and decoder structure and uses jump connections to increase information exchange between different scales, thus achieving effective segmentation of medical images in a simple way. However, U-Net is a pure CNN structure. Since CNN pays more attention to features such as texture, it is difficult to capture global context information. To address this problem, Transformer was introduced into medical image segmentation and has been favored by many scholars for its powerful ability to capture global context. However, due to its quadratic complexity and difficulties in image locality and translation invariance, it will affect the segmentation task of medical images. In addition, some decoders integrate local convolutional attention mechanisms to refine the features extracted by the encoder and ultimately generate high-resolution segmentation outputs. However, due to the frequent use of expensive convolution blocks, their computational overhead is large, which limits their application.

[0004] It can be seen that due to the inherent limitations of CNN itself, its ability to extract global image information is limited, while Transformer mostly extracts spatial dependencies and contextual information, and is very complex; at the same time, simple connections may not be able to effectively process details in decoding, and frequent use of convolutional blocks will also lead to increased computational overhead. Summary of the Invention

[0005] The purpose of the present invention is to propose a cascade medical image segmentation method based on dual-branch feature fusion, so as to improve the feature utilization of medical images while reducing network parameters, thereby improving the segmentation quality of medical images; the present invention constructs a cascade medical image segmentation network with dual-branch feature fusion, which mainly includes a dual-branch feature encoder, a feature interaction fusion module (EA-CBAIM) and a decoder. The CNN branch and the VSSD branch in the dual-branch feature encoder respectively obtain multi-resolution features, and the EA-CBAIM feature interaction fusion module realizes important feature enhancement. Finally, the decoder captures and combines feature information of multiple scales, gradually refines the results, and improves the segmentation quality.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A cascade medical image segmentation method based on dual-branch feature fusion, characterized by comprising the following steps:

[0008] Obtain a public medical image dataset and perform data preprocessing on the original images and labeled images in the medical image dataset to form a training set;

[0009] A cascaded medical image segmentation network based on dual-branch feature fusion was established, comprising a dual-branch encoder, a feature interaction fusion module, and a decoder. The dual-branch encoder consisted of a CNN branch and a VSSD branch. The CNN branch and the VSSD branch each extracted multi-resolution features, which were enhanced by the feature interaction fusion module to obtain fused features. The fused features were cascaded and decoded by the decoder to obtain the segmentation result.

[0010] The cascade medical image segmentation network based on dual-branch feature fusion is trained according to the training set to obtain a medical image segmentation model;

[0011] The medical image to be segmented is preprocessed and then input into the medical image segmentation model, which then outputs the segmentation result.

[0012] Furthermore, the data preprocessing process is as follows: the original images in the medical image dataset are cropped to the preset model input size, the cropped original images are converted to tensor type and normalized as input images; the label images in the medical image dataset are cropped to the same size, the cropped label images are converted to tensor type, and together with the input images, they constitute training samples.

[0013] Furthermore, the CNN branch includes: an initial feature extraction module and a first Bottleneck module, a second Bottleneck module, a third Bottleneck module, and a fourth Bottleneck module connected in sequence; the input image x is passed through the initial feature extraction module to obtain a feature map The first to fourth Bottleneck modules serve as the encoding layer of the convolutional neural network, which are composed of 3, 4, 6, and 3 stacked Bottleneck units respectively; the first to fourth Bottleneck modules generate feature maps in turn

[0014] Furthermore, the Bottleneck unit includes three convolutional layers connected in sequence, where the first and second convolutional layers are followed by a BN layer and a ReLU activation function layer, and the third convolutional layer is followed by a BN layer. The input of the Bottleneck is represented as enter After three convolutional layers, the feature map is obtained Feature Map With input Perform residual connection to get the output of Bottleneck

[0015] Furthermore, the VSSD branch includes: a Patch Embedding layer connected in sequence with a first VSSD module, a second VSSD module, a third VSSD module, and an MSA module; the input image x passes through the Patch Embedding layer and divides the input of the first VSSD module into The first to third VSSD modules are composed of 2, 4, and 8 stacked VSSD units respectively, and the MSA module is composed of 4 stacked MSA units. The outputs of the first to third VSSD modules are downsampled respectively; the first to third VSSD modules and MSA modules generate feature maps in turn.

[0016] Furthermore, in the VSSD unit, the input is represented as enter First, pass the convolution layer with a convolution kernel of 3×3, and perform a residual connection between its output and input to obtain the feature map Feature Map Through the LayerNorm layer, the feature map is obtained Feature Map Through the NC-SSD module, in the NC-SSD module, the feature map The output feature map is obtained through the linear layer (Linear) The feature map It is divided into three parts: feature map Feature Map Through the deep convolution layer, and then through the SiLU activation function to obtain the feature map The feature map Divided into three parts, namely feature maps The feature map At the same time, input the NC-SSD unit to obtain the output feature map of the NC-SSD unit Specifically expressed as:

[0017]

[0018] Where D represents the weight coefficient controlling the skip connection, * represents element-by-element multiplication, A is a training parameter, and the shapes of A for the first to third VSSD modules are 2, 4, and 8, respectively.

[0019] Feature Map After passing through the LayerNorm layer Multiply, and then pass the multiplication result through the linear layer to obtain the output feature map of the NC-SSD module Specifically expressed as:

[0020]

[0021] Among them, LN means LayerNorm normalization, and Linear means linearization;

[0022] The output feature map of the NC-SSD module will be With feature map Add together to get the feature map And input LPU unit, LPU unit output feature map Specifically expressed as:

[0023]

[0024] Feature Map Get the feature map through the LayerNorm layer And input it into the MLP module to get the feature map Specifically expressed as:

[0025]

[0026] Among them, Linear represents the linear layer, and GELU represents the GELU activation function;

[0027] Feature Map With feature map Perform residual connection to obtain the output feature map of VSSD unit

[0028] Furthermore, the feature interaction fusion module (EA-CBAIM) includes: 4 feature interaction fusion units, the feature interaction fusion units include: edge attention module (EAM), convolutional attention module (CBAM) and feature interaction module (FIM);

[0029] The feature maps obtained by the CNN branch and the VSSD branch are input into the feature interaction fusion module. For the i-th, i=1, 2, 3, 4-th feature interaction fusion units:

[0030] The feature map of the CNN branch Input edge attention module to get feature map

[0031]

[0032]

[0033] Among them, Conv represents convolution, BN represents batch normalization, and ReLU represents ReLU activation function;

[0034] The feature map of the VSSD branch Input convolutional attention module, which consists of channel attention (CA) and spatial attention (SA). The output feature map of convolutional attention module is

[0035]

[0036] AdaptiveAvgPool represents adaptive average pooling, AdaptiveMaxPool represents adaptive maximum pooling, ReLU represents ReLU activation function, Sigmoid represents Sigmoid activation function, concat represents feature concatenation, Conv represents convolution layer, Mean represents the calculation of the mean along the first dimension of the feature, and Max represents the calculation of the maximum value along the first dimension of the feature;

[0037] The feature map With feature map Input feature interaction module to perform feature interaction fusion, which is specifically expressed as follows:

[0038]

[0039] Among them, AdaptiveMaxPool represents adaptive maximum pooling;

[0040] Finally, the fusion feature map M is obtained i , specifically expressed as:

[0041]

[0042] Among them, Sigmoid represents the Sigmoid activation function.

[0043] Furthermore, the decoder adopts the cascade attention method to fuse multi-scale features and finally predicts the output; in the decoding process, the fusion feature map M 4 The feature map is obtained through the MSCB module Then upsample through the EUCB module to obtain the feature map Feature Map and fusion feature map M 3 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map

[0044] Feature Map As the input of the next level MSCB module, the feature map is obtained through the MSCB module The feature map is obtained through the EUCB module Feature Map and fusion feature map M 2 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map

[0045] Feature Map As the input of the next level MSCB module, the feature map is obtained through the MSCB module The feature map is obtained through the EUCB module Feature Map and fusion feature map M 1 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map

[0046] Feature Map As the input of the next level MSCB module, the feature map is obtained through the MSCB multi-scale convolution block

[0047] The feature map Perform convolution and adjust the number of channels to match the feature map The same; then the convolution results are upsampled by 2 times, 4 times and 8 times respectively, and the upsampled results are compared with the feature map Add element by element to get the feature recovery map M up ;

[0048] The feature recovery map M up Convolution is performed and the convolution result is upsampled by 4 times to obtain the final segmentation output.

[0049] Furthermore, the MSCB module is specifically represented as follows:

[0050]

[0051] in, Represents the input of the MSCB module, PWC represents point-by-point convolution, BN represents batch normalization, R6 represents the ReLu6 activation function, CS represents channel shuffling; MADC represents the MSDC module, which is expressed as:

[0052]

[0053] in, Represents the input of the MSDC module, DWC ks Represents a depthwise convolution with a kernel size of ks, BN represents batch normalization, R6 represents the ReLu6 activation function, and KS represents the convolution kernel set;

[0054] The EUCB module is specifically represented as follows:

[0055]

[0056] in, Represents the input of the EUCB module, DWC represents depthwise convolution, and UP represents upsampling;

[0057] The LGAG module is specifically represented as follows:

[0058]

[0059] Among them, GC represents the group convolution with a convolution kernel of 3×3.

[0060] Furthermore, the specific training process is as follows: setting the loss function, using the AdamW optimizer to complete the training of the cascade medical image segmentation network based on dual-branch feature fusion, and obtaining a medical image segmentation model; using gradient clipping during training, and using structural similarity loss as the loss function.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The present invention relates to an effective cascade medical image segmentation network based on dual-branch feature fusion attention. The network can improve the feature utilization of medical images and accurately segment targets while reducing parameters.

[0063] 2. The medical image network involved proposes an edge attention-convolutional block attention interaction module EA-CBAIM;

[0064] 3. Experiments have demonstrated the effectiveness of the proposed network in medical image segmentation and its good generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Schematic diagram of the process of the cascade medical image segmentation method based on dual-branch feature fusion in the present invention.

[0066] Figure 2 Schematic diagram of the structure of the cascaded medical image segmentation network based on dual-branch feature fusion in the present invention.

[0067] Figure 3 It is a structural diagram of the VSSD module in the present invention.

[0068] Figure 4 It is a structural diagram of the EA-CBAIM feature interaction fusion module in the present invention.

[0069] Figure 5 Schematic diagram of the structure of the MSCB multi-scale convolution block in the present invention.

[0070] Figure 6 Schematic diagram of the structure of the EUCB efficient upconvolution block in the present invention.

[0071] Figure 7 This is a structural diagram of the LGAG large core grouping attention gate mechanism module in the present invention. DETAILED DESCRIPTION

[0072] In order to make the objectives, technical solutions and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments; it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention.

[0073] The present invention proposes a cascade medical image segmentation method based on dual-branch feature fusion, constructs a dual-branch feature fusion cascade medical image segmentation network, mainly including a dual-branch feature encoder, a feature interaction fusion module (EA-CBAIM) and a decoder. The feature encoder is used to extract multi-resolution features, the feature interaction fusion module is used to enhance important features, and the decoder is used to capture and combine information from different scales, gradually refine the results, and improve the segmentation quality. The following is a detailed description with reference to the accompanying drawings and embodiments.

[0074] This embodiment provides a cascade medical image segmentation method based on dual-branch feature fusion, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0075] Step 1: Obtain five public medical image datasets, namely the CVC-300 dataset, the CVC-ClinicDB dataset, the CVC-ClolonDB dataset, the ETIS-LaribPolypDB dataset, and the Kvasir-SEG dataset;

[0076] The training set consists of 900 images from the Kvasir-SEG dataset and 550 images from the CVC-ClinicDB dataset, totaling 1,450 images;

[0077] The test set consists of the following: 100 remaining Kvasir-SEG dataset images, 62 remaining CVC-ClinicDB dataset images, 380 CVC-ClolonDB dataset images, 196 ETIS-LaribPolypDB dataset images, and 60 CVC-300 dataset images;

[0078] Step 2: Preprocess the medical image dataset. Use center cropping to uniformly crop the original images and label images in the medical image dataset to a size of 352×352. Then convert them into tensor types and normalize the original images. However, the label images do not need to be normalized.

[0079] Step 3: Establish a cascade medical image segmentation network based on dual-branch feature fusion, such as Figure 2 As shown, it specifically includes: a dual-branch encoder, a feature interaction fusion module and a decoder;

[0080] 1) For the encoder: The dual-branch encoder includes: CNN branch, VSSD branch and EA-CBAIM feature interaction fusion module. The EAM edge attention module, CBAM convolutional attention module and FIM feature interaction module constitute the EA-CBAIM feature interaction fusion module;

[0081] Furthermore, for the CNN branch;

[0082] The CNN branch includes: an initial feature extraction module and a first Bottleneck module, a second Bottleneck module, a third Bottleneck module, and a fourth Bottleneck module connected in sequence; an input image x∈R H×W×3 , H and W represent the spatial resolution of the input image, and the number of channels of the input image is 3; the input image x is extracted by the initial feature extraction module to obtain the feature map The initial feature extraction module is expressed as:

[0083]

[0084] Where Conv represents convolution, BN represents batch normalization, ReLU represents ReLu activation function, and MaxPool represents maximum pooling. In this embodiment, the input image x first undergoes initial convolution through a convolution layer with a convolution kernel of 7×7, then passes through a BN layer, a ReLu activation function layer, and finally undergoes pooling through a 3×3 maximum pooling layer.

[0085] The first to fourth Bottleneck modules serve as convolutional neural network encoding layers, and are sequentially composed of 3, 4, 6, and 3 Bottleneck units stacked to complete feature extraction. The Bottleneck unit includes: three convolutional layers connected in sequence, wherein the first and second convolutional layers are followed by a BN layer and a ReLU activation function layer, and the third convolutional layer is followed by a BN layer. The input of Bottleneck is represented as enter After three convolutional layers, the feature map is obtained Feature Map With input Perform residual connection to get the output of Bottleneck In this embodiment, the first to fourth Bottleneck modules generate feature maps in sequence The sizes are The number of channels C defaults to 64, and these feature maps contain rich spatial detail information;

[0086] Furthermore, for the VSSD branch;

[0087] The VSSD branch adopts a mainstream pyramid structure and uses the pre-trained weights of VSSD on ImageNet-1K for parameter initialization to accelerate the convergence of the model. The VSSD branch includes: a patchEmbeding layer connected in sequence to the first VSSD module, the second VSSD module, the third VSSD module, and the MSA module (Multi-head Self-Attention).

[0088] The Patch Embedding layer consists of three convolution blocks. The first convolution block is composed of a convolution layer with a 3×3 convolution kernel, a BN layer, and a ReLU activation function. The second convolution block is composed of two convolution layers with 3×3 convolution kernels. The output of the first convolution layer passes through a BN layer and a ReLU activation function, and the output after the second convolution layer only passes through a BN layer. The output of the second convolution block is residually connected with the output of the first convolution block. The third convolution block is composed of a convolution layer with a 3×3 convolution kernel and a convolution layer with a 1×1 convolution kernel. The output of the convolution layer with a 3×3 convolution kernel passes through a BN layer and a ReLU activation function, and the output of the convolution layer with a 1×1 convolution kernel only passes through a BN layer. In this embodiment, the input image x passes through the Patch Embedding layer and is divided into inputs suitable for entering the VSSD module.

[0089] The first to third VSSD modules are respectively composed of 2, 4, and 8 stacked VSSD units, and the MSA module is composed of 4 stacked MSA units. The outputs of the first to third VSSD modules are downsampled respectively. In this embodiment, the first to third VSSD modules and the MSA module generate feature maps in sequence. The sizes are The default number of channels C is 64;

[0090] The VSSD unit is as follows Figure 3 As shown, the input of the VSSD unit is expressed as enter First, pass the convolution layer with a convolution kernel of 3×3, and perform a residual connection between its output and input to obtain the feature map Feature Map Through the LayerNorm layer, the feature map is obtained Feature Map Through the NC-SSD module, in the NC-SSD module, the feature map The output feature map is obtained through the linear layer (Linear) By Split the feature map It is divided into three parts: feature map Feature Map The feature map is obtained by passing the convolution kernel to a 3×3 depth convolution layer and then through the SiLU activation function. By Split the feature map Divided into three parts, namely feature maps The feature map At the same time, the NC-SSD unit (Non-Causal State Space Duality) is input to convert the feature map As the Value value (V), Key value (K), Qurey value (Q) in the traditional Transformer, the output feature map of the NC-SSD unit is obtained Specifically expressed as:

[0091]

[0092] Where D represents the weight coefficient controlling the skip connection, * represents element-by-element multiplication, and A is a training parameter. Specifically, the matrix A is regarded as a scalar, and is used to determine the degree of information added by the current image tag through logarithmic and exponential transformations. Specifically, a uniformly distributed tensor of [1, 16] is first generated, and its logarithm is taken as a trainable parameter. The negative exponential of the trainable parameter is used to obtain A. The shape of A is related to the number of attention heads. In this embodiment, the shapes of A for the first to third VSSD modules are 2, 4, and 8, respectively.

[0093] Feature Map After passing through the LayerNorm layer Multiply, and then pass the multiplication result through the linear layer to obtain the output feature map of the NC-SSD module Specifically expressed as:

[0094]

[0095] Among them, LN means LayerNorm normalization, and Linear means linearization;

[0096] The output feature map of the NC-SSD module will be With feature map Add together to get the feature map And input the local perception unit (LPU) part, the local perception unit includes a convolution layer with a convolution kernel of 3×3, and the local perception unit outputs a feature map Specifically expressed as:

[0097]

[0098] Feature Map Get the feature map through the LayerNorm layer And input it into the MLP module to get the feature map Specifically expressed as:

[0099]

[0100] Among them, Linear represents the linear layer, and GELU represents the GELU activation function;

[0101] Feature Map With feature map Perform residual connection to obtain the output feature map of VSSD unit

[0102] The output of the MSA module is the feature map In the MSA module, the input of the MSA module is first transformed linearly and split along the last dimension into Value (V), Key (K), and Qurey (Q). Next, the output is obtained through the Attention unit. The output of each head is rearranged and passed through the linear layer to obtain the final output. Attention (Q, K, V) is specifically expressed as:

[0103]

[0104] Where d represents the dimension of Q and K, and Softmax represents the normalized exponential function;

[0105] 2) EA-CBAIM feature interaction fusion module;

[0106] The feature maps obtained by the CNN branch and the VSSD branch are input into the EA-CBAIM feature interaction fusion module, which includes: 4 EA-CBAIM units; Figure 4 As shown, for the i-th, i=1, 2, 3, 4 EA-CBAIM units:

[0107] The feature map of the CNN branch The extracted edge information is enhanced by the EAM edge attention module. The edge attention consists of a convolution layer with a 1×1 convolution kernel, batch normalization, and ReLU activation function. At the same time, the obtained output feature map is residually connected with the original input feature map to obtain the feature map. The edge attention is calculated as follows:

[0108]

[0109]

[0110] The feature map of the VSSD branch Through the CBAM convolutional attention module, CBAM consists of two parts: channel attention CA and spatial attention SA. The two parts are sequentially derived along the channel and space dimensions. The feature map is obtained by multiplying the feature map of CA with the original feature map. Afterwards The feature map obtained by SA is Multiply to get the output feature map of the CBAM convolutional attention module The calculation process of CBAM is as follows:

[0111]

[0112] Among them, AdaptiveAvgPool represents adaptive average pooling, AdaptiveMaxPool represents adaptive maximum pooling, ReLU represents ReLU activation function, Sigmoid represents Sigmoid activation function, concat represents feature concatenation, Conv represents convolution layer, Mean represents the calculation of the mean along the first dimension of the feature, and Max represents the calculation of the maximum value along the first dimension of the feature;

[0113] The feature map after edge attention EAM VSSD feature map by CABM Input the FIM feature interaction module for feature interaction fusion. The FIM feature interaction module includes: a convolution layer with a convolution kernel of 1×1 and an adaptive maximum pooling layer; the FIM feature interaction module is specifically expressed as:

[0114]

[0115] Among them, AdaptiveMaxPool represents adaptive maximum pooling;

[0116] Finally, the fusion feature map M is obtained i , specifically expressed as:

[0117]

[0118] 3) For the decoder: The decoder uses cascade attention to fuse multi-scale features and finally predicts the output; in the decoding process, the fusion feature map M 4 The feature map is obtained through the MSCB module Further use the EUCB module to upsample to obtain the feature map Feature Map and fusion feature map M 3 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map

[0119] Feature Map As the input of the next level MSCB multi-scale convolution block, the feature map is obtained through the MSCB module The feature map is obtained through the EUCB module Feature Map and fusion feature map M 2 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map

[0120] Feature Map As the input of the next level MSCB multi-scale convolution block, the feature map is obtained through the MSCB module The feature map is obtained through the EUCB module Feature Map and fusion feature map M 1 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map

[0121] Feature Map As the input of the next level MSCB multi-scale convolution block, the feature map is obtained through the MSCB multi-scale convolution block

[0122] The feature map Perform convolution with a convolution kernel of 1×1 and adjust the number of channels to match the feature map The same; then the convolution results are upsampled by 2 times, 4 times and 8 times respectively, and the upsampled results are compared with the feature map Add element by element to get the feature recovery map M up ;

[0123] The feature recovery map M up Perform convolution with a convolution kernel of 1×1 and upsample the convolution result by 4 times to obtain the final segmentation output;

[0124] Furthermore, for the MSCB module:

[0125] like Figure 5 As shown in the figure, in the MSCB module, a point-by-point convolution layer is used to expand the number of channels, followed by a BN layer and a ReLU6 activation function, and MSDC is used to capture multi-scale and multi-resolution contexts. In order to compensate for the channel relationship that cannot be noticed by the deep convolution in MSDC, a channel shuffling operation is added to enhance the association between channels. Finally, a point-by-point convolution layer and a BN layer are used to encode the dependency between channels. The MSCB module is specifically expressed as follows:

[0126]

[0127] in, Represents the input of the MSCB module, PWC represents point-by-point convolution, BN represents batch normalization, R6 represents ReLu6 activation function, and CS represents channel shuffling; MSDC represents the MSDC module, which is composed of DWC convolutions with different convolution kernels in parallel, specifically expressed as:

[0128]

[0129] in, Represents the input of the MSDC module, DWC ks represents a depthwise convolution with a kernel size of ks, BN represents batch normalization, R6 represents the ReLu6 activation function, KS represents the convolution kernel set, and KS is set to [1, 3, 5];

[0130] Furthermore, for the EUCB module:

[0131] like Figure 6 As shown in the figure, EUCB is an efficient upconvolution block that gradually upsamples the feature map of the current stage to match the dimension and resolution of the feature map of the next jump connection. First, the input of the EUCB module is upsampled by 2 times, and then a depthwise convolution with a convolution kernel of 3×3 is used. The convolution result is passed through a BN layer and a ReLU activation function to enhance the upsampled feature map. Finally, a convolution layer with a convolution kernel of 1×1 is used to reduce the number of channels to match other feature maps. The EUCB module is specifically represented as follows:

[0132]

[0133] in, Represents the input of the EUCB module, DWC represents depthwise convolution, and UP represents upsampling;

[0134] Furthermore, for the LGAG module:

[0135] like Figure 7 As shown in Figure 1, LGAG is a large-core grouped attention gate that is used to gradually combine feature maps with attention coefficients. These coefficients are learned by the network to improve the activation of relevant features and suppress irrelevant features. At the same time, gating signals derived from high-level features are used to control the flow of information at different stages. The LGAG module is specifically expressed as:

[0136]

[0137] Among them, GC represents the group convolution with a convolution kernel of 3×3. The two input feature maps are added after the group convolution and BN layer respectively. Then, the convolution layer with a convolution kernel of 1×1 and the BN layer are used to obtain a single-channel feature map. Finally, the result is obtained by the Sigmoid activation function to obtain the attention coefficient, which is added to the input feature M. i-1 Element-wise multiplication to generate attention gate features Captures greater contextual information at a lower computational cost;

[0138] Step 4: Set the loss function and use the AdamW optimizer to complete the training of the cascade medical image segmentation network based on dual-branch feature fusion to obtain the medical image segmentation model. During the training process, gradient clipping is used to prevent gradient explosion and avoid training instability caused by excessive gradients, so that the model can converge faster. The epoch is set to 200. The loss function uses structural similarity loss, which is specifically expressed as:

[0139]

[0140] in, and Represent the weighted binary cross entropy function and weighted IoU function respectively;

[0141] Step 5: The medical image to be segmented is cropped to a preset size and then input into the medical image segmentation model, which then outputs the segmentation result.

[0142] The performance of the above model is tested using a test set to further illustrate the beneficial effects of the present invention.

[0143] The Dice coefficient and the intersection-over-union (IoU) ratio are used as evaluation indicators to obtain the corresponding model evaluation values. The Dice coefficient is usually used to calculate the similarity between two true values ​​and predicted values, and its value range is [0, 1]. The IoU calculates the intersection-over-union ratio of the true value to the predicted value, and its value range is [0, 1]. The closer to 1, the better the segmentation effect. The calculation formulas of Dice and IoU are as follows:

[0144]

[0145]

[0146] Among them, pred represents the predicted value and true represents the true label value.

[0147] Table 1 shows the comparison of Dice and IoU between the proposed model and U-Net.

[0148] Table 1

[0149]

[0150] As can be seen from the table, without data enhancement, the Dice coefficient and IoU effects of the cascaded medical image segmentation method based on dual-branch feature fusion proposed in the present invention are better; in addition, under the image input of shape (3,352,352), the parameter amount of the present invention is 26.56M, and under the same input, the parameter amount of U-Net is 34.52M. It can be seen that the present invention can obtain similar or even better segmentation results with fewer parameters; in summary, the above results reflect the effectiveness of the cascaded medical image segmentation method based on dual-branch feature fusion proposed in the present invention.

[0151] The above description is only a specific embodiment of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes; all disclosed features, or all steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A cascade medical image segmentation method based on dual-branch feature fusion, characterized in that: The following steps are involved: Obtain a public medical image dataset and perform data preprocessing on the original images and labeled images in the medical image dataset to form a training set; A cascaded medical image segmentation network based on dual-branch feature fusion was established, comprising a dual-branch encoder, a feature interaction fusion module, and a decoder. The dual-branch encoder consisted of a CNN branch and a VSSD branch. The CNN branch and the VSSD branch each extracted multi-resolution features, which were enhanced by the feature interaction fusion module to obtain fused features. The fused features were cascaded and decoded by the decoder to obtain the segmentation result. The feature interaction fusion module includes: 4 feature interaction fusion units, and the feature interaction fusion units include: edge attention module, convolution attention module and feature interaction module; The feature maps obtained by the CNN branch and the VSSD branch are input into the feature interaction fusion module. For the i-th, i=1, 2, 3, 4-th feature interaction fusion units: The feature map of the CNN branch Input edge attention module to get feature map Among them, Conv represents convolution, BN represents batch normalization, and ReLU represents ReLU activation function; The feature map of the VSSD branch Input convolutional attention module, which consists of channel attention and spatial attention. The output feature map of convolutional attention module is Among them, AdaptiveAvgPool represents adaptive average pooling, AdaptiveMaxPool represents adaptive maximum pooling, ReLU represents ReLU activation function, Sigmoid represents Sigmoid activation function, concat represents feature concatenation, Conv represents convolution layer, Mean represents the calculation of the mean along the first dimension of the feature, and Max represents the calculation of the maximum value along the first dimension of the feature; The feature map With feature map Input feature interaction module to perform feature interaction fusion, which is specifically expressed as follows: Among them, AdaptiveMaxPool represents adaptive maximum pooling; Finally, the fusion feature map M is obtained i , specifically expressed as: Among them, Sigmoid represents the Sigmoid activation function; The decoder adopts the cascade attention method to fuse multi-scale features and finally predicts the output; in the decoding process, the fusion feature map M 4 The feature map is obtained through the MSCB module Then upsample through the EUCB module to obtain the feature map Feature Map and fusion feature map M 3 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map Feature Map As the input of the next level MSCB module, the feature map is obtained through the MSCB module The feature map is obtained through the EUCB module Feature Map and fusion feature map M 2 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map Feature Map As the input of the next level MSCB module, the feature map is obtained through the MSCB module The feature map is obtained through the EUCB module Feature Map and fusion feature map M 1 At the same time, input the LGAG module to obtain the feature map The feature map With feature map Add to get the feature map Feature Map As the input of the next level MSCB module, the feature map is obtained through the MSCB multi-scale convolution block The feature map i=2,3,4 for convolution, and adjust the number of channels to match the feature map The same; then the convolution results are upsampled by 2 times, 4 times and 8 times respectively, and the upsampled results are compared with the feature map Add element by element to get the feature recovery map M up ; The feature recovery map M up Perform convolution and upsample the convolution result by 4 times to obtain the final segmentation output; The cascade medical image segmentation network based on dual-branch feature fusion is trained according to the training set to obtain a medical image segmentation model; The medical image to be segmented is preprocessed and then input into the medical image segmentation model, which then outputs the segmentation result.

2. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 1, characterized in that: The data preprocessing process is as follows: the original images in the medical image dataset are cropped to the preset model input size, the cropped original images are converted into tensor types and normalized as input images; The label images in the medical image dataset are cropped to the same size, and the cropped label images are converted into tensor types, which together with the input images constitute training samples.

3. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 1, characterized in that: The CNN branch includes an initial feature extraction module and a first Bottleneck module, a second Bottleneck module, a third Bottleneck module, and a fourth Bottleneck module connected in sequence; an input image x is passed through the initial feature extraction module to obtain a feature map The first to fourth Bottleneck modules serve as the encoding layer of the convolutional neural network, which are composed of 3, 4, 6, and 3 stacked Bottleneck units respectively; the first to fourth Bottleneck modules generate feature maps in turn i=1,2,3,4.

4. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 3, characterized in that: The Bottleneck unit consists of three convolutional layers connected in sequence, where the first and second convolutional layers are followed by a batch normalization layer and a ReLU activation function layer, and the third convolutional layer is followed by a batch normalization layer. The input of the Bottleneck is represented as enter After three convolutional layers, the feature map is obtained Feature Map With input Perform residual connection to get the output of Bottleneck 5. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 1, characterized in that: The VSSD branch includes: a Patch Embedding layer connected in sequence with a first VSSD module, a second VSSD module, a third VSSD module, and an MSA module; the input image x passes through the Patch Embedding layer and divides the input of the first VSSD module into The first to third VSSD modules are composed of 2, 4, and 8 stacked VSSD units respectively, and the MSA module is composed of 4 stacked MSA units. The outputs of the first to third VSSD modules are downsampled respectively; the first to third VSSD modules and MSA modules generate feature maps in turn. i=1,2,3,4.

6. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 5, characterized in that: In the VSSD unit, the input is represented as enter First, pass the convolution layer with a convolution kernel of 3×3, and perform a residual connection between its output and input to obtain the feature map Feature Map Through the LayerNorm layer, the feature map is obtained Feature Map Through the NC-SSD module, in the NC-SSD module, the feature map The output feature map is obtained through the linear layer The feature map It is divided into three parts: feature map Feature Map Through the deep convolution layer, and then through the SiLU activation function to obtain the feature map The feature map Divided into three parts, namely feature maps The feature map At the same time, input the NC-SSD unit to obtain the output feature map of the NC-SSD unit Specifically expressed as: Where D represents the weight coefficient controlling the skip connection, * represents element-by-element multiplication, A is a training parameter, and the shapes of A for the first to third VSSD modules are 2, 4, and 8, respectively. Feature Map After passing through the LayerNorm layer Multiply, and then pass the multiplication result through the linear layer to obtain the output feature map of the NC-SSD module Specifically expressed as: Among them, LN represents LayerNorm normalization, and Linear represents linear layer; The output feature map of the NC-SSD module will be With feature map Add together to get the feature map And input LPU unit, LPU unit output feature map Specifically expressed as: Feature Map Get the feature map through the LayerNorm layer And input it into the MLP module to get the feature map Specifically expressed as: Among them, Linear represents the linear layer, and GELU represents the GELU activation function; Feature Map With feature map Perform residual connection to obtain the output feature map of VSSD unit 7. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 1, characterized in that: The MSCB module is specifically represented as follows: in, Represents the input of the MSCB module, PWC represents point-by-point convolution, BN represents batch normalization, R6 represents the ReLu6 activation function, CS represents channel shuffling; MSDC represents the MSDC module, which is expressed as: in, Represents the input of the MSDC module, DWC ks Represents a depthwise convolution with a kernel size of ks, BN represents batch normalization, R6 represents the ReLu6 activation function, and KS represents the convolution kernel set; The EUCB module is specifically represented as follows: in, Represents the input of the EUCB module, DWC represents depthwise convolution, and UP represents upsampling; The LGAG module is specifically represented as follows: Among them, GC represents the group convolution with a convolution kernel of 3×3.

8. The cascade medical image segmentation method based on dual-branch feature fusion according to claim 1, characterized in that: The specific training process is as follows: setting the loss function, using the AdamW optimizer to complete the training of the cascade medical image segmentation network based on dual-branch feature fusion, and obtaining the medical image segmentation model; using gradient clipping during training, and using structural similarity loss as the loss function.

Images