Medical image segmentation method based on axial MLP and spatial cross-gating

Abstract

The application provides a medical image segmentation method based on an axial MLP and a spatial cross-gating, and the method comprises the following steps: constructing a medical image segmentation model MSAFNet based on an axial MLP and a spatial cross-gating; obtaining a medical image dataset, training the medical image segmentation model based on the axial MLP and the spatial cross-gating, and obtaining a trained medical image segmentation model based on the axial MLP and the spatial cross-gating; obtaining a medical image to be segmented; inputting the medical image to be segmented into the trained medical image segmentation model based on the axial MLP and the spatial cross-gating, performing image segmentation on the medical image to be segmented, and obtaining a target segmentation image. The application mixes multi-scale features through a multi-axis hybrid residual channel attention block MX-RCAB, pays attention to local details and global dependency, filters redundant information through a spatial cross-gating block SCGB, and captures bottom-level detail information with discriminative features, so that the problem of low precision of an existing medical image segmentation method is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing, and particularly relates to a medical image segmentation method based on an axial MLP and a spatial cross-gated. BACKGROUND

[0002] Medical image segmentation plays an important role in modern medicine, and it can assist doctors in making more efficient and accurate decisions in disease diagnosis, treatment planning and surgical guidance. However, medical image data is complex and diverse, and images produced by different imaging devices such as CT, MRI and ultrasound have strong differences. Images from the same device also have large differences due to individual differences between patients and different lesion morphologies, and there are often artifacts and noise in the images, which increase the difficulty of medical image segmentation. In addition, medical image segmentation requires image labeling, which needs to be completed manually by professional doctors, which is not only time-consuming and laborious, but also highly dependent on the personal experience of doctors. Therefore, it is still a challenging task to achieve high-precision automatic image segmentation in a complex medical environment.

[0003] In recent years, medical image segmentation methods based on deep learning have made significant progress. In the CNN framework, models represented by UNet and its variants use encoder-decoder structures and powerful local feature extraction capabilities to significantly improve segmentation performance. However, CNN-based methods are limited by the size of the local receptive field, and there are deficiencies in capturing long-distance context features. In order to solve this problem, the Transformer method based on self-attention mechanism emerged as the times require, which can effectively overcome the limitations in computer vision tasks, but due to the huge parameter quantity it brings, it limits its application in high-dimensional medical image segmentation. Recently, multi-layer perceptron with token mixing MLP blocks has achieved amazing performance on large-scale datasets. MLP-based models have attracted attention due to their simple architecture, but they have the problem of insufficient generalization ability in small data set medical image segmentation tasks.

[0004] However, although the current popular medical image segmentation network has made some progress, it is difficult to achieve a good balance between network complexity and segmentation accuracy, and existing methods still have deficiencies in capturing image details and handling image diversity, and there is a problem of low segmentation accuracy.

[0005] Therefore, how to improve the medical image segmentation accuracy is a problem to be solved in the field. SUMMARY

[0006] The medical image segmentation method, system and device based on axial MLP and spatial cross-gating are provided to solve the defects of the prior art.

[0007] To achieve the above object, the present application adopts the following technical scheme:

[0008] The medical image segmentation method based on axial MLP and spatial cross-gating is provided, and the method comprises the following steps:

[0009] S1, a medical image segmentation model MSAFNet based on axial MLP and spatial cross-gating is constructed; the medical image segmentation model comprises an encoder and a decoder, the encoder comprises a CNN block and a multi-axis hybrid residual channel attention block MX-RCAB, and the decoder comprises a spatial cross-gating block SCGB and a decoder block DecoderBlock;

[0010] S2, a medical image data set is obtained, the medical image segmentation model based on axial MLP and spatial cross-gating is trained, and a trained medical image segmentation model based on axial MLP and spatial cross-gating is obtained;

[0011] S3, a medical image to be segmented is obtained;

[0012] S4, the medical image to be segmented is input into the trained medical image segmentation model based on axial MLP and spatial cross-gating, the medical image to be segmented is subjected to image segmentation, and a target segmentation image is obtained.

[0013] Further, in the medical image segmentation model MSAFNet based on axial MLP and spatial cross-gating, the CNN block is used to extract multi-scale hierarchical feature maps x land input the extracted hierarchical feature maps into a multi-axis mixed residual channel attention block MX-RCAB for global information interaction in space and channels; the CNN block is composed of multiple convolution layers, pooling layers and activation functions.

[0014] Further, the multi-axis mixed residual channel attention block MX-RCAB includes two branches in space and channels, in the space branch, a multi-axis feature mixing MLP is used to fuse local and global features; in the channel branch, a residual channel attention module RCAB is used to enhance channel features.

[0015] Then, the feature maps processed by the space branch and the channel branch are spliced, and all channels are fused through the input residual connection to obtain the output features of the encoder.

[0016] Further, in the space branch, a multi-axis feature mixing MLP is used to fuse local and global features, specifically including:

[0017] For input features x l ∈R C×H×W , l∈{1, 2, …, L}, where C, H, W represent the channel number, height and width of the input feature map respectively, L represents the depth of the encoder, which is divided into N non-overlapping windows according to the block manner of the Vision Transformer, and the shape of each window is C×g×g, where g is the window size; then, the feature map is divided into local branch and global branch along the channel dimension, and each branch uses a transpose MLP attention mechanism Transpose MLPAttention to enhance feature interaction.

[0018] In Transpose MLPAttention, the input feature X∈R C×H×W is divided into X A and X B along the channel dimension, where X A ,X B ∈R C / 2×H×W ; then, X A is processed to generate attention weights, and the specific process is as follows:

[0019] X A =Softmax(W2·ReLU(W1·X A +b1)+b2)

[0020] Where W1, W2 are the weight matrices of the MLP, b1, b2 are the bias terms, ReLU is the activation function, and Softmax function is used for normalization to generate attention weights X A ; subsequently, the calculated attention weights XA X after applying linear transformation B The weighted operation is performed as follows:

[0021]

[0022] where ⊙ denotes element-wise multiplication, is the weighted feature map, W3 is the weight matrix of the linear transformation layer, and b3 denotes the bias value of the linear layer.

[0023] Further, in the channel branch, a residual channel attention module RCAB is used to enhance the channel features, specifically including:

[0024] For the input feature x l ∈R C×H×W , l ∈ {1, 2,..., L}, a channel attention weight is generated through a convolution layer, an MLP layer and a Squeeze-Excitation module, and then the attention weight is multiplied with the original feature map in the channel dimension; in addition, the addition of residual connection allows the input feature map to be added to the weighted feature map; the process is represented as:

[0025]

[0026] SE(X) = σ(W fc ((Pool(X)))) ⊙ X

[0027] where LN denotes layer normalization, MLP denotes a multi-layer perceptron layer, SE denotes a Squeeze-Excitation module, Pool denotes a global average pooling operation, W fc denotes the weight matrix of the fully connected layer, σ is a sigmoid activation function, ⊙ denotes element-wise multiplication, X denotes the input feature of the Squeeze-Excitation module, is the output feature map enhanced by the residual channel attention mechanism.

[0028] Further, the input of the spatial cross-gating block SCGB is composed of the features X and Y output by the encoder and the decoder, which are respectively aligned in dimension through a convolution layer and produce corresponding feature maps; then, preliminary feature activation is performed:

[0029] X act = σ(FC(LN(Conv(X))))

[0030] Y act = σ(FC(LN(Conv(Y))))

[0031] where Conv is a 3x3 convolution layer, LN is layer normalization, FC is a fully connected layer, and σ is a GELU activation function.

[0032] The core component of SCGB is the cross-gating module CGB, which applies a linear layer to the block-wise serialized windows to generate query Q, key K and value V, respectively.

[0033] Q X ,K X ,V X =Linear(X act )

[0034] Q Y ,K Y ,V Y =Linear(Y act )

[0035] Then, the dot product between query Q and key K is calculated, and after the Softmax activation function, the generated attention weight is weighted summed with value V to obtain the new feature representation:

[0036]

[0037]

[0038] where d k is a scaling factor to balance the size of the dot product result;

[0039] In addition, residual connection is introduced for data transmission; in order to ensure that the output channel has the same channel size as the input X and Y, a fully connected layer is used to complete the linear transformation, which is specifically represented as:

[0040] Output x =W4(X act ⊙Attention Y )+X

[0041] Output y =W5(Y act ⊙Attention x )+Y

[0042] where ⊙ represents element-wise dot product, W4, W5 represent the corresponding MLP linear layer mapping features, X act and Y act represent the features after preliminary activation.

[0043] Further, the decoder block DecoderBlock gradually recovers the spatial details of the target image and enhances the feature expression by upsampling the low-resolution feature map output by the decoder and pixel-by-pixel fusing the corresponding resolution feature map from the encoder with the upsampled feature map; the upsampling operation is realized by a bilinear interpolation method to expand the low-resolution feature map to a higher resolution.

[0044] Further, when training the medical image segmentation model based on the axial MLP and the spatial cross-gated,

[0045] The loss function is composed of cross-entropy loss CELoss and Dice similarity coefficient loss DiceLoss, and the total loss function is defined as:

[0046] L = 0.5L CE + 0.5L Dice

[0047] The cross-entropy loss CELoss is used to measure the difference between the predicted value and the true label, and is suitable for semantic segmentation tasks of pixel-level classification, and the calculation formula is as follows:

[0048]

[0049] Where N represents the number of samples in a batch, C represents the total number of classifications, y ij represents the true label of the i-th sample in the j-th class, p ij represents the predicted probability of the i-th label in the j-th class.

[0050] The Dice similarity coefficient loss DiceLoss is based on the Dice similarity coefficient and is used to evaluate the overlap between the segmentation result and the true label, and the calculation formula is as follows:

[0051]

[0052] Where p i is the segmentation probability predicted by the model, g i is the true segmentation label.

[0053] The application also proposes a medical image segmentation system based on the axial MLP and the spatial cross-gated, characterized in that the medical image segmentation system executes the medical image segmentation method based on the axial MLP and the spatial cross-gated.

[0054] The application also proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the above method.

[0055] Compared with the prior art, the following beneficial effects are achieved:

[0056] 1. The medical image segmentation method based on the axial MLP and the spatial cross-gating of the present application inputs a medical image to be segmented into a trained medical image segmentation model based on the axial MLP and the spatial cross-gating, performs image segmentation on the medical image to be segmented, and obtains a target segmentation image; the medical image segmentation model based on the axial MLP and the spatial cross-gating comprises a multi-axis mixed residual channel attention block MX-RCAB and a spatial cross-gating block SCGB, thereby improving the accuracy of medical image segmentation.

[0057] 2. The medical image segmentation method based on the axial MLP and the spatial cross-gating of the present application, the multi-axis mixed residual channel attention block MX-RCAB captures more global information of a large receptive field while retaining the integrity of details through a global and local feature fusion module and a residual channel reweighting mechanism, improves the feature discrimination ability of the model when processing complex structures, and further improves the accuracy of image segmentation.

[0058] 3. The medical image segmentation method based on the axial MLP and the spatial cross-gating of the present application, the spatial cross-gating block SCGB module adaptively fuses the low-level features of the encoder and the high-level features of the decoder through a cross-gating mechanism, effectively filters redundant information and retains key features, improves the utilization rate of bottom-level features, and further improves the accuracy of image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0060] Figure 1 The medical image segmentation method based on the axial MLP and the spatial cross-gating provided by the embodiments of the present application is shown in the schematic diagram.

[0061] Figure 2 The network structure diagram of the medical image segmentation model MSAFNet based on the axial MLP and the spatial cross-gating of the present application is shown.

[0062] Figure 3 The structure diagram of the multi-axis mixed residual channel attention block MX-RCAB provided by the embodiments of the present application is shown.

[0063] Figure 4 The structure diagram of the MLP attention mechanism T-MLP-Attn provided by the embodiments of the present application is shown.

[0064] Figure 5 A structural diagram of the SE module provided by the embodiment of the present application.

[0065] Figure 6 A structural diagram of the spatial cross-gating block SCGB provided by the embodiment of the present application.

[0066] Figure 7 A visual result diagram of the qualitative comparison of different methods provided by the embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0068] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0069] The present application will be further described below with reference to the drawings and specific embodiments, but not as a limitation of the present application.

[0070] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, the singular forms “a”, “an” and “the” are intended to include the plural forms, unless the context clearly indicates otherwise.

[0071] It should be further understood that the term “and / or” used in the specification and the appended claims of the present application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0072] The present application proposes a medical image segmentation method based on axial MLP and spatial cross-gating. As shown in the figure, the medical image segmentation method based on axial MLP and spatial cross-gating includes the following steps S1 to S4. Figure 1

[0073] ​S1, constructing a medical image segmentation model MSAFNet based on an axial MLP and a spatial cross-gating; the medical image segmentation model comprises an encoder and a decoder, the encoder comprises a CNN block, a multi-axis mixed residual channel attention block MX-RCAB, and the decoder comprises a spatial cross-gating block SCGB and a decoder block DecoderBlock.

[0074] The network structure of the medical image segmentation model MSAFNet based on the axial MLP and the spatial cross-gating is as shown in the figure. Figure 2 The model adopts an encoder-decoder architecture, combines the feature extraction of a CNN, the global information capture of an MX-RCAB, and the feature fusion capability of an SCGB. In the encoder, multi-scale hierarchical features are extracted through a CNN block (CNNBlock), and global information interaction in space and channels is performed in an MX-RCAB. In the decoder, the SCGB module effectively fuses multi-level features, and a high-resolution segmentation result is generated in combination with the up-sampling features of a decoder block (DecoderBlock). This design improves the precision and robustness of the model in complex medical image segmentation tasks.

[0075] Encoder part

[0076] CNN block

[0077] In the medical image segmentation model MSAFNet based on the axial MLP and the spatial cross-gating, the CNN block extracts multi-scale hierarchical feature maps x l from the input medical image layer by layer, and inputs the extracted hierarchical feature maps into a multi-axis mixed residual channel attention block MX-RCAB for global information interaction in space and channels; the CNN block is composed of multiple convolution layers, pooling layers, and activation functions.

[0078] Specifically, in the encoder part of MSAFNet, the CNNBlock serves as a basic feature extraction module and is responsible for extracting low-level features from the input medical image layer by layer. The module is composed of multiple convolution layers, pooling layers, and activation functions, and through stepwise down-sampling, the spatial resolution of the input image is reduced while the number of channels is increased, and different scale feature maps x l are preliminarily extracted, where L represents the depth of the encoder.

[0079] Multi-axis mixed residual channel attention block (MX-RCAB)

[0080] The multi-axis mixed residual channel attention block MX-RCAB includes two branches of space and channel, in the space branch, the multi-axis feature fusion MLP is used to fuse local and global features; in the channel branch, the residual channel attention module RCAB is used to enhance the channel features; then, the feature maps processed by the space branch and the channel branch are spliced, and all channels are fused through the input residual connection to obtain the output features of the encoder. The structure of the multi-axis mixed residual channel attention block MX-RCAB is as shown in Figure 3 .

[0081] Specifically, in the space branch, the multi-axis feature fusion MLP (MAF-MLP) is used to fuse local and global features, specifically including:

[0082] For the input feature x l ∈R C×H×W ,l∈{1,2,…,L}, wherein C, H, and W represent the channel number, height, and width of the input feature map respectively, L represents the depth of the encoder, which is divided into N non-overlapping windows in the block manner of the Vision Transformer, and the shape of each window is C×g×g, wherein g is the window size; then, the feature map is divided into a local branch and a global branch along the channel dimension, and each branch separately uses the transpose MLP attention mechanism (T-MLP-Attn) to enhance feature interaction.

[0083] The local branch improves the model's perception of local information by using the attention mechanism within the window, and the global branch strengthens the representation of global features by using the attention mechanism between windows. For feature maps of different sizes, the window size g is set flexibly to ensure that the model can effectively mix global and local features, thereby improving the accuracy of overall feature expression.

[0084] The structure of the MLP attention mechanism (T-MLP-Attn) is as shown in Figure 4 .

[0085] In the T-MLP-Attn, the input feature X∈R C×H×W is divided into X A and X B along the channel dimension, wherein X A ,X B ∈R C / 2×H×W ; then, X A is processed to generate attention weights, and the specific process is as follows:

[0086] XA = Softmax(W2 ReLU(W1 X + b1) + b2) A

[0087] where W1, W2 are the weight matrices of the MLP, b1, b2 are the bias terms, ReLU is the activation function, and the Softmax function is used for normalization to generate attention weights X A ; then, the calculated attention weights X A are applied to the linearly transformed X B , to perform a weighting operation:

[0088]

[0089] where denotes element-wise multiplication, is the weighted feature map, W3 is the weight matrix of the linear transformation layer, and b3 represents the bias value of the linear layer.

[0090] The MLP-based attention mechanism captures high-order relationships in spatial dimensions with a small number of parameters by applying linear transformations in different spatial dimensions.

[0091] In the channel branch, a residual channel attention module RCAB is used to enhance channel features, specifically including:

[0092] For input feature x l ∈R C×H×W , l ∈ {1, 2,..., L}, a channel attention weight is generated after passing through a convolution layer, an MLP layer, and a Squeeze-Excitation module, and then the attention weight is multiplied element-wise with the original feature map in the channel dimension. In addition, the addition of a residual connection allows the input feature map to be added to the weighted feature map. The process is represented as:

[0093]

[0094] SE(X) = σ(W fc ((Pool(X)))) 0 X

[0095] where LN represents layer normalization, MLP represents a multi-layer perceptron layer, SE represents a Squeeze-Excitation module, Pool represents a global average pooling operation, W fc represents the weight matrix of the fully connected layer, σ is the sigmoid activation function, 0 represents element-wise multiplication, X represents the input feature of the Squeeze-Excitation module, is the output feature map enhanced by the residual channel attention mechanism.

[0096] ​The attention weight is multiplied with the original feature map element by element in the channel dimension, thereby adjusting the weight of each channel; in addition, the addition of the residual connection allows the input feature map to be added to the weighted feature map, which is conducive to preserving the original feature information.

[0097] Finally, the spatial branch and the channel branch processed feature maps are spliced, and all channels are fused through the input residual connection to obtain the output feature of the encoder. This process not only optimizes the overall feature representation, but also enables the model to better capture the subtle features and diversity of different organs, ensuring robustness in various anatomical structures.

[0098] Decoder part

[0099] The spatial cross-gate block SCGB has a structure diagram as shown in Figure 6 .

[0100] The skip connection is a common feature interaction method in visual models and is very important for improving model performance. However, directly passing the features of the encoder to the decoder may cause noise propagation, and directly connecting high-resolution features and low-resolution features may affect the full fusion of high-level semantic information and low-level detail information, thereby affecting the accuracy of the segmentation result. Based on this, the present application proposes a new SCGB module, which effectively filters noise and optimizes feature fusion by introducing a cross-gate mechanism in the spatial dimension, thereby improving the recognition and segmentation accuracy of organ boundaries.

[0101] The input of the spatial cross-gate block SCGB is composed of the features X and Y output by the encoder and the decoder, which are aligned in dimension through convolution layers and generate corresponding feature maps; then, preliminary feature activation is performed:

[0102] X act =σ(FC(LN(Conv(X))))

[0103] Y act =σ(FC(LN(Conv(Y))))

[0104] Where Conv is a 3x3 convolution layer, LN is layer normalization, FC is a fully connected layer, and σ is a GELU activation function.

[0105] The core component of the SCGB is the cross-gate module CGB, which retains the original channel features of the feature map. The cross-gate module applies a linear layer to the blocked serial window to generate query Q, key K, and value V, respectively.

[0106] Q X ,K X ,V X =Linear(Xact )

[0107] Q Y ,K Y ,V Y =Linear(Y act )

[0108] Then, the dot product between the query Q and the key K is calculated, and after the Softmax activation function, the generated attention weight is weighted summed with the value V to obtain the new feature representation:

[0109]

[0110]

[0111] where d k is a scaling factor to balance the size of the dot product result;

[0112] This process realizes the cross-gating of X and Y features in the spatial dimension, enhances the interaction between features, and enables efficient information transmission and fusion between the two feature maps.

[0113] In addition, residual connections are introduced for data transmission; in order to ensure that the output channel has the same channel size as the input X and Y, a fully connected layer is used to complete the linear transformation, which is specifically represented as:

[0114] Output x =W4(X act ⊙Attention Y )+X

[0115] Output y =W5(Y act ⊙Attention x )+Y

[0116] where ⊙ represents element-wise dot product, W4, W5 represent the corresponding MLP linear layer mapping features, X act and Y act represent the features after preliminary activation.

[0117] Residual connections are introduced for data transmission to ensure better training effect of the model.

[0118] Decoder block DecoderBlock

[0119] The decoder block DecoderBlock gradually recovers the spatial details of the target image and enhances the feature expression by upsampling the decoder output low-resolution feature map and pixel-by-pixel fusing the corresponding resolution feature map from the encoder with the upsampled feature map; the upsampling operation is realized by a bilinear interpolation method to expand the low-resolution feature map to a higher resolution.

[0120] With the upsampling, the feature map gradually recovers to the spatial size of the original medical image, and by fusing the high-level semantic features and the low-level detail information, the finally generated feature map can more accurately locate the key regions in the image while retaining the global information.

[0121] S2, obtain a medical image data set, train the medical image segmentation model based on the axial MLP and the spatial cross-gating to obtain the trained medical image segmentation model based on the axial MLP and the spatial cross-gating.

[0122] In a specific embodiment, the data set includes Synapse and ACDC.

[0123] The Synapse data set contains 30 patients and 3779 axial contrast-enhanced abdominal clinical CT images. Each CT volume is composed of 85-198 512x512 pixel slices, and the voxel spatial resolution is ([0.54-0.54]x[0.98-0.98]x[2.5-5.0]) mm3. The data set is divided in the same way as TransUNet, with 18 cases for model training and 12 cases for model testing. The ACDC data set collects MRI scans of 100 patients. A series of short-axis slices cover the heart from the bottom of the left ventricle to the top of the ventricle, with a slice thickness of 5 to 8 mm. The spatial resolution in the short-axis plane is 0.83 to 1.75 mm2 / pixel. Each patient scan is manually annotated for the left ventricle (LV), right ventricle (RV), and myocardium (MYO). In the experiment, 70 patients are used for training, 10 for validation, and 20 for testing.

[0124] In a specific embodiment, when training the medical image segmentation model based on the axial MLP and the spatial cross-gating, the loss function is composed of two parts, the cross-entropy loss CELoss and the Dice similarity coefficient loss DiceLoss, and the total loss function is defined as:

[0125] L = 0.5L CE + 0.5L Dice

[0126] Cross-entropy loss CELoss is used to measure the difference between the predicted value and the true label, which is suitable for semantic segmentation tasks of pixel-level classification, and its calculation formula is as follows:

[0127]

[0128] Where N represents the number of samples in a batch, C represents the total number of classifications, y ij represents the true label of the i th sample in the j th class, p ij represents the predicted probability of the i th label in the j th class.

[0129] Dice similarity coefficient loss DiceLoss is based on the Dice similarity coefficient, which is used to evaluate the overlap between the segmentation result and the true label, and its calculation formula is as follows:

[0130]

[0131] Where p i is the segmentation probability predicted by the model, g i is the true segmentation label.

[0132] S3, obtaining a medical image to be segmented.

[0133] The medical image to be segmented can be an image that needs to be segmented. Specifically, the medical image to be segmented includes a target segmentation object; in a specific embodiment, the target segmentation object includes an aorta, a gallbladder, a left kidney, a right kidney, a liver, a pancreas, a spleen, a stomach, etc.

[0134] S4, inputting the medical image to be segmented into the trained medical image segmentation model based on axial MLP and spatial cross-gate, performing image segmentation on the medical image to be segmented, and obtaining a target segmentation image.

[0135] In a specific embodiment, the target segmentation image includes an aorta, a gallbladder, a left kidney, a right kidney, a liver, a pancreas, a spleen, a stomach, etc.

[0136] The medical image segmentation model based on axial MLP and spatial cross-gate of the application effectively mixes multi-scale features in space and channels through a multi-axis mixed residual channel attention block (MX-RCAB), solving the problem of high computational complexity when capturing long-distance dependencies of complex medical images. At the same time, a spatial cross-gate block (SCGB) is designed to solve the problem of boundary detail loss caused by feature mismatch in traditional skip connection, ensuring the full integration of high-level semantic features and low-level detail features, and further enhancing the generalization ability of the model.

[0137] Experimental method and analysis

[0138] To verify the effectiveness of the medical image segmentation model based on axial MLP and spatial cross-gating of the application, experiments are conducted on two public datasets, Synapse and ACDC.

[0139] To accurately evaluate the performance of segmentation, the application uses the Dice Similarity Coefficient (DSC) and the Hausdorff Distance (HD) as the main evaluation indicators, which are defined as follows:

[0140]

[0141] where A represents the predicted segmentation result, B represents the true segmentation label, |A∩B| represents the intersection area of the predicted result and the true label, |A| and |B| represent the area of the predicted result and the true label respectively. The value of DSC ranges from 0 to 1, and the larger the value, the closer the predicted result is to the true label.

[0142]

[0143] where d(a,b) represents the Euclidean distance between points a and b, sup and inf represent the supremum and infimum respectively. HD measures the farthest error distance between the predicted result and the true label, and the smaller the value, the closer the predicted result is to the true label. To exclude unreasonable distances caused by some outliers and maintain the stability of the overall value, the application uses HD95 (the top 95% of distances from small to large) for performance evaluation.

[0144] All experiments are conducted on an NVIDIA RTX 3090 GPU, and the training and testing of the model are implemented using the PyTorch framework. During model training, simple data augmentation such as random rotation and flipping is performed on the samples in the dataset to improve the robustness of the model; the ResNet50 in the encoder is initialized using pre-trained weights on ImageNet to speed up the convergence of the model. The input resolution of the model is 224x224; according to the size of the CNN down-sampled feature map, the window size g is set to 8, 4 and 2 respectively to ensure effective mixing of multi-scale features; the model is optimized using the SGD optimizer, with momentum set to 0.9 and weight decay set to 0.0001; the initial learning rate is set to 0.005, the batch size is 12, and the training is performed for 400 rounds.

[0145] To verify the effectiveness of the model proposed in the present application, based on the training parameter settings and the same configuration environment described above, the U-Net, AttentionUNet, nnUNet based on the CNN backbone network, the most advanced medical image segmentation methods TransUNet, Swin U-Net, MT-UNet, UNeXt, RMMLP and CSSNet based on the Transformer and MLP backbone network are selected respectively for comparative experiments on the Synapse and ACDC data sets, and the experimental results of different models on the two data sets are shown in Table 1 and Table 2, wherein the bold font represents the optimal result, and the underlined font represents the suboptimal result.

[0146] Table 1 Experimental results of different models on Synapse data set

[0147]

[0148]

[0149] As can be seen from Table 1, the network structure MSAFNet proposed in the present application has excellent segmentation performance on the Synapse data set, with the highest average Dice coefficient of 85.59 and the minimum average HD95 coefficient of 11.63. Specifically, the Dice values of MSAFNet on gallbladder, left kidney, pancreas and stomach are all optimal, especially on small organs such as gallbladder and pancreas, which perform particularly outstandingly, reaching 88.11 and 84.37 respectively. Compared with traditional CNN networks and high-complexity Transformer models, MSAFNet uses MX-RCAB modules to fuse information in multiple dimensions, effectively capturing the details of organs with complex anatomical structures and clear boundaries such as kidneys. At the same time, the SCGB module significantly improves the quality of feature representation when dealing with small organs such as gallbladder and pancreas with fuzzy boundaries by suppressing noise, enabling the model to accurately distinguish the contours and boundaries of these organs. In addition, the Dice coefficient difference between different organs is significantly reduced, which reflects the advantage of MSAFNet in processing multi-scale complex organ structures. The excellent performance of HD95 index further shows that the method proposed in the present application has higher precision and stability in boundary positioning.

[0150] Table 2 Experimental results of different models on ACDC data set

[0151]

[0152]

[0153] Table 2 shows the cardiac segmentation performance of different network models on the ACDC dataset. Compared with the latest MLP-based medical image segmentation methods (such as RMMLP, CSSNet), MSAFNet significantly improves the segmentation results on different size organs, verifying the effectiveness of the MLP-based MX-RCAB module. In addition, compared with the Transformer model suitable for large datasets (such as SwinUNet, MT-UNet, etc.), MSAFNet is more suitable for medical image segmentation tasks with small datasets, achieving the best segmentation effect under the same parameters. As a highly adaptable standard model, nnUNet has a rich image preprocessing and post-processing process, and shows the best segmentation performance on multiple medical image datasets. MSAFNet achieves an average Dice coefficient of 92.37%, slightly higher than the 92.32% of nnUNet, and performs significantly better in left ventricular and right ventricular segmentation, with 90.56% and 96.48%, respectively. Therefore, MSAFNet not only has comparable performance to nnUNet, but also reduces the consumption of computing resources, making it more adaptable in practical applications.

[0154] To verify the effect of the medical image segmentation model based on axial MLP and spatial cross-gating in the multi-shape and multi-scale organ segmentation of the application, the segmentation images of four different models are selected for visualization, as shown in Figure 7

[0155] As can be seen from Figure 7 , the segmentation results of UNet, TransUNet and UNext have the problems of segmentation loss and edge detail loss, and a small part of them also have the phenomenon of identifying the background area as the organ area. However, the segmentation results of the method proposed in the application are more complete on both large organs such as the stomach and small organs such as the pancreas, and the overall segmentation result is closest to the gold standard. This is due to the fact that MSAFNet combines feature maps of different scales, capturing more effective low-level detail information and global dependencies from the spatial and channel dimensions, thereby improving the accuracy of target segmentation. On the other hand, when segmenting highly similar organs such as the left and right kidneys, MSAFNet effectively filters and optimizes the features generated by the encoder, thereby reducing information redundancy and preventing important information loss. Overall, MSAFNet performs excellently in handling complex structures and multi-organ medical image segmentation tasks, effectively overcoming the difficulties of over-segmentation of traditional CNN and insufficient boundary segmentation of hybrid models.

[0156] ​To verify the effectiveness of each component in the proposed network structure, multiple comparative experiments are designed on the Synapse dataset. ResNet50-UNet is used as the baseline, and key components in the model are gradually increased or replaced, including MX-RCAB, T-MLP-Attn, and SCGB, etc.

[0157] Effectiveness of MX-RCAB

[0158] To verify the effectiveness of the proposed MX-RCAB module, it is compared with advanced Transformer and MLP methods, including ViP, ViT, MLP-Mixer, and R50-ViT encoder modules. The experimental results are shown in Table 3.

[0159] Table 3 Ablation experiment of MAF-MLP effectiveness

[0160]

[0161] As can be seen from Table 3, compared with the pure ViT encoder, MX-RCAB improves the average DSC and HD by 8.91% and 17.69%, respectively. Even compared with the hybrid model R50-ViT, its segmentation performance is still improved by 6.8% and 6.31%, respectively. These results show that the hybrid model combining MX-RCAB and CNN can surpass the CNN-Transformer hybrid model and is expected to become a new method for medical image segmentation. Among the MLP-based models, MX-RCAB has significant improvement compared with MLP-Mixer and ViP, with average DSC improved by 18.1% and 7.1%, respectively. This shows that the MX-RCAB based on space and channel can further process the feature maps from CNN and fine-tune the features in different dimensions, which is more effective than the ordinary token mixing method.

[0162] Effectiveness of T-MLP-Attn and RCAB

[0163] To verify the effectiveness of each component in MX-RCAB, an ablation experiment is conducted, in which T-MLP-Attn and RCAB are removed and replaced with a simple MLP as the baseline. The experimental results are shown in Table 4.

[0164] Table 4 Ablation experiment of MX-RCAB with different structures

[0165]

[0166]

[0167] When different components are added to the baseline respectively, the average DSC and HD are obviously improved. Compared with the simple MLP, the T-MLP-Attn containing the split-head MLP and the activation function can further enhance the feature learning ability of the network; at the same time, the RCAB realizes the dynamic adjustment of the feature channel by weighting the important channel and suppressing the unimportant channel feature. The experimental results show that the two modules of MX-RCAB, T-MLP-Attn and RCAB, are effective, and the complete MX-RCAB structure can achieve the best experimental results.

[0168] Effectiveness of SCGB

[0169] In order to verify the effectiveness of the SCGB for codec feature fusion, SA, gMLP and SCGB are added to the model proposed in the application respectively. The experimental results are shown in Table 5.

[0170] Table 5 Ablation experiment of SCGB effectiveness

[0171]

[0172] The experimental results show that after using SA instead of the SCGB proposed in the application, the average DSC and HD decrease by 0.94 and 8.05 respectively. This shows that the SCGB can effectively filter the redundant information transmitted by the encoder and guide the network to focus on more important underlying detailed information. Compared with gMLP, SCGB combines the advantages of MLP and SA, which makes the average DSC and HD increase by 0.71 and 1.81 respectively, verifying the contribution of SCGB to the performance improvement of the model.

[0173] In view of the problems of insufficient segmentation accuracy, low generalization ability of hybrid model and missing boundary details of traditional models in medical image segmentation tasks, the application proposes a medical image segmentation model based on axial MLP and spatial cross-gating, namely MSAFNet. In the decoder part, MSAFNet obtains mixed features of space and channel with linear complexity through MX-RCAB, wherein MAF-MLP applies T-MLP-Attn in local and global branches to obtain high-quality multi-scale feature maps. In addition, in the process of codec feature fusion, SCGB effectively captures underlying detailed information at different levels, thereby realizing fine feature fusion. The comparative experimental results show that the average DSC of MSAFNet on Synapse and ACDC datasets reaches 85.59% and 92.37% respectively, which shows better segmentation effect than similar models.

[0174] The medical image segmentation system based on axial MLP and spatial cross-gating proposed in the embodiment of the application executes the medical image segmentation method based on axial MLP and spatial cross-gating.

[0175] The medical image segmentation system based on axial MLP and spatial cross-gating described above can be implemented in the form of a computer program that can run on a computer device.

[0176] The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory can include a non-volatile storage medium and an internal memory.

[0177] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions that, when executed, can cause the processor to perform a medical image segmentation method based on axial MLP and spatial cross-gating.

[0178] The processor is configured to provide computing and control capabilities to support the operation of the entire computer device.

[0179] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, can cause the processor to perform a medical image segmentation method based on axial MLP and spatial cross-gating.

[0180] The network interface is configured to communicate with other devices over a network. Those skilled in the art can understand that the computer device structure described above is only part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0181] The processor is configured to run a computer program stored in the memory, which program implements the medical image segmentation method based on axial MLP and spatial cross-gating described in Embodiment One.

[0182] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0183] Those skilled in the art can understand that all or part of the processes in the method of the above embodiments can be completed by instructing the relevant hardware by a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to realize the process steps of the above method embodiments.

[0184] The application further provides a storage medium. The storage medium can be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to make the processor execute the medical image segmentation method based on axial MLP and spatial cross-gating according to Embodiment 1.

[0185] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0186] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0187] In several embodiments provided by the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0188] The steps in the method of the embodiments of the application can be adjusted, combined and deleted in sequence according to actual needs. The units in the device of the embodiments of the application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0189] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a storage medium. Based on such an understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0190] It should be noted that the above only describes the preferred embodiments of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A medical image segmentation method based on axial MLP and spatial cross-gating, characterized in that, The method comprises the steps of: S1, constructing a medical image segmentation model MSAFNet based on an axial MLP and a spatial cross-gating; the medical image segmentation model comprises an encoder and a decoder, the encoder comprises a CNN block and a multi-axis mixed residual channel attention block MX-RCAB, and the decoder comprises a spatial cross-gating block SCGB and a decoder block Decoder Block; The multi-axis mixed residual channel attention block MX-RCAB comprises a spatial branch and a channel branch; in the spatial branch, a multi-axis feature mixing MLP is used to fuse local and global features; in the channel branch, a residual channel attention module RCAB is used to enhance channel features; then, the feature maps processed by the spatial branch and the channel branch are spliced, and all channels are fused through input residual connection to obtain the output features of the encoder; The input of the spatial cross-gating block SCGB is composed of the features X and Y output by the encoder and the decoder, which are aligned in dimension through a convolution layer and generate corresponding feature maps; then, preliminary feature activation is performed: X act = σ(FC(LN(Conv(X)))) Y act = σ(FC(LN(Conv(Y)))) wherein Conv is a 3*3 convolution layer, LN is layer normalization, FC is a full connection layer, and sigma is a GELU activation function; The core component of the SCGB is a cross-gating module CGB, which applies a linear layer to the segmented serial window to generate a query Q, a key K and a value V respectively; Q X ,K X ,V X = Linear(X act ) Q Y ,K Y ,V Y = Linear(Y act ) Then, the dot product between the query Q and the key K is calculated, and after the Softmax activation function, the generated attention weight is weighted and summed with the value V to obtain a new feature representation: where d k is a scaling factor to balance the magnitude of the dot product result; In addition, a residual connection is introduced for data transmission; in order to ensure that the output channel has the same channel size as the input X and Y, a full connection layer is used to complete linear transformation, which is specifically represented as: Output x = W4(X act ⊙ Attention Y + X Output y = W5(Y act ⊙Attention x ) + Y wherein, ⊙ denotes element-wise multiplication, W4, W5denote corresponding MLP linear layer mapping features, X act and Y act denote the features after preliminary activation; S2, obtaining a medical image dataset, training the medical image segmentation model based on the axial MLP and the spatial cross-gating to obtain a trained medical image segmentation model based on the axial MLP and the spatial cross-gating; S3, obtaining a medical image to be segmented; S4, inputting the medical image to be segmented into the trained medical image segmentation model based on the axial MLP and the spatial cross-gating to perform image segmentation on the medical image to be segmented and obtain a target segmentation image.

2. The method of claim 1, wherein, In the medical image segmentation model MSAFNet based on the axial MLP and spatial cross-gating, the CNN block is used to extract multi-scale hierarchical feature maps x from the input medical image layer by layer l , l e {1, 2, …, L}, L represents the depth of the encoder, and the extracted hierarchical feature maps are input into the multi-axis mixed residual channel attention block MX-RCAB for global information interaction in space and channels; the CNN block is composed of multiple convolution layers, pooling layers and activation functions.

3. The method of claim 1, wherein, In the spatial branch, the multi-axis feature mixing MLP is used to fuse local and global features, which specifically comprises: For input feature x l ∈R C×H×W , l ∈ {1, 2,..., L}, where C, H, W represent the channel number, height and width of the input feature map respectively, L represents the depth of the encoder, which is divided into N non-overlapping windows in the patch manner of Vision Transformer, and the shape size of each window is C x g x g, where g is the window size; then, the feature map is divided into local branch and global branch along the channel dimension, and each branch uses Transpose MLP Attention to enhance feature interaction. In Transpose MLP Attention, the input feature X e R C×H×W is split along the channel dimension into X A and X B , where X A ,X B e R C / 2×H×W ; then X A is processed to generate attention weights, in particular: X A = Softmax(W2 ReLU(W1 X + b1) + b2) A +b1)+b2) where W1, W2 are weight matrices of the MLP, b1, b2 are bias terms, ReLU is an activation function, and the Softmax function is used for normalization to generate attention weights X A ; the calculated attention weights X A are then applied to the linearly transformed X B , to perform a weighting operation: where denotes element-wise multiplication, is the weighted feature map, W3 is the weight matrix of the linear transformation layer, and b3 denotes the bias value of the linear layer.

4. The method of claim 1, wherein, In the channel branch, the residual channel attention module RCAB is used to enhance the channel features, which specifically comprises: For input feature x l ∈R C×H×W , l ∈ {1, 2, …, L}, C, H, W represent the channel number, height and width of the input feature map respectively, L represents the depth of the encoder, the channel attention weight is generated through the convolution layer, MLP layer and Squeeze-Excitation module, and then the attention weight is multiplied with the original feature map in the channel dimension; in addition, the addition of the residual connection allows the input feature map to be added to the weighted feature map; the process is represented as: SE(X) = σ(W fc ((Pool(X)))) ⊙ X where LN denotes layer normalization, MLP denotes a multi-layer perceptron layer, SE denotes a Squeeze-Excitation module, Pool denotes a global average pooling operation, W fc denotes a weight matrix of a fully connected layer, σ is a sigmoid activation function, ⊙ denotes element-wise multiplication, X denotes an input feature of the Squeeze-Excitation module, is an output feature map enhanced by the residual channel attention mechanism.

5. The method of claim 1, wherein, The decoder block Decoder Block performs an upsampling operation on the low-resolution feature map output by the decoder, and simultaneously performs pixel-by-pixel fusion of the corresponding resolution feature map from the encoder and the upsampled feature map, so as to gradually restore the spatial details of the target image and enhance the feature expression; The upsampling operation is realized by a bilinear interpolation method, which expands the low-resolution feature map to a higher resolution.

6. The method of claim 1, wherein, When training the medical image segmentation model based on the axial MLP and the spatial cross-gating, The loss function is composed of cross-entropy loss CELoss and Dice similarity coefficient loss DiceLoss, and the total loss function is defined as: L = 0.5L CE + 0.5L Dice The cross-entropy loss CELoss is used to measure the difference between the predicted value and the true label, and is suitable for semantic segmentation tasks of pixel-level classification, and the calculation formula is as follows: where N represents the number of samples in a batch, C represents the total number of classifications, y ij represents the true label of the ith sample in the jth class, p ij represents the predicted probability of the ith label in the jth class; The Dice similarity coefficient loss DiceLoss is based on the Dice similarity coefficient and is used to evaluate the overlap between the segmentation result and the true label, and the calculation formula is as follows: where p i is the model-predicted segmentation probability, g i is the true segmentation ground truth.

7. A medical image segmentation system based on axial MLP and spatial cross-gating, characterized by, The medical image segmentation system executes the medical image segmentation method based on the axial MLP and spatial cross-gating as claimed in claim 1.

8. A computer device, comprising: The device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the method as claimed in any one of claims 1 to 6.

Images