Medical image segmentation model training method based on multi-scale feature interaction

Abstract

The present application relates to the technical field of image processing, in particular to a medical image segmentation model training method based on multi-scale feature interaction. In order to break through the limitation of traditional attention mechanism in global calculation, the DSWIA module is based on window division strategy, and the feature interaction in the local window and the cross-scale information transmission are realized. It not only effectively reduces the calculation complexity, but also enhances the model's ability to capture different scale features. The decoder enhances feature expression through multi-scale convolution, and combines spatial attention mechanism and local similarity perception sampler to effectively mine the complex spatial structure and local details in the image, realize fine feature reconstruction and accurate segmentation prediction. The cross-attention fusion module dynamically fuses features of different scales and types through hybrid attention mechanism, effectively improves the model's understanding ability of complex structure and segmentation robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a medical image segmentation model training method based on multi-scale feature interaction. BACKGROUND

[0002] As a core technology in computer-aided diagnosis, medical image segmentation plays an important role in tasks such as lesion identification and organ positioning. In recent years, various convolutional neural network (CNN) architectures based on U-shaped structures have been widely used in medical image segmentation tasks. Representative models include UNet, UNet++, 3D-UNet, and nnUNet, which have become standard techniques for achieving high-quality segmentation. These models use an encoder-decoder architecture to effectively preserve spatial detail information by fusing shallow and deep features through skip connections. Variants such as Attention UNet and ResUNet further introduce attention mechanisms and residual structures to improve the model's attention to key regions, achieving better performance in multiple medical segmentation tasks.

[0003] Although CNN-based models have achieved remarkable results, they have limitations in modeling global and long-range dependencies, limited by inherent local receptive fields and induction bias. Inspired by the Transformer architecture in natural language processing, researchers have introduced attention mechanisms into image processing to enhance feature modeling capabilities. For example, Vision Transformer (ViT) uses image patch sequences and a Transformer encoder to model global context. However, the computational complexity of the Transformer grows quadratically with the input resolution, resulting in significant computational overhead when processing high-resolution medical images. Swin Transformer reduces complexity and improves efficiency by using local window attention, but there is room for improvement in multi-scale feature fusion and boundary detail preservation, especially when dealing with complex medical image targets with fuzzy boundaries.

[0004] Medical images typically have high resolution and contain a large number of fine-grained structures. How to effectively capture the relevance of long-range dependencies and multi-scale features while controlling the consumption of computing resources is a key research issue. In addition, medical image segmentation requires higher boundary accuracy than general semantic segmentation tasks, posing greater challenges to the model's ability to model details. SUMMARY

[0005] Therefore, the present application provides a medical image segmentation model training method based on multi-scale feature interaction, which at least solves one problem in the prior art.

[0006] In a first aspect, the present application provides a medical image segmentation model training method based on multi-scale feature interaction, which includes the following steps:

[0007] A medical image segmentation model is constructed, comprising an encoder, a feature interaction module, and a decoder; wherein the feature interaction module includes a feature alignment module and a DSWIA module;

[0008] A medical image segmentation model was trained using a medical image dataset. An encoder was used to encode the medical images, obtaining at least four levels of features. , , , In the feature alignment module, make and , and Feature alignment is performed to obtain the transformed features. and , and In the DSWIA module, sequentially with After selecting features through a window loop, the four groups of features are interacted with to obtain the features. and ,Will and Perform feature interaction to obtain features and Use decoder to , , , Decode;

[0009] The parameters of the medical image segmentation model are updated based on the target loss.

[0010] In some optional embodiments, the window cyclic selection includes:

[0011] For features and , will feature Divided evenly into channels Groups of non-overlapping order Each set of features contains M One channel, among which M yes and The greatest common divisor;

[0012] from Take out continuously from the group The group forms a new feature subset, with a window size of [size missing]. M*k Each time the window moves M Select features within the window to form a new set of features, until the feature subsets are repeatedly combined. Each set of features contains...M*k a channel;

[0013] wherein, is , is ; represents the number of channels of, represents the number of channels of; represents the height of, represents the height of; represents the width of, represents the width of; represents a positive integer.

[0014] In some optional embodiments, the DSWIA module comprises a feature window division and feature stitching module, a sparse feature interaction attention module, a feature reshaping and restoration module, in the feature window division and feature stitching module, the features are divided into windows, and the divided window features are respectively flattened and stitched; in the sparse feature interaction attention module, cross-layer feature fusion is realized through window attention and feature enhancement operations; in the feature reshaping and restoration module, the output features of the feature window division and feature stitching module are reshaped and merged.

[0015] In some optional embodiments, the decoder comprises a graph convolution attention module, an upsampling block, a local similarity perception sampler, a cross-attention fusion module and a lightweight segmentation head, processed by the graph convolution attention module and the upsampling block, is input to the local similarity perception sampler together with for processing, and then aggregated with to output , a segmentation map is generated by the graph convolution attention module, the lightweight segmentation head and the upsampling block; processed by the graph convolution attention module and the upsampling block, is input to the local similarity perception sampler together with for processing, and then aggregated with to output , a segmentation map is generated by the graph convolution attention module, the lightweight segmentation head and the upsampling block; processed by the graph convolution attention module and the upsampling block, is input to the local similarity perception sampler together with for processing, and then aggregated with to output ; The graph convolution attention module is input into a cross attention fusion module, and the cross attention fusion module outputs and respectively. , and respectively output segmentation maps and through a light segmentation head and an upsampling block; finally , , and are aggregated by addition, and the final prediction segmentation map is output after being activated by a Sigmoid activation function.

[0016] In some optional embodiments, the graph convolution attention module is composed of a graph convolution block (GCB) and a spatial attention block (SPA), and the operation on the input feature is as follows:

[0017]

[0018] wherein, represents the input feature; represents the graph convolution attention module; represents the graph convolution block; represents the spatial attention block.

[0019] In some optional embodiments, the local similarity perception sampler sequentially performs local cosine similarity calculation, similarity-guided offset prediction, and deformable feature sampling on the input feature.

[0020] In some optional embodiments, the cross attention fusion module captures the local correlation between features through spatial attention, dynamically balances the original feature and the attention feature through the gating mechanism, and performs global interaction through cross attention.

[0021] In a second aspect, the present application provides a medical image segmentation method based on multi-scale feature interaction, which comprises the following steps:

[0022] inputting the image to be segmented into a medical image segmentation model, and outputting a segmentation result from the medical image segmentation model;

[0023] wherein the medical image segmentation model is obtained by the medical image segmentation model training method as described above.

[0024] In a third aspect, the present application provides an electronic device, which comprises:

[0025] at least one processor;

[0026] and a memory in communication connection with the at least one processor;

[0027] The memory stores instructions, and the instructions, when executed by the at least one processor, implement the medical image segmentation model training method based on multi-scale feature interaction as described above.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium storing instructions, and the instructions, when executed by a processor, implement the medical image segmentation model training method based on multi-scale feature interaction as described above.

[0029] Due to the adoption of the above technical solutions, the embodiments of the present application have at least the following beneficial effects:

[0030] (1) A multi-scale interaction mechanism based on local window division is designed, which fuses local attention and cross-scale information transmission, reduces the computational cost, enhances the modeling ability of multi-scale features, and solves the problems of complex computation and weak fusion ability of traditional attention mechanisms;

[0031] (2) The spatial attention mechanism and the local similarity perception sampler are introduced to realize the enhancement of feature maps and the modeling of complex spatial relationships, and improve the segmentation ability of the model for medical image detail information. The local similarity perception sampler adaptively adjusts the sampling strategy by analyzing the similarity features of the local area of the image, accurately captures the subtle structure and texture information in the image, and provides richer feature expression for the segmentation task;

[0032] (3) A cross-attention mechanism that fuses features of different scales and semantic levels is designed to realize adaptive complementation and fusion between features, effectively improve the adaptability and segmentation performance of the model for complex medical image scenes. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 FIG. 1 is a schematic diagram of the network structure of the medical image segmentation model in an embodiment of the present application.

[0034] Figure 2 FIG. 2 is a schematic diagram of the feature alignment module in an embodiment of the present application.

[0035] Figure 3 FIG. 3 is a schematic diagram of the double-scale window interaction attention module (DSWIA) in an embodiment of the present application.

[0036] Figure 4 FIG. 4 is a schematic diagram of the sparse feature interaction attention module in an embodiment of the present application.

[0037] Figure 5 FIG. 5 is a schematic diagram of the processing flow of the first layer feature and the third layer aligned feature through four iterations of the double-scale window interaction attention (DSWIA) module in an embodiment of the present application.

[0038] Figure 6 This is a schematic diagram of the structure of the graph convolutional attention (GCAB) module in one embodiment of the present invention.

[0039] Figure 7 This is a schematic diagram of the structure of the upconvolution block in one embodiment of the present invention.

[0040] Figure 8 This is a schematic diagram of the structure of a Local Similarity Aware Sampler (LSAS) in one embodiment of the present invention.

[0041] Figure 9 This is a schematic diagram of the cross-attention fusion (CAFB) module in one embodiment of the present invention.

[0042] Figure 10 This is a schematic diagram of the structure of a linear gating module in one embodiment of the present invention.

[0043] Figure 11 Visualizations of segmentation results for different models on the Synapse dataset.

[0044] Figure 12 Visualizations of segmentation results for different models on the ACDC dataset.

[0045] Figure 13 Visualizations of segmentation results for different models on the ISIC2018 dataset. Detailed Implementation

[0046] The following will provide a clear and complete description of the concept and technical effects of the present invention, so as to fully explain the purpose, solution and effects of the present invention.

[0047] To enhance the model's ability to model multi-scale features and depict boundary details, this invention provides a training method for a medical image segmentation model based on multi-scale feature interactions. Specifically, a medical image segmentation model is trained, and medical image segmentation is achieved through this model. For ease of representation, the medical image segmentation model in this invention is labeled MFIUNet.

[0048] like Figure 1As shown, the medical image segmentation model (MFIUNet) includes an encoder, a feature interaction module, and a decoder, wherein the feature interaction module includes a feature alignment (FA) module and a dual-scale window interaction attention (DSWIA) module. The encoder can use a pre-trained PVTv2 for four-stage encoding. In the encoder, the input image is encoded through four stages to obtain four layers of hierarchical features with different semantic information levels, respectively marked as 、 、 、 , and the number of feature channels are 64, 128, 320, and 512, respectively. Then, the channel dimensions of each hierarchical feature are subjected to window division, boundary pixel padding, and flattening operations to form features marked as 、 、 、 . Subsequently, the features and are sent to the DSWIA module for local-global information interaction, and the features and are output, respectively. Similarly, the features and are sent to the DSWIA module, and the features and are output, respectively. Then, the decoder fuses the output features of the encoder 、 、 and the output features of the DSWIA module 、 、 、 and gradually restores the feature resolution. Moreover, the bidirectional cross-attention is used to further optimize the shallow semantic features and improve the precision of the segmentation boundary. Finally, the up-sampling operation with different size convolutions is used to restore the picture resolution to a uniform size, and the four predicted maps 、 、 、 are added and output as the final prediction map after the Softmax or Sigmoid activation function.

[0049] Let the original input feature be marked as , then the features of each layer after down-sampling by PVTv2 are as shown in equation (1):

[0050]

[0051] wherein, Representation of feature map The number of channels; This represents the number of channels in the feature map. Indicates the height of the feature map, Indicates the width of the feature map.

[0052] To reduce the number of fusion operations and thus lower computational costs, features can be optimized. and Perform dimensional transformations to make... and , and The channel alignment is as shown in equation (2):

[0053]

[0054] Then, the transformed features Perform operations such as pixel filling to achieve feature alignment output. For the transformed features and If the number of feature channels Less than Then based on the number of channels Adjustment based on benchmark Number of channels Make it with Number of channels Alignment. The specific operation for channel alignment can be: ... Divided evenly into channels Groups of non-overlapping order Each set of features contains M One channel ( M yes and (greatest common divisor); then from Take out continuously from the group The features are grouped into new subsets until repeated combinations of the features are encountered. Therefore, the generated feature subsets can be represented as:

[0055]

[0056] in, This indicates channel splicing. Figure 2 Showing the transformed features A flowchart illustrating the channel alignment process, where... Indicates a separation operation. This indicates a splicing operation.

[0057] The DSWIA module is used for fusing partial features of different stages, realizing sufficient interaction and complementation of cross-scale information, thereby effectively reducing the loss of multi-scale feature information and improving the multi-scale feature capturing capability of the model; and by window division, two-dimensional spatial position information and one-dimensional sequence position information are organically combined; by local attention in the window, fine-grained features are captured, and then information interaction between windows is realized by high-level features, effectively solving the problem that single-scale attention cannot balance local details and global context; this double-scale window attention mechanism is suitable for high-resolution image understanding tasks and can establish effective feature correlation between different semantic levels.

[0058] As shown in Figure 3 , the DSWIA module includes a feature window division and feature splicing module, a sparse feature interaction attention module, and a feature remodeling and restoration module.

[0059] In the feature window division and feature splicing module, for the channel-aligned features and , the features are divided into the same number of windows , and the size of the divided feature map is as shown in equations (4) and (5):

[0060]

[0061] wherein, denotes a separation operation; denotes the target number of windows, and denote the height and width of the divided window of the feature map , respectively, and denote the height and width of the divided window of the feature map , respectively, denotes a series of features separated from the feature according to the window size, denotes a series of features separated from the feature according to the window size. The divided window features are respectively subjected to a flattening operation and a splicing operation, as shown in equation (6):

[0062]

[0063] wherein, denotes a flattening operation, denotes channel splicing, denotes the spliced features of and of the corresponding window.

[0064] The sparse feature interaction attention module includes a sliding window attention module and a fully connected layer (as shown in Figure 4 The features are input into the sparse feature interaction attention module, and cross-layer feature fusion is achieved through window attention and feature enhancement operations, as shown in equations (7) to (9):

[0065]

[0066] wherein, represents the sliding window attention, represents the fully connected layer; , , , , respectively represent weight matrices; , , respectively represent bias vectors; represents the Gumbel Softmax operation; represents the TopK sparse operation, which dynamically selects the top k most relevant feature positions through the Gumbel Softmax operation, to achieve sparse feature interaction; represents the feature reshaping operation, represents the GeLU activation function; represents the feature after the sparse feature interaction attention.

[0067] In the feature reshaping and restoration module, the output features of the feature window division and feature splicing module are reshaped and merged, etc., to obtain the output feature information of the corresponding stage, as shown in the following equation:

[0068]

[0069] wherein, respectively correspond to the features of each window after the sparse feature interaction attention, represents the separation operation, represents the merging operation, represents the merged features, represents the merged features.

[0070] As shown in Figure 5 , taking the first layer and the third layer stage feature information modeling as an example, the third layer features are divided into four groups of features through Figure 2 window cycle selection, which are respectively sequentially combined with and after interaction Input four iterations of DSWIA modules to interact features and realize multi-stage feature enhancement. Then, get the feature shape size of each stage, and restore the output feature map of each stage after four interactions into a feature block. The specific operation is: according to the feature block position determined by the feature selection structure, splice the feature block; if the positions of two feature blocks are the same, the deeper feature block covers and rewrites the shallow feature block, and finally splices to form ; this process finally outputs and .

[0071] The second layer feature and the fourth layer feature are input into the DSWIA module, and are selectively iterated 4 times, each time two corresponding layer features are output as new inputs, and the feature information is improved after each iteration fusion, and finally outputs and , and restores , to the original shape, as shown in equation (11) and equation (12):

[0072]

[0073] wherein, represents the cyclic feature splicing operation.

[0074] After the above operation, the feature interaction module finally outputs , , and .

[0075] By adjusting the number of iterations of the DSWIA module autonomously, the network can dynamically adjust the contribution of different modules, which significantly improves the modeling ability of the medical image segmentation model.

[0076] The decoder includes a graph convolution attention module (GCAB), an up-sampling block (UpConvolution), a local similarity-aware sampler (LSAS), a cross-attention fusion module (CAFB), and a light segmentation head (LSH). GCAB is mainly used to robustly enhance local and global features. UpConvolution is used for up-sampling to match the dimensions and resolutions of the feature maps from the next hop connection. The LSAS module is used to align features from different scales. LSH is used for segmentation output.

[0077] As shown in Figure 6 , GCAB is composed of a graph convolution block (GCB) and a spatial attention block (SPA), and the operation of GCAB on the input feature is shown in equation (13):

[0078]

[0079] wherein, denotes the input feature, denotes a graph convolution attention module, which is more efficient than a convolution attention module (CAM) due to the use of graph convolution; denotes a graph convolution block; denotes a spatial attention block.

[0080] GCB is used to enhance features from different scales, which is composed of a graph convolution layer (GC) ) and two convolution layers (Conv) ). Each convolution layer is followed by a batch normalization layer (BN) ) and a ReLU activation layer (ReLU) ). The operation of GCB on the input feature is shown in equations (14) and (15):

[0081]

[0082] wherein, denotes a dynamic graph convolution (DGC), and denote batch normalization and ReLU activation function, respectively, denotes a graph convolution, denotes a GELU activation function.

[0083] SPA generates an attention weight map by analyzing the spatial position importance of the input feature to dynamically identify key regions in the feature map and selectively enhance the features of these regions. The operation of SPA on the input feature is shown in equation (16):

[0084]

[0085] wherein, denotes a Sigmoid activation function; and denote the maximum pooling result and the average pooling result obtained along the channel dimension, respectively, denotes a convolution layer with an inner margin of 3 and a size of ; denotes a Hadamard product.

[0086] As shown in Figure 7 , the up-sampling block (UpConvolution) gradually up-samples the features of the current layer to match the dimension of the next skip connection. Each up-sampling layer is composed of an up-sampling layer with a scale factor of 2 , having equal input channel groups depthwise separable convolution , batch normalization , ReLU function activation , and convolution comprises. The operation of UpConvolution on input features is shown in equation (17):

[0087]

[0088] The LSAS module innovatively utilizes the local similarity features within the sliding window to guide the deformable convolution sampling process. It dynamically generates spatial offsets by analyzing the geometric consistency relationship between multi-scale features, thereby achieving feature alignment of input features. Unlike traditional fixed sampling patterns, this module can adapt to the structural features of image content, accurately align features while preserving high-frequency details, and is suitable for multi-scale feature fusion tasks.

[0089] As shown in Figure 8 , the LSAS module sequentially performs local cosine similarity calculation, similarity-guided offset prediction, and deformable feature sampling on input features.

[0090] After group normalization, for each spatial position , a local window with a window size of and an inflation rate of is defined to extract local feature blocks, as shown in equation (18):

[0091]

[0092] wherein denotes the group normalization operation, denotes the flattening operation of local window features, and denotes the restoration of local window features to the original size. For each local window feature , the cosine similarity of the feature vector composed of the center position and its neighborhood position can be calculated by equation (19):

[0093]

[0094] wherein denotes the Euclidean distance of the L2 norm of the vector. Each pixel position can calculate ​cosine similarity information, so the similarity feature can be obtained wherein .

[0095] The is concatenated with the input feature and input into the offset generator. The concatenated feature is divided into groups, and the offset and scaling factor are calculated according to the grouping to achieve independent scaling of each group of offsets. The offset is calculated by equations (20) to (22):

[0096]

[0097] wherein, and correspond to the pixel offsets of the features and respectively, represents the initial position bias, and represent the similarity features of and respectively, represents the spatial down-sampling operation, represents the Sigmoid activation function. The offset is generated, represents the number of offset groups of the feature ( represents the x-direction and y-direction offsets of the group). By strategically dividing the feature into different groups, a unique offset is assigned to achieve finer resampling.

[0098] The feature is resampled according to the learned offset to achieve deformable sampling for super-resolution. First, define the coordinate network to achieve center alignment, as shown in equation (23):

[0099]

[0100] wherein, represents the operation on the pixel values in the entire feature matrix, represents the pixel value corresponding to the or the position. Then, the grid coordinates are normalized to adapt to grid sampling according to the predicted offset . Next, the coordinates are adjusted to the grouping sampling mode using and dimension transformation, and finally each group of features is independently sampled using . This process can be represented by equations (24) and (25):

[0101]

[0102] wherein, denotes the input feature , denotes the predicted offset between the input feature and the output feature , denotes the linear normalization, denotes the spatial upsampling, denotes the feature reshaping, denotes the grid sampling, denotes the final grid sampling result.

[0103] The CAFB module realizes the dynamic fusion of double-branch features through the cross-attention mechanism. The module innovatively combines local attention mechanism and global transformer, realizes feature depth complementation through spatial-channel double adaptive mechanism, and maintains the independence of each branch at the same time. It is suitable for multi-scale feature fusion and symmetric network architecture and other fine feature interaction tasks, and a better balance between computational efficiency and feature expression ability is achieved. The linear gating module in the CAFB module can adjust the distribution of features after a series of transformations, making the feature distribution more stable. In medical image segmentation, the image features of different cases are quite different. The linear gating module reduces the training fluctuations caused by such differences, so that the model can stably use feature information when processing medical images of different patients, improving the robustness of segmentation.

[0104] As shown in Figure 9 , in the CAFB module, the input features and are respectively mapped into query vectors and key-value vectors through 3x3 convolution layers, and the spatial attention map Attention is generated by using element-wise product to capture the local correlation between features, as shown in the following formula:

[0105]

[0106] wherein, denotes the convolution of , denotes the Sigmoid activation function, denotes the Hadamard product, and denote the attention weighted features of features and , respectively. Then, the dynamic weights and are learned by using channel concatenation and 3x3 convolution, respectively.fuses the original feature and the attention enhanced feature in an adaptive manner. This process dynamically balances the original feature and the attention feature by utilizing a gating mechanism, realizing a spatial granularity class compression excitation (SE) mechanism. The operation flow can be represented by the following equations:

[0107]

[0108] wherein, denotes channel concatenation, and denotes a dynamic weight tensor, and correspond to the original feature weight, and correspond to the attention feature weight. Finally, a symmetric cross-attention is introduced to further promote the global interaction between the two branches, inputting the output features of the previous stage and into linear layers to obtain projection matrices , , , , , ; swapping the matrices and and inputting them into a multi-head attention mechanism, which operates as follows:

[0109]

[0110] wherein, and denote the dimensions of the two attention heads, denotes a Softmax activation function, denotes a multi-head attention mechanism, denotes the attention weight obtained through the attention mechanism, denotes the attention weight obtained through the attention mechanism. As shown in Figure 10 , the results are input into a linear gating module, and the entire process of the linear gating module is expressed as:

[0111]

[0112] wherein, denotes a linear layer, denotes a ReLU activation function, and denotes a normalization layer.

[0113] LSH, as a part of the decoder, receives the refined features from the middle two stages and the two output feature maps of CAFB as input, and predicts four output segmentation maps. Each LSH is composed of a convolution and an up-sampling, which takes a feature map with channels as input, outputs a segmentation map with the number of channels equal to the number of target classes, and is up-sampled to the size of the original feature map. Four output segmentation maps are obtained from the four prediction heads of the first three stages of the decoder. Finally, the final segmentation output is calculated using the additive aggregation:

[0114]

[0115] where, are the weights of each prediction head, denotes the activation function, and the Sigmoid activation function is usually applied to binary segmentation, and the Softmax activation is applied to multi-class segmentation to obtain the final prediction output.

[0116] The DICE loss and the cross-entropy loss are used as the segmentation loss, and the total loss is the sum of the DICE and the cross-entropy loss, as shown in the following formula:

[0117]

[0118] where, represent the weights of the DICE loss and the cross-entropy loss , respectively.

[0119] In order to verify the image segmentation effect of the medical image segmentation model (MFIUNet) in the embodiment of the application, experiments are performed on three public image datasets, and compared with the existing mainstream models for medical image segmentation, and five-fold cross-validation is performed to further verify the stability or generalization of the model. The three public image datasets are:

[0120] ​​​​​​​​Synapse Abdominal Multi-Organ Dataset: This dataset contains 30 abdominal CT scans, including 3779 axially contrast-enhanced slices; each CT scan contains 85-198 512×512 pixel slices; 18 scans from the dataset were randomly selected for training and 12 scans for validation; the experiment segmented 8 abdominal organs, namely the aorta (AORTA), gallbladder (GB), left kidney (KL), right kidney (KR), liver (Liver), pancreas (PC), spleen (SP), and stomach (SM);

[0121] The ACDC Heart Organ Dataset contains 100 cardiac MRI scans, each consisting of three sub-organs: the right ventricle (RV), myocardium (Myo), and left ventricle (LV). 70 samples were used for training, 10 for validation, and 20 for testing.

[0122] ISIC2018 Skin Cancer Dataset: This dataset contains 2,594 images of skin lesions of varying sizes and their corresponding annotations, divided into training, validation, and test sets in a 7:1:2 ratio.

[0123] All experiments were conducted using PyTorch 2.4.0 on a single NVIDIA RTX 4080 GPU with 16GB of VRAM. To address hardware memory limitations, all images were resized to 244×244 pixels, and data augmentation was performed using random rotation and flipping. In the experiments, the predicted head weights were used. , , and Set all values ​​to 1.0 and optimize the weights during training; adjust the weights of the DICE loss function in the total loss function. Set the weight of the cross-entropy loss to 0.7. Set to 0.3. Use PVTv2-b2 as the encoder, and apply pre-trained weights from ImageNet to the PVTv2-b2 backbone network. Train using the AdamW optimizer, with both the learning rate and weight decay set to 0.3. See Table 1.

[0124] Table 1 Training Parameter Information

[0125]

[0126] The performance of different models was evaluated using the Dice score, Hausdorff distance (HD95), and mean intersection-union ratio (mIoU), where the Dice score, mIoU, and HD95 were calculated by equations (37), (38), and (39):

[0127]

[0128] wherein, and denote the ground truth and predicted segmentation map of the image, respectively; denotes the Dice score; denotes the mIoU; denotes the HD95, denotes the Hausdorff distance.

[0129] Table 2 shows the segmentation results of different models on the Synapse dataset. It can be seen that MFIUNet shows superior performance, and the segmentation performance on most organs (such as gallbladder, left kidney, right kidney, liver and stomach) reaches the optimal, and the segmentation effect on a small part of organs (such as aorta, pancreas and spleen) reaches the sub-optimal. It is worth noting that compared with the sub-optimal model (PVT-EMCAD-B2), the segmentation DICE score of MFIUNet on SM is improved by about 3.7, and the segmentation performance on KL, KR and Liver is also improved by more than 0.5. In addition, compared with PVT-EMCAD-B2, MFIUNet improves the overall DICE score by about 1.3, while the HD95 is reduced by 1.7, further indicating that MFIUNet has higher accuracy and robustness in organ boundary positioning.

[0130] Table 2 Segmentation results of different models on Synapse dataset (unit: %)

[0131]

[0132] In the table, ↑ indicates the higher the better, and ↓ indicates the lower the better, indicates that the source data is missing.

[0133] As Figure 11 shown, the visual segmentation results of MFIUNet show that it can accurately extract the boundary of the target region and effectively maintain the integrity of the organ edge. In contrast, EMCAD and CASCADE have fuzzy phenomena at the organ boundary; MissFormer and SwinUnet perform poorly in distinguishing organs from the background, with more false positive regions. Overall, in the abdominal organ segmentation task, MFIUNet outperforms other models, indicating that MFIUNet can effectively capture multi-scale features and alleviate the impact of semantic differences, thereby improving segmentation accuracy.

[0134] Table 3 shows the segmentation results of different models on the ACDC dataset. It can be seen that the DICE scores of MFIUNet on the right ventricle (RV), myocardium (Myo), and left ventricle (LV) are 91.38%, 89.92%, and 96.00%, respectively. Although slightly lower than Lite-MixedNet in RV segmentation, MFIUNet performs better in Myo and LV segmentation, with an average DICE score improvement of 0.36 over Lite-MixedNet. These results indicate that MFIUNet exhibits strong overall segmentation performance in cardiac organ segmentation tasks.

[0135] Table 3 Segmentation results of different models on the ACDC dataset (%)

[0136]

[0137] Figure 12 The visualization segmentation results of different models on the ACDC dataset are shown. From the first two rows and the last row of samples, it can be seen that models such as EMCAD and MissFormer have obvious shortcomings in right ventricular segmentation, while MFIUNet can accurately identify and segment the three main components of the heart organ. The third and fourth rows of samples further demonstrate that MFIUNet has higher accuracy in locating the boundaries of the left ventricle and myocardium compared to other models, showing its advantage in complex structure recognition.

[0138] Table 4 shows the segmentation results of different models on the ISIC2018 dataset. Compared with I2U-Net, MFIUNet has an average Dice score improvement of 0.69, indicating that MFIUNet exhibits excellent performance in skin lesion segmentation.

[0139] Table 4 Segmentation results of different models on the ISIC2018 dataset (%)

[0140]

[0141] Figure 13For the visualization of the segmentation results of different models on the ISIC2018 dataset, it can be seen that MFIUNet is significantly better than other models in terms of segmentation accuracy, and its segmentation boundary is highly consistent with the true label (Label). In the first and second rows of samples, UNet, HSHUNet and other models have obvious boundary blur and detail loss in the edge area of the target, while MFIUNet can more accurately depict the boundary of skin lesions. The advantage is mainly due to the innovative cross-scale window interactive attention mechanism introduced by MFIUNet, which effectively alleviates the semantic gap in the feature fusion process by fusing high-level semantics and low-level details at multiple levels, thereby achieving more stable and coherent semantic expression in complex scenarios.

[0142] In order to verify the influence of the main modules in MFIUNet on the performance of the model, an ablation experiment was conducted.

[0143] Under the same configuration, the segmentation effects of models containing and not containing the DSWIA module on the Synapse dataset were compared and evaluated, and the results are shown in Table 5. After introducing the DSWIA module, the overall performance of the model was significantly improved. In terms of overall performance, the average Dice improved from 84.00% to 84.94%, mIoU increased by 1.40, and HD95 decreased significantly by 5.2, indicating that the segmentation accuracy and boundary positioning ability of the model were both enhanced. At the organ level, the Dice scores of organs such as KL, KR, Liver and SM were significantly improved, especially on organs with complex shapes or blurred boundaries. Although the Dice scores of AORTA and PC decreased slightly, overall, the DSWIA module effectively enhanced the multi-scale feature fusion capability and improved the segmentation robustness and generalization ability of the model.

[0144] Table 5 Ablation experiment results of DSWIA module (unit: %)

[0145]

[0146] The difference between the performance of the model with and without the LSAS module on the Synapse dataset is compared, and the results are shown in Table 6. In the case of adding the LSAS module, the overall performance indicators of the model are improved, the Dice score is increased from 84.29% to 84.94%, the HD95 is reduced from 18.37 to 13.98, and the mIoU is increased from 75.26% to 76.47%, indicating that the LSAS module significantly improves the segmentation accuracy and boundary precision of the model. Except for the slight decrease in the pancreas (PC) indicator, the Dice score of all other organs has been improved to varying degrees, especially the gallbladder (GB) and left kidney (KL), which have been significantly improved, indicating that the LSAS module has significantly improved the segmentation effect of these structures. The experimental results show that the LSAS module can effectively enhance the feature expression ability and spatial attention mechanism of the model, and improve the segmentation accuracy and boundary detail capture ability of the model for multiple organs.

[0147] Table 6 LSAS module ablation experiment results (unit: %)

[0148]

[0149] The performance of the two models with and without the CAFB module on the Synapse dataset is compared, and the results are shown in Table 7. The introduction of the CAFB module significantly improves the multi-organ segmentation performance of the model on the Synapse dataset, with the overall Dice coefficient increasing from 83.21% to 84.94%, the HD95 distance decreasing from 17.15 to 13.98, and the mIoU increasing from 74.16% to 76.47%, indicating that the CAFB module effectively enhances the segmentation accuracy and boundary positioning ability. In terms of organs, the left kidney (KL) and stomach (SM) perform the most outstanding, with Dice increasing by 2.99 and 5.09, respectively. The pancreas (PC) and gallbladder (GB) also have significant improvements, with increases of 1.91 and 0.78, respectively. Although the liver (Liver) and other large organs have a smaller increase of 0.58, the overall trend verifies that the CAFB module significantly improves the recognition ability of small organs and complex structures by optimizing the complementary features of the double branch, and improves the global segmentation consistency.

[0150] Table 7 CAFB module ablation experiment results (unit: %)

[0151]

[0152] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. As long as the same or equivalent means achieve the technical effects of the present application, they should belong to the protection scope of the present application. Within the protection scope of the present application, the technical solutions and / or embodiments can have various modifications and changes.

Claims

1. A method for training a medical image segmentation model based on multi-scale feature interaction, characterized in that, The method comprises the following steps: constructing a medical image segmentation model comprising an encoder, a feature interaction module and a decoder; wherein the feature interaction module comprises a feature alignment module and a DSWIA module; The medical image segmentation model is trained using a medical image dataset, wherein the medical image is encoded using an encoder to obtain at least four levels of features 、 、 、 ; in the feature alignment module, the features of and 、 and are aligned to obtain transformed features and 、 and ; in the DSWIA module, the four groups of features obtained by sequentially selecting and are interacted to obtain features and , and the features of and are interacted to obtain features and ; the decoder is used to decode 、 、 、 ; updating parameters of the medical image segmentation model according to a target loss; wherein the window cycle selection comprises: For each feature and , the features are evenly divided into groups of non-overlapping sequential arrangement groups , each group of features containing M channels, where M is the greatest common divisor of and . From the set of features is taken consecutively The set forms a new subset of features with a window size of M*k The window is moved by M The features within the window are selected to form a new set of features until the subset of features appears repeatedly, each set of features containing M*k channels; wherein is , is ; represents the number of channels of represents the number of channels of represents the height of represents the height of represents the width of represents the width of represents a positive integer The DSWIA module comprises a feature window division and feature splicing module, a sparse feature interaction attention module, and a feature remodeling and restoration module. In the feature window division and feature splicing module, features are divided into window features are flattened and spliced respectively; in the sparse feature interaction attention module, cross-layer feature fusion is achieved through window attention and feature enhancement operation; in the feature remodeling and restoration module, the output features of the feature window division and feature splicing module are remodeled and merged.

2. The method of claim 1, wherein, The decoder includes a graph convolution attention module, an upsampling block, a local similarity perception sampler, a cross attention fusion module and a lightweight segmentation head, After processing by the graph convolution attention module and the upsampling block, are jointly input to the local similarity perception sampler for processing, and then are aggregated with to output , The graph convolution attention module, the lightweight segmentation head and the upsampling block generate a segmentation map ; The graph convolution attention module and the upsampling block, and are jointly input to the local similarity perception sampler for processing, and then are aggregated with to output , The graph convolution attention module, the lightweight segmentation head and the upsampling block generate a segmentation map ; The graph convolution attention module and the upsampling block, and are jointly input to the local similarity perception sampler for processing, and then are aggregated with to output ; The graph convolution attention module and are jointly input to the cross attention fusion module, and and are respectively output, and are respectively output by the lightweight segmentation head and the upsampling block to output segmentation maps and ; finally , , and are aggregated by addition, and after being activated by a Sigmoid activation function, an ultimate prediction segmentation map is output.

3. The method of claim 2, wherein, The graph convolution attention module is composed of a graph convolution block and a spatial attention block, and the operation on the input feature is as shown in the following formula: wherein, represents an input feature; represents a graph convolution attention module; represents a graph convolution block; represents a spatial attention block.

4. The method of claim 2, wherein, The local similarity perception sampler sequentially performs local cosine similarity calculation, similarity-guided offset prediction and deformable feature sampling on the input feature.

5. The method of claim 2, wherein, The cross-attention fusion module captures the local correlation between features through spatial attention, dynamically balances the original feature and the attention feature by using the gating mechanism, and performs global interaction through cross-attention. 6.A medical image segmentation method based on multi-scale feature interaction, characterized in that, The method comprises the following steps: inputting the image to be segmented into the medical image segmentation model, and outputting a segmentation result from the medical image segmentation model; wherein the medical image segmentation model is obtained by the medical image segmentation model training method as claimed in any one of claims 1-5.

7. An electronic device, comprising: comprise: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions that, when executed by the at least one processor, implement the medical image segmentation model training method based on multi-scale feature interaction as claimed in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The memory stores instructions that, when executed by the processor, implement the medical image segmentation model training method based on multi-scale feature interaction as claimed in any one of claims 1-5.

Images