A mobile sam-based lightweight breast mass MRI automatic segmentation method

Abstract

This invention discloses a lightweight automatic MRI segmentation method for breast masses based on MobileSAM. The method inputs the breast MRI image to be detected into a breast mass MRI image segmentation model, outputting the localization and prediction results of the breast mass. The model uses the MobileSAM lightweight image coding network as its backbone, and introduces an adapter to achieve adaptation and efficient fine-tuning of the breast image domain. A separable hollow pyramid fusion module is designed in the decoding branch, employing multi-branch deep separable hollow convolution to aggregate multi-scale context, enhancing the feature discrimination of small masses and blurred boundaries. Furthermore, a coordinate-channel-space joint attention gating module is introduced, using decoded semantics as a guide to filter skip features, strengthening the spatial location and detail reconstruction of the mass boundary. This invention achieves end-to-end automatic segmentation without interactive prompts, significantly improving the accuracy and robustness of breast MRI mass segmentation while maintaining lightweight design and low training costs.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and more particularly to a lightweight automatic MRI segmentation method for breast masses based on MobileSAM. Background Technology

[0002] Breast cancer is the most common malignant tumor among women worldwide, and early and accurate diagnosis is crucial for improving patient survival rates. Magnetic resonance imaging (MRI) of the breast, with its high soft tissue resolution and multi-sequence imaging capabilities, has become an important imaging tool for breast cancer screening, diagnosis, and treatment evaluation. Compared to mammography, MRI has significant advantages in detecting dense breast tissue and multifocal, multicentric lesions. However, MRI images themselves are characterized by high data complexity, low tissue contrast, and significant noise interference. In particular, breast masses often exhibit blurred edges (such as spiculated or invasive growth), irregular shapes, and large scale differences (especially small lesions). These factors introduce significant subjective differences and time costs for manual interpretation by physicians, posing a serious challenge to fully automated, high-precision computer-aided segmentation.

[0003] In recent years, deep learning-based medical image segmentation methods have made significant progress, especially the encoder-decoder architecture represented by U-Net and its derivative models (such as U-Net++ and Attention U-Net), which have performed well in many medical image segmentation tasks. However, when it comes to the specific task of segmenting breast MRI masses, such general architectures still have obvious limitations: First, when dealing with low-contrast regions, small-scale masses, and complex glandular backgrounds, the models are prone to problems such as coarse segmentation boundaries, missed detection of small lesions, or adhesion to surrounding tissues; Second, the model performance heavily depends on large-scale, high-quality labeled data, while the labeling cost of medical images is extremely high, and the consistency of labeling is difficult to guarantee.

[0004] Meanwhile, the Segment Anything Model (SAM), proposed by Meta in 2023, pioneered a new paradigm for general visual segmentation, achieving zero-shot generalization through interactive prompts. However, the original SAM model, based on the ViT-Huge architecture, has a parameter count as high as 637M, resulting in enormous computational and storage overhead, making it difficult to deploy in real time on commonly used clinical hardware. More importantly, SAM was originally designed for general segmentation and interactive editing of natural images. Its pre-training data and task objectives differ significantly from those of medical images. When directly transferred to breast MRI segmentation, it often faces problems such as insufficient modeling of medical-specific features (e.g., weak boundaries, small targets) and unstable segmentation results. Furthermore, its reliance on interactive prompts such as points and boxes cannot meet the urgent need for fully automated, non-interactive operation processes in large-scale clinical screening.

[0005] Therefore, how to construct a more refined, generalizable, and clinically deployable segmentation modeling mechanism for fuzzy infiltrative boundaries and small target lesions while ensuring model lightweighting and inference efficiency has become a key research focus and challenge in the field of breast cancer MRI image segmentation. Summary of the Invention

[0006] To address the aforementioned shortcomings of existing technologies, this invention provides a lightweight automatic MRI segmentation method for breast masses based on MobileSAM. By constructing a lightweight MobileSAM encoding branch and a decoding branch that includes a separable hollow pyramid fusion module and a coordinate-channel-space joint attention gating module, this invention solves the technical problems of existing breast MRI mass segmentation models having a large number of parameters, high computational overhead, insufficient segmentation accuracy for small target masses and blurred boundaries, and difficulty in achieving fully automatic and accurate segmentation in resource-constrained scenarios.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A lightweight MRI automatic segmentation method for breast masses based on MobileSAM is proposed. The method acquires a breast MRI image to be detected, inputs the breast MRI image to be detected into a pre-trained breast mass MRI image segmentation model, and outputs the pixel-level breast mass segmentation result of the breast MRI image to be detected.

[0009] The breast mass MRI image segmentation model includes an encoding branch and a decoding branch. The encoding branch includes a shallow convolutional feature extraction module and a MobileSAM encoding module with an adapter, connected in sequence, for multi-scale feature extraction and domain adaptation of the input breast MRI image, outputting multi-level feature maps including shallow details and deep semantics respectively. The decoding branch includes a separable hollow pyramid fusion module and a coordinate-channel-space joint attention gating module, connected in sequence, for cross-scale aggregation, attention filtering and resolution-level recovery of the multi-level feature maps output by the encoding branch, and finally outputting a pixel-level mass segmentation mask with the same resolution as the input breast MRI image as the pixel-level breast mass segmentation result.

[0010] As a preferred embodiment, the shallow convolutional feature extraction module includes two cascaded convolutional units, each of which includes a cascaded 3×3 convolutional layer, a BN layer, and a ReLU activation layer.

[0011] The input feature map is used as the input to the first convolutional unit. It is processed by a 3×3 convolutional layer for feature extraction, and then normalized by a BN layer. After non-linear activation by a ReLU activation layer, the primary features are obtained. The primary features output by the first convolutional unit are used as the input to the second convolutional unit for shallow feature extraction. After performing the same operation as the first convolutional unit, the original-scale shallow feature map output by the second convolutional unit is used as the output of the shallow convolutional feature extraction module.

[0012] As a preferred embodiment, the MobileSAM coding module with adapter includes a shallow coding unit, three TinyViT coding units, and three Reshape convolutional units connected in sequence; the shallow coding unit includes a cascaded segmentation embedding unit and an MBConv module, and the TinyViT coding unit includes a cascaded downsampling fusion unit and a TinyViT attention module with adapter.

[0013] In the MobileSAM encoding module with adapter, the original-scale shallow feature map obtained from the shallow convolutional feature module is used as the input to the shallow encoding unit for initial encoding, outputting a first encoded feature map at a scale of 1 / 2. The output of the shallow encoding unit is used as the input to the first TinyViT encoding unit, and the output of the first TinyViT encoding unit is used as the input to the second TinyViT encoding unit and the first Reshape convolutional unit, respectively. After processing by the first Reshape convolutional unit, a second encoded feature map at a scale of 1 / 4 is output. The output of the second TinyViT encoding unit is used as the input to the third TinyViT encoding unit and the second Reshape convolutional unit, respectively. After processing by the second Reshape convolutional unit, a third encoded feature map at a scale of 1 / 8 is output. The output of the third TinyViT encoding unit is used as the input to the third Reshape convolutional unit, and after processing by the third Reshape convolutional unit, a fourth encoded feature map at a scale of 1 / 16 is output.

[0014] The first, second, third, and fourth encoded feature maps are used as the output of the encoding branches to form a multi-level feature map that includes shallow details and deep semantics.

[0015] As a preferred embodiment, the MBConv module includes a cascaded first 1×1 convolutional layer, a first GELU activation layer, a 3×3 depth convolutional layer, a second GELU activation layer, a second 1×1 convolutional layer, a descent path layer, and a third GELU activation layer. The output of the descent path layer is added to the input of the MBConv module by residual addition, and then passed through the third GELU activation layer to obtain the output of the MBConv module.

[0016] The TinyViT attention module with adapter includes a window attention submodule and an MLP submodule connected in sequence. The window attention submodule includes a cascaded layer normalization layer, a window attention layer, and a descent path layer, and a first adapter connected in series after the descent path layer. The output of the first adapter is added to the input of the window attention submodule by residual addition, and this is used as the output of the window attention submodule. The MLP submodule includes a cascaded local depth convolutional layer, a layer normalization layer, an MLP layer, and a descent path layer, and a second adapter connected in parallel after the descent path layer. The output of the window attention submodule is used as the input of the local depth convolutional layer, and the output of the local depth convolutional layer is used as the input of the layer normalization layer and the second adapter, respectively. The output of the layer normalization layer is processed by the MLP layer and the descent path layer in sequence, and then added to the output of the second adapter by residual addition, to obtain the output of the MLP submodule, which is used as the overall output of the TinyViT attention module.

[0017] The first adapter and the second adapter each include a lower projection layer, a first GELU activation layer, an upper projection layer, and a second GELU activation layer. The output of the second GELU activation layer is added to the input of the adapter by residual addition, and then used as the output of the first adapter and the second adapter.

[0018] As a preferred embodiment, the decoding branch includes four-scale separable hollow pyramid fusion modules, as well as four-scale coordinate-channel-space joint attention gating modules and four-scale decoding and reconstruction dual convolution modules that correspond one-to-one with the four-scale separable hollow pyramid fusion modules.

[0019] The input to the decoding branch includes a shallow feature map, a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map; the input to the original-scale separable hollow pyramid fusion module includes a shallow feature map, a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map. Figure 5 Level feature maps; the input to the 1 / 2-scale separable hollow pyramid fusion module includes a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map. Figure 4 Level feature maps; the input to the 1 / 4-scale separable hollow pyramid fusion module includes second, third, and fourth coded feature maps. Figure 3 Level 1 feature map; The input of the 1 / 8 scale separable hollow pyramid fusion module includes two levels of feature maps: the third-level coded feature map and the fourth-level coded feature map;

[0020] In the decoding branch, the fourth encoded feature map at a scale of 1 / 16 is used as the starting point for progressive upsampling. After bilinear upsampling, a fourth upsampled feature map at a scale of 1 / 8 is obtained. The fourth upsampled feature map and the fused jump feature output from the separable hollow pyramid fusion module at a scale of 1 / 8 are input into the coordinate-channel-space joint attention gating module at a scale of 1 / 8. The output jump feature map at a scale of 1 / 8 is then input into the decoding and reconstruction dual convolution module at a scale of 1 / 8, and after bilinear upsampling, a third upsampled feature map at a scale of 1 / 4 is obtained.

[0021] The third upsampled feature map and the fused jump feature output from the 1 / 4 scale separable hollow pyramid fusion module are input together into the 1 / 4 scale coordinate-channel-space joint attention gating module. The output 1 / 4 scale jump feature map is then input into the 1 / 4 scale decoding and reconstruction dual convolution module, and then bilinear upsampling is performed to obtain the second upsampled feature map at the 1 / 2 scale.

[0022] The second upsampled feature map and the fused jump feature output from the 1 / 2 scale separable hollow pyramid fusion module are input together into the 1 / 2 scale coordinate-channel-space joint attention gating module. The output 1 / 2 scale jump feature map is then input into the 1 / 2 scale decoding and reconstruction dual convolution module, and then bilinear upsampling is performed to obtain the first upsampled feature map at the original scale.

[0023] The first upsampled feature map and the fused jump feature output from the original-scale separable hollow pyramid fusion module are input together into the original-scale coordinate-channel-space joint attention gating module. The output original-scale jump feature map is then input into the original-scale decoding and reconstruction dual convolution module, and after bilinear upsampling and 1×1 convolution mapping, a pixel-level mass segmentation mask with the same resolution as the input breast MRI image is output as the overall output of the decoding branch.

[0024] As a preferred embodiment, the processing steps of the separable dilated pyramid fusion module include: aligning the input multi-level feature maps to the target scale and concatenating them along the channel dimension to obtain a fused input feature map at that scale; performing 1×1 convolutional dimensionality reduction on the fused input feature map to obtain a dimensionality-reduced feature map; constructing multiple parallel branches, each branch using depthwise separable dilated convolution to process the dimensionality-reduced feature map, wherein the dilation rate of each branch is set incrementally according to the branch number; performing two-dimensional random dropout processing on the feature maps output by each branch, concatenating the processed branch feature maps along the channel dimension to obtain a multi-branch concatenated feature map; and performing 1×1 convolutional fusion processing on the multi-branch concatenated feature map to obtain a fused jump feature at that scale.

[0025] As a preferred embodiment, the coordinate-channel-space joint attention gating module includes a coordinate attention branch, a channel attention branch, and a spatial attention branch; wherein, the input of the coordinate-channel-space joint attention gating module includes an upsampled feature map of the corresponding scale and a fused jump feature, and the upsampled feature map is used as a guiding feature to sequentially perform horizontal and vertical coordinate attention enhancement, global channel attention enhancement, and spatial position attention enhancement on the fused jump feature;

[0026] In the coordinate-channel-space joint attention gating module, the upsampled feature map of the corresponding scale and the fused jump feature are concatenated by channels and used as the input of the coordinate attention branch. The feature is compressed in the horizontal and vertical directions by global average pooling layers along the horizontal and vertical directions, respectively, to obtain horizontal pooling feature maps and vertical pooling feature maps. These are then concatenated in the channel dimension and processed by convolutional layers. The processed feature map is split into two independent feature maps, which are activated by Sigmoid and then added to the input feature map by residual addition to obtain the feature map enhanced by coordinate attention.

[0027] The feature map enhanced by coordinate attention is used as the input of the channel attention branch. Global spatial information is compressed by global average pooling layer and global max pooling layer respectively to obtain average pooling feature map and max pooling feature map. Then, the average pooling feature map and max pooling feature map are processed by shared multilayer perceptron and added element by element. The added feature map is passed through sigmoid activation layer and then added with the residual of the input feature map of the channel attention branch to obtain the feature map enhanced by channel attention.

[0028] The feature map enhanced by channel attention is used as the input of the spatial attention branch. The input feature map is compressed in the channel dimension by channel compression to obtain the spatial attention feature map. The spatial attention feature map is processed by 7×7 convolution and then Sigmoid activation is used to map the convolutional feature map into spatial attention weights. The spatial attention weights are added to the input feature map by residual addition to obtain the feature map enhanced by spatial attention.

[0029] The feature map enhanced by spatial attention is activated by Sigmoid and then added to the input of the coordinate-channel-space joint attention gating module. The output is the jump feature map gated by the coordinate-channel-space joint attention gating module.

[0030] As a preferred embodiment, the breast mass MRI image segmentation model is trained in the following manner:

[0031] Pre-annotated pixel-level breast MRI images of the lump region are used as training samples to form a training sample set, which is then input into the breast lump MRI image segmentation model. A total loss function consisting of segmentation region loss and edge prediction loss is constructed, and the model parameters of the breast lump MRI image segmentation model are optimized and updated with the goal of minimizing the total loss function, thereby training the breast lump MRI image segmentation model.

[0032] As a preferred embodiment, the segmentation region loss is:

[0033] ;

[0034] In the formula, The loss function for segmenting regions; The region BCE loss function; The Dice loss function for the region;

[0035] The region BCE loss function is:

[0036] ;

[0037] In the formula, This represents the total number of pixels in the image. For the first The real label of each pixel; The model predicts the first The probability that a pixel belongs to a tumor region;

[0038] The Dice loss function for the region is:

[0039] ;

[0040] In the formula, The model predicts the first The probability that a pixel belongs to a tumor region; For the first The real label of each pixel; It is a very small constant;

[0041] The edge prediction loss is:

[0042] ;

[0043] In the formula, The edge prediction loss function; The edge BCE loss function; Edge Dice loss function;

[0044] The edge BCE loss function is:

[0045] ;

[0046] In the formula, For the first Predicted edge strength of each pixel; For the first The true edge label of each pixel;

[0047] .

[0048] As a preferred embodiment, the total loss function is:

[0049] ;

[0050] In the formula, This is the total loss function.

[0051] Compared with the prior art, the present invention has the following technical effects:

[0052] 1. This invention constructs a lightweight coding branch based on MobileSAM and introduces an adapter to achieve domain-adaptive fine-tuning of breast MRI images. Under the premise of freezing most of the main parameters, only a small number of incremental parameters are optimized, which significantly reduces the number of trainable parameters and fine-tuning costs of the model. It effectively solves the problem that general large models are difficult to deploy in resource-constrained scenarios and achieves a balance between lightweight model and high segmentation accuracy.

[0053] 2. This invention employs a pyramid-style multi-scale feature fusion mechanism in the decoding branch, designs a separable hollow pyramid fusion module, and uses multi-branch depthwise separable hollow convolution to extract contextual information from different receptive fields in parallel. Through cross-scale feature aggregation, it enhances the feature discrimination ability for small target masses and low-contrast blurred boundaries, as well as the identification and segmentation ability for lesions with significant morphological differences and complex background interference. Furthermore, in the skip connection fusion process, a coordinate-channel-space joint attention gating module is introduced to perform multi-dimensional screening and enhancement of the fused skip features guided by the decoded semantic features. This strengthens the joint modeling of lesion spatial location information, channel dependencies, and key boundary responses, thereby reducing the risk of losing detailed information during the stepwise recovery process and improving the segmentation accuracy of small targets and blurred boundaries.

[0054] 3. Unlike existing interactive segmentation methods that typically require additional prompts (such as dots / boxes / text hints) to locate the target region, this invention introduces an edge prediction loss function during training. By explicitly supervising the prediction quality of the mass edge region through the Sobel operator, the model's ability to characterize spiculated and invasive boundaries is further enhanced. Ultimately, this invention achieves end-to-end fully automatic segmentation without manual prompts. While maintaining low training costs and fast inference, it significantly improves the accuracy, robustness, and clinical applicability of mass segmentation in breast MRI images. Attached Figure Description

[0055] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0056] Figure 1 This is a structural diagram of the MRI image segmentation model for breast masses used in the method disclosed in this invention;

[0057] Figure 2 This is a structural diagram of the MBConv module in the MobileSAM encoding module of this invention embodiment;

[0058] Figure 3 This is a structural diagram of the TinyViT attention module and adapter in the MobileSAM encoding module of this invention embodiment;

[0059] Figure 4 This is a structural diagram of the separable hollow pyramid fusion module in an embodiment of the present invention;

[0060] Figure 5 This is a structural diagram of the coordinate-channel-space joint attention gating module in an embodiment of the present invention;

[0061] Figure 6 This is a schematic diagram of the segmentation results in an embodiment of the present invention. Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0064] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not indicate that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0065] Example:

[0066] Existing general-purpose segmentation models such as U-Net and its variants are prone to missed detections and rough edges when dealing with small target masses and blurred boundaries, making it difficult to meet the clinical demand for high-precision segmentation. While the SAM large-scale model proposed in recent years has strong generalization ability, its large number of parameters and high computational cost, as well as its reliance on interactive prompting mechanisms, make it difficult to achieve fully automated deployment in resource-constrained primary healthcare scenarios. To overcome these shortcomings, this invention proposes a lightweight automatic MRI segmentation method for breast masses based on MobileSAM. This method uses a breast mass MRI image segmentation model to locate and segment breast masses in breast MRI images, thereby obtaining the breast mass segmentation results. The model uses lightweight MobileSAM as the encoding backbone and introduces an adapter to achieve domain-adaptive fine-tuning. Under the premise of freezing the backbone parameters, only a small number of incremental parameters are optimized, which significantly reduces training costs and model complexity. At the same time, a separable hollow pyramid fusion module and a coordinate-channel-space joint attention gating module are designed at the decoding end to enhance multi-scale context awareness and fine boundary modeling capabilities, respectively, ultimately achieving end-to-end automatic segmentation without interactive prompts. While ensuring lightweight design and inference efficiency, this invention significantly improves the segmentation accuracy of small target tumors and ambiguous boundaries, providing a feasible technical solution for intelligent clinical breast cancer screening that balances high performance and low resource consumption.

[0067] Specifically, the present invention proposes a lightweight automatic breast mass MRI segmentation method based on MobileSAM. This method inputs the acquired breast MRI image to be detected into a pre-trained breast mass MRI image segmentation model and outputs the pixel-level breast mass segmentation result of the breast MRI image to be detected.

[0068] To better illustrate the technical solution of this invention, the following sections provide a more detailed explanation.

[0069] I. MRI Image Segmentation Model for Breast Masses

[0070] The breast mass MRI image segmentation model of the present invention includes an encoding branch and a decoding branch, such as Figure 1As shown, the breast mass MRI image segmentation model includes an encoding branch and a decoding branch. The encoding branch includes a shallow convolutional feature extraction module and a MobileSAM encoding module with an adapter, connected in sequence, used to perform multi-scale feature extraction and domain adaptation on the input breast MRI image, outputting multi-level feature maps including shallow details and deep semantics. The decoding branch includes a separable hollow pyramid fusion module and a coordinate-channel-space joint attention gating module, connected in sequence, used to perform cross-scale aggregation, attention filtering, and progressive resolution restoration on the multi-level feature maps output by the encoding branch, finally outputting a pixel-level mass segmentation mask with the same resolution as the input breast MRI image as the pixel-level breast mass segmentation result. The pixel-level breast mass segmentation result is only used as annotation data for doctors to focus on the breast mass region, thereby better assisting doctors in their work.

[0071] The encoding and decoding branches of the MRI image segmentation model for this breast mass are described in detail below.

[0072] 1. Encoded branches

[0073] In this embodiment, the encoding branch sequentially includes a shallow convolutional feature extraction module and a mobile segmentation pre-training encoding module with adapter. It is mainly used to extract multi-scale feature representations from breast images from shallow to deep layers, which not only preserves details such as texture and weak boundaries, but also obtains stable high-level semantic information. It also aligns the features at each stage to the scale and channel specifications required by the network, providing a unified feature interface for subsequent multi-scale fusion and decoding reconstruction, thereby improving the segmentation and discrimination ability of low-contrast lesion areas and blurred boundaries.

[0074] The following sections will provide a detailed introduction to each module of this coding branch.

[0075] 1.1 Shallow Convolution Feature Extraction Module

[0076] In this embodiment, the shallow convolutional feature extraction module includes two cascaded convolutional units, each of which includes a cascaded 3×3 convolutional layer, a BN layer, and a ReLU activation layer; this structure extracts detailed information such as texture and weak boundaries without reducing spatial resolution.

[0077] In practice, this module directly processes the input breast MRI image, with input feature dimensions of B×3×H×W. First, the input feature map is used as input to the first convolutional unit, where features are extracted through a 3×3 convolutional layer. After normalization through a BN layer, non-linear activation is performed through a ReLU activation layer to obtain primary features. The primary features output from the first convolutional unit are used as input to the second convolutional unit for shallow feature extraction. After performing the same operations as the first convolutional unit, the original-scale shallow feature map x0_0 output from the second convolutional unit is used as the output of the shallow convolutional feature extraction module. This module maps the number of channels to 32, and the output feature dimensions are B×32×H×W. This module is used to preserve texture variations, weak boundary cues, and small target structural information in breast images, and its output provides high-resolution detail support for subsequent pyramid fusion and boundary refinement reconstruction.

[0078] 1.2 MobileSAM encoding module with adapter

[0079] In this embodiment, the module uses MobileSAM's TinyViT as the pre-trained semantic coding backbone and introduces an adapter to achieve domain adaptation for the breast image segmentation task. Specifically, the module includes a shallow coding unit, three TinyViT coding units, and three Reshape convolutional units connected in sequence; the shallow coding unit includes a cascaded segmentation embedding unit and an MBConv module, and the TinyViT coding unit includes a cascaded downsampling fusion unit and a TinyViT attention module with an adapter;

[0080] In the MobileSAM encoding module with adapter, the original-scale shallow feature map x0_0 obtained from the shallow convolutional feature module is used as the input of the shallow encoding unit for initial encoding, outputting a first encoded feature map x1_0 at a scale of 1 / 2. The output of the shallow encoding unit is used as the input of the first TinyViT encoding unit, and the output of the first TinyViT encoding unit is used as the input of the second TinyViT encoding unit and the first Reshape convolutional unit, respectively. After processing by the first Reshape convolutional unit, a second encoded feature map x2_0 at a scale of 1 / 4 is output. The output of the second TinyViT encoding unit is used as the input of the third TinyViT encoding unit and the second Reshape convolutional unit, respectively. After processing by the second Reshape convolutional unit, a third encoded feature map x3_0 at a scale of 1 / 8 is output. The output of the third TinyViT encoding unit is used as the input of the third Reshape convolutional unit, and after processing by the third Reshape convolutional unit, a fourth encoded feature map x4_0 at a scale of 1 / 16 is output.

[0081] like Figure 2As shown, the MBConv module includes a cascaded first 1×1 convolutional layer, a first GELU activation layer, a 3×3 depth convolutional layer, a second GELU activation layer, a second 1×1 convolutional layer, a descent path layer, and a third GELU activation layer. The output of the descent path layer is added to the input of the MBConv module by residual addition, and then passed through the third GELU activation layer to obtain the output of the MBConv module.

[0082] In specific implementation, the input feature specification of the MobileSAM encoding module is B×3×H×W. The module first performs scale normalization processing, and the normalized input feature specification is B×3×1024×1024. This processing is used to meet the requirement of input resolution consistency of the pre-trained encoding backbone. Subsequently, the normalized input is fed into the TinyViT backbone and staged encoding is performed. The structure configuration of the TinyViT backbone is as follows: embed_dims equals 64, 128, 160, 320; depths equals 2, 2, 6, 2; num_heads equals 2, 4, 5, 10; window_sizes equals 7, 7, 14, 7; drop_path_rate equals 0. The backbone outputs four stages of semantic features through a phased forward process, and performs scale remapping processing through Reshape convolution. The semantic features of stage one are remapped to 1 / 2 resolution, with a spatial size of H / 2 × W / 2; the semantic features of stage two are remapped to 1 / 4 resolution, with a spatial size of H / 4 × W / 4; the semantic features of stage three are remapped to 1 / 8 resolution, with a spatial size of H / 8 × W / 8; and the semantic features of stage four are remapped to 1 / 16 resolution, with a spatial size of H / 16 × W / 16. This processing ensures spatial alignment of the multi-scale interfaces in subsequent networks.

[0083] Subsequently, the back-mapping features undergo channel alignment processing, specifically: in Stage 1, feature channels are mapped from 64 to 64, and the output feature x1_0 has a specification of B×64×H / 2×W / 2; in Stage 2, feature channels are mapped from 128 to 128, and the output feature x2_0 has a specification of B×128×H / 4×W / 4; in Stage 3, feature channels are mapped from 160 to 256, and the output feature x3_0 has a specification of B×256×H / 8×W / 8; in Stage 4, feature channels are mapped from 320 to 512, and the output feature x4_0 has a specification of B×512×H / 16×W / 16. This processing is used to match the channel specifications at the decoding end and form a unified multi-scale coding interface. The module loads MobileSAM pre-trained weights to initialize the TinyViT backbone.

[0084] Meanwhile, this embodiment inserts an adapter within the main structure. Specifically, the adapter module is only inserted into Stages 1 to 3 of the MobileSAM encoder, corresponding to multiple TinyViT attention modules, such as... Figure 3 As shown, the TinyViT attention module with adapter includes a window attention submodule and an MLP submodule connected in sequence. The window attention submodule includes a cascaded layer normalization layer, a window attention layer, and a descent path layer, and a first adapter connected in series after the descent path layer. The output of the first adapter is added to the input of the window attention submodule by residual addition, and this is used as the output of the window attention submodule. The MLP submodule includes a cascaded local depthwise convolutional layer, a layer normalization layer, an MLP layer, and a descent path layer, and a second adapter connected in parallel after the descent path layer. The output of the module serves as the input to the local depthwise convolutional layer. The output of the local depthwise convolutional layer serves as the input to the layer normalization layer and the second adapter. The output of the layer normalization layer is processed sequentially by the MLP layer and the descent path layer, and then residually added to the output of the second adapter to obtain the output of the MLP submodule, which serves as the overall output of the TinyViT attention module. Both the first and second adapters include a downprojection layer, a first GELU activation layer, an upprojection layer, and a second GELU activation layer. The output of the second GELU activation layer is residually added to the adapter's input, and then residually added to the first and second adapters. Therefore, the adapter simultaneously embeds the window attention submodule and the MLP submodule in the form of residual bypass increments. This provides learnable feature compensation and domain adaptation capabilities for each TinyViT attention module while minimizing perturbation to the pre-trained backbone representation. The adapter provides learnable feature compensation capabilities without significantly perturbing the pre-trained backbone, and it allows for lightweight fine-tuning while freezing the backbone parameters. During training, adapter parameters are involved in optimization, while most parameters of the backbone remain frozen. The adapter performs incremental correction on stage features in the feature pathway, focusing on semantic differences in low-contrast regions, blurred boundaries, and small target masses. This mechanism improves the model's adaptability to breast imaging data distribution with lower training costs, while maintaining the semantic extraction efficiency and stability of the backbone.

[0085] In summary, the MobileSAM coding structure with adapter effectively enhances the network's performance in breast image segmentation by combining the adapter with TinyViT's powerful semantic extraction capabilities, particularly in detail recovery in complex backgrounds and low-contrast regions. The introduction of the adapter allows the network to perform domain adaptation for breast image data with low-cost fine-tuning, and improves its accuracy and robustness in breast image segmentation by enhancing detail feature extraction and boundary strengthening capabilities.

[0086] 2. Decoding branch

[0087] In this embodiment, the decoding section is used to progressively restore the deep semantics obtained from the encoding end to the original resolution. During the restoration process, the encoded features at different scales are first aggregated at multiple scales and then fused by skip fusion, thereby utilizing both global context and local detail information. In this decoding branch, the deep semantics from the encoding branch are progressively restored to the original resolution. Multi-scale aggregation and skip fusion are introduced during the restoration process. The design of this encoding branch utilizes both global context and local detail information. During decoding, five levels of encoded features are used as input, with the input feature set being x0_0, x1_0, x2_0, x3_0, and x4_0. The decoding branch first generates multi-scale fused skip features, and then progressively upsamples from a 1 / 16 scale. At each level, gated fusion and convolutional reconstruction are performed, ultimately outputting a pixel-level segmentation result with a feature specification of B×1×H×W. This process enhances the reconstruction capability for low-contrast bulk regions, small target structures, and blurred boundaries.

[0088] In this embodiment, the decoding branch includes four-scale separable hollow pyramid fusion modules, and four-scale coordinate-channel-space joint attention gating modules and four-scale decoding and reconstruction dual convolution modules that correspond one-to-one with the four-scale separable hollow pyramid fusion modules.

[0089] The input to the decoding branch includes a shallow feature map, a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map; the input to the original-scale separable hollow pyramid fusion module includes a shallow feature map, a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map. Figure 5 Level feature maps; the input to the 1 / 2-scale separable hollow pyramid fusion module includes a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map. Figure 4 Level feature maps; the input to the 1 / 4-scale separable hollow pyramid fusion module includes second, third, and fourth coded feature maps. Figure 3 Level 1 feature map; The input of the 1 / 8 scale separable hollow pyramid fusion module includes two levels of feature maps: the third-level coded feature map and the fourth-level coded feature map;

[0090] In the decoding branch, the fourth encoded feature map at a scale of 1 / 16 is used as the starting point for progressive upsampling. After bilinear upsampling, a fourth upsampled feature map at a scale of 1 / 8 is obtained. The fourth upsampled feature map and the fused jump feature output from the separable hollow pyramid fusion module at a scale of 1 / 8 are input into the coordinate-channel-space joint attention gating module at a scale of 1 / 8. The output jump feature map at a scale of 1 / 8 is then input into the decoding and reconstruction dual convolution module at a scale of 1 / 8, and after bilinear upsampling, a third upsampled feature map at a scale of 1 / 4 is obtained.

[0091] The third upsampled feature map and the fused jump feature output from the 1 / 4 scale separable hollow pyramid fusion module are input together into the 1 / 4 scale coordinate-channel-space joint attention gating module. The output 1 / 4 scale jump feature map is then input into the 1 / 4 scale decoding and reconstruction dual convolution module, and then bilinear upsampling is performed to obtain the second upsampled feature map at the 1 / 2 scale.

[0092] The second upsampled feature map and the fused jump feature output from the 1 / 2 scale separable hollow pyramid fusion module are input together into the 1 / 2 scale coordinate-channel-space joint attention gating module. The output 1 / 2 scale jump feature map is then input into the 1 / 2 scale decoding and reconstruction dual convolution module, and then bilinear upsampling is performed to obtain the first upsampled feature map at the original scale.

[0093] The first upsampled feature map and the fused jump feature output from the original-scale separable hollow pyramid fusion module are input together into the original-scale coordinate-channel-space joint attention gating module. The output original-scale jump feature map is then input into the original-scale decoding and reconstruction dual convolution module, and after bilinear upsampling and 1×1 convolution mapping, a pixel-level mass segmentation mask with the same resolution as the input breast MRI image is output as the overall output of the decoding branch.

[0094] The following sections will provide a detailed introduction to each module of the decoding branch.

[0095] 2.1 Separable Hollow Pyramid Fusion Module

[0096] like Figure 4 As shown, the processing steps of the separated void pyramid fusion module include the following:

[0097] The input multi-level feature maps are aligned to the target scale and concatenated along the channel dimension to obtain a fused input feature map at that scale. The fused input feature map is then subjected to 1×1 convolutional dimensionality reduction to obtain a dimensionality-reduced feature map. Multiple parallel branches are constructed, each using depthwise separable dilated convolution to process the dimensionality-reduced feature map. The dilation rate of each branch is set incrementally according to the branch number. Specifically, the dilation rates of the original scale fusion units are 1, 2, 4, 8, and 16; the 1 / 2 scale fusion units are 1, 2, 4, and 8; the 1 / 4 scale fusion units are 1, 2, and 4; and the 1 / 8 scale fusion units are 1 and 2. Two-dimensional random dropout processing is applied to the feature maps output from each branch. The random dropout rate of the four units is... With probabilities 0.15, 0.15, 0.25, and 0.25, the processed branch feature maps are concatenated along the channel dimension to obtain a multi-branch concatenated feature map. The multi-branch concatenated feature map is then subjected to 1×1 convolutional fusion processing to obtain fused jump features at that scale, with feature specifications of B×32×H×W, B×64×H / 2×W / 2, B×128×H / 4×W / 4, and B×256×H / 8×W / 8, respectively. Through the above processing, the separable hollow pyramid fusion module performs cross-scale aggregation on the multi-level feature maps input from the encoded branches, generating fused jump features corresponding to the original scale, 1 / 2 scale, 1 / 4 scale, and 1 / 8 scale, denoted as SAPF1, SAPF2, SAPF3, and SAPF4, respectively.

[0098] In practice, this module is set up before decoding begins to aggregate the multi-scale features output by the encoding branch across scales for subsequent level-by-level decoding. Each level of fusion does not use the same feature set, but rather reduces shallow detail inputs in a pyramid manner, retaining only deeper semantics to participate in fusion, thereby achieving progressive feature integration from fine to coarse and from local to global. Specifically, taking the original scale fusion unit as an example, its input is a five-level feature set, namely x0_0, x1_0, x2_0, x3_0, and x4_0, with feature dimensions of B×32×H×W, B×64×H / 2×W / 2, B×128×H / 4×W / 4, B×256×H / 8×W / 8, and B×512×H / 16×W / 16, respectively. In the 1 / 2 scale fusion unit, the original scale shallow features are no longer introduced; only deeper features are fused, namely x1_0, x2_0, x3_0, and x4_0, with feature dimensions of B×64×H / 2×W / 2, B×128×H / 4×W / 4, B×256×H / 8×W / 8, and B×512×H / 16×W / 16, and so on.

[0099] 2.2 Coordinate-Channel-Spatial Joint Attention Gating Module

[0100] like Figure 5As shown, the coordinate-channel-space joint attention gating module includes a coordinate attention branch, a channel attention branch, and a spatial attention branch;

[0101] The input of the coordinate-channel-space joint attention gating module includes upsampled feature maps of the corresponding scale and fused jump features. The upsampled feature maps are used as guiding features to sequentially perform horizontal and vertical coordinate attention enhancement, global channel attention enhancement, and spatial position attention enhancement on the fused jump features.

[0102] In the coordinate-channel-space joint attention gating module, the upsampled feature map of the corresponding scale and the fused jump feature are concatenated by channels and used as the input of the coordinate attention branch. The feature is compressed in the horizontal and vertical directions by global average pooling layers along the horizontal and vertical directions, respectively, to obtain horizontal pooling feature maps and vertical pooling feature maps. These are then concatenated in the channel dimension and processed by convolutional layers. The processed feature map is split into two independent feature maps, which are activated by Sigmoid and then added to the input feature map by residual addition to obtain the feature map enhanced by coordinate attention.

[0103] The feature map enhanced by coordinate attention is used as the input of the channel attention branch. Global spatial information is compressed by global average pooling layer and global max pooling layer respectively to obtain average pooling feature map and max pooling feature map. Then, the average pooling feature map and max pooling feature map are processed by shared multilayer perceptron and added element by element. The added feature map is passed through sigmoid activation layer and then added with the residual of the input feature map of the channel attention branch to obtain the feature map enhanced by channel attention.

[0104] The feature map enhanced by channel attention is used as the input of the spatial attention branch. The input feature map is compressed in the channel dimension by channel compression to obtain the spatial attention feature map. The spatial attention feature map is processed by 7×7 convolution and then Sigmoid activation is used to map the convolutional feature map into spatial attention weights. The spatial attention weights are added to the input feature map by residual addition to obtain the feature map enhanced by spatial attention.

[0105] The feature map enhanced by spatial attention is activated by Sigmoid and then added to the input of the coordinate-channel-space joint attention gating module. The output is the jump feature map gated by the coordinate-channel-space joint attention gating module.

[0106] 2.3 Decoding and Reconstructing Dual Convolution

[0107] In this embodiment, the decoding and reconstruction dual-convolution module includes two cascaded convolutional units. Each convolutional unit includes a cascaded 3×3 convolutional layer, a BN layer, and a ReLU activation layer. It performs local reconstruction and refinement of the spliced ​​features and pushes the channels back to the target channel of that level. The first-level input is B×768×H divided by 8×W divided by 8, and the output is B×256×H divided by 8×W divided by 8. The second-level input is B×384×H divided by 4×W divided by 4, and the output is B×128×H divided by 4×W divided by 4. The third-level input is B×192×H divided by 2×W divided by 2, and the output is B×64×H divided by 2×W divided by 2. The fourth-level input is B×96×H×W, and the output is B×32×H×W. Finally, a 1×1 convolution is used to map the original-scale decoded features to the category output, with an input of B×32×H×W and an output of B×1×H×W. This module is used to generate the final segmentation results for training supervision and inference output.

[0108] In the decoding branch, the decoding branch uses the fourth encoded feature map x4_0 at a scale of 1 / 16 as the starting point for progressive upsampling. After bilinear upsampling, a fourth upsampled feature map at a scale of 1 / 8 is obtained. This upsampled feature map, along with the corresponding scale fusion jump feature output by the separable hollow pyramid fusion module, is input to the fourth coordinate-channel-space joint attention gating module. After gating and channel concatenation, a fusion feature map at that scale is obtained. This fusion feature map is then processed by the fourth decoding and reconstruction dual convolution module to obtain a reconstructed feature map at that scale. The previous-level upsampled feature map obtained after bilinear upsampling of the reconstructed feature map at each scale serves as the guiding feature for that level of decoding. It is input to the next coordinate-channel-space joint attention gating module along with the fusion jump feature at the next scale, progressively restoring the resolution until the original scale is restored. The original-scale reconstructed feature map output by the final decoding and reconstruction dual convolution module is mapped by a 1×1 convolution to output a pixel-level mass segmentation mask with the same resolution as the input breast MRI image, which serves as the overall output of the decoding branch.

[0109] In specific implementation, the overall process of this embodiment starts with the input of a breast MRI image. The model receives input features B×3×H×W. The shallow convolution module first extracts the original scale details of the input and outputs a shallow feature map x0_0 with a feature specification of B×32×H×W. The MobileSAM encoding module with adapter then performs semantic encoding. This module first resamples the input to a fixed encoding size, and the normalized input specification is B×3×1024×1024. The TinyViT backbone completes the stage encoding of the normalized input. The module obtains four-level stage features and backmaps them to the corresponding scale of the input. The module then completes channel alignment through 1×1 convolution, and the output forms four-level encoded outputs x1_0, x2_0, x3_0, and x4_0, with specifications of B×64×H / 2×W / 2, B×128×H / 4×W / 4, B×256×H / 8×W / 8, and B×512×H / 16×W / 16, respectively. At this point, the coding branch forms a five-level feature set from x0_0 to x4_0, providing multi-scale input for subsequent decoding.

[0110] The decoding stage first enters the separable hollow pyramid fusion module, which receives the encoded five-level feature set B×32×H×W, B×64×H / 2×W / 2, B×128×H / 4×W / 4, B×256×H / 8×W / 8, and B×512×H / 16×W / 16. This module performs cross-scale aggregation at the four target scales according to a pyramid strategy, outputting four-level fused skip features SAPF1 to SAPF4, with output feature specifications of B×32×H×W, B×64×H / 2×W / 2, B×128×H / 4×W / 4, and B×256×H / 8×W / 8, respectively. This output serves as the skip input source for the decoding backbone.

[0111] The decoding backbone progresses from the fourth encoded feature map x4_0. The progressive upsampling, same-scale fusion, and decoding backbone input construction module first performs bilinear upsampling on x4_0, with an input specification of B×512×H / 16×W / 16 and an output specification of B×512×H / 8×W / 8. The attention gating module then receives two same-scale inputs. The guiding feature originates from the upsampled output B×512×H / 8×W / 8, and the fused jump feature originates from the pyramid fusion output SAPF4, with a specification of B×256×H / 8×W / 8. This module first performs gating filtering and outputs the gated jump feature B×256×H / 8×W / 8, then performs channel concatenation and outputs the decoding input feature B×768×H / 8×W / 8. The decoding dual-convolution reconstruction module performs local reconstruction and refinement on this input, outputting a reconstructed feature map B×256×H / 8×W / 8.

[0112] The second-stage decoding uses B×256×H / 8×W / 8 as the bilinear upsampling input, and the bilinear upsampling output is B×256×H / 4×W / 4. The attention-gated reception guidance feature B×256×H / 4×W / 4 and the fused skip feature SAPF3 are of size B×128×H / 4×W / 4. The gated output is B×128×H / 4×W / 4, the spliced ​​output is B×384×H / 4×W / 4, and the decoding dual convolutional reconstruction module outputs B×128×H / 4×W / 4.

[0113] The third-level decoder uses the output of the previous layer, B×128×H / 4×W / 4, as the bilinear upsampling input. The bilinear upsampling output is B×128×H / 2×W / 2. The attention-gated receiving guidance feature B×128×H / 2×W / 2 is fused with the skip feature SAPF2, with a specification of B×64×H / 2×W / 2. The gated output is B×64×H / 2×W / 2, the concatenated output is B×192×H / 2×W / 2, and the decoding dual convolutional reconstruction module outputs B×64×H / 2×W / 2.

[0114] The fourth-level decoder uses the previous layer's output B×64×H / 2×W / 2 as the bilinear upsampling input, with a bilinear upsampling output of B×64×H×W. The attention gating module receives the guiding feature B×64×H×W and the fused jump feature SAPF1, with a specification of B×32×H×W. The gating output is B×32×H×W, and the concatenated output is B×96×H×W. The decoding dual-convolutional reconstruction module outputs B×32×H×W.

[0115] The process finally enters the pixel-level prediction output module, which performs a 1×1 convolution mapping on B×32×H×W and finally outputs the segmentation prediction with output feature size B×1×H×W.

[0116] 3. Training of MRI image segmentation model for breast masses

[0117] The breast mass MRI image segmentation model was trained in the following manner:

[0118] Pre-annotated pixel-level breast MRI images of the lump region are used as training samples to form a training sample set, which is then input into the breast lump MRI image segmentation model. A total loss function consisting of segmentation region loss and edge prediction loss is constructed, and the model parameters of the breast lump MRI image segmentation model are optimized and updated with the goal of minimizing the total loss function, thereby training the breast lump MRI image segmentation model.

[0119] In this embodiment, the segmentation loss function used by the model during training is a hybrid loss function, which adopts a combination of weighted BCE and SoftDice: by introducing a larger weight to positive samples to alleviate class imbalance, and then using the Dice term to directly optimize the region overlap, the two are linearly weighted in a fixed ratio to obtain a consistency constraint that takes into account both pixel-level classification and overall shape.

[0120] The region BCE loss function measures the model's performance by comparing the probability values ​​output by the model with the true label values. Its mathematical expression is:

[0121] ;

[0122] In the formula, This represents the total number of pixels in the image. For the first The true label (0 or 1) of each pixel; The model predicts the first The probability (a value between 0 and 1) that a pixel belongs to a tumor region;

[0123] The region Dice loss function is a loss function based on the Dice coefficient. Its purpose is to minimize the Dice coefficient (or maximize its negative value), thereby improving the similarity between the segmentation result and the true label. Its mathematical expression is:

[0124] ;

[0125] In the formula, The model predicts the first The probability that a pixel belongs to a tumor region; For the first The real label of each pixel; It is a very small constant;

[0126] In summary, the loss of the segmented region is:

[0127] ;

[0128] In the formula, The loss function for segmenting regions; The region BCE loss function; The Dice loss function for the region;

[0129] Breast tumors often present with spiculated, invasive margins, and their contours tend to adhere or break against a low-contrast glandular background. To explicitly emphasize edge transition zones in the network, this experiment extracts gradient magnitudes from the predicted probability map using the Sobel operator, normalizes them to 0–1 edge intensity maps, and performs alignment supervision with the ground truth edge maps obtained from the ground truth mask via morphological gradients. In terms of loss mechanism, a combination of BCE and Dice is used to jointly constrain the response intensity and spatial overlap of edge pixels, encouraging the model to provide clear and detailed contour predictions at the gland-mass interface.

[0130] For prediction probability map The Sobel operator is used to extract gradient edges, and the partial derivatives mentioned above are approximated by two 3×3 convolution kernels for edge detection. The horizontal kernel... Emphasizing horizontal brightness variations, the middle column is 0, ignoring vertical effects, with symmetrical sides but opposite signs, and a vertical core. Emphasize the vertical brightness variation, with 0 in the middle, ignore the horizontal influence, and make both sides symmetrical but with opposite signs. , They are respectively:

[0131] ;

[0132] The normalized predicted edge strength is obtained through calculation. It is represented as:

[0133] ;

[0134] The edge BCE loss function is:

[0135] ;

[0136] In the formula, For the first Predicted edge strength of each pixel; For the first The true edge label of each pixel;

[0137] ;

[0138] In summary, the edge prediction loss is:

[0139] ;

[0140] In the formula, The edge prediction loss function; The edge BCE loss function; Edge Dice loss function;

[0141] The total loss function is:

[0142] ;

[0143] In the formula, This is the total loss function.

[0144] II. Implementation Examples

[0145] The network built and trained based on the above steps was used to test its segmentation performance using a self-built dataset. The segmentation results were then compared by incorporating the model into the dataset. Examples of the segmentation results are shown below. Figure 6 As shown, the segmentation results and comparisons are shown in Table 1.

[0146] Table 1. Segmentation Results and Comparison

[0147] method IOU Dice Total number of parameters Number of training parameters Unet 0.7378 0.8492 7.84M 7.84M Unet++ 0.7461 0.8546 9.16M 9.16M AttUnet 0.7499 0.8571 7.98M 7.98M This method 0.7746 0.8730 10.39M 4.64M

[0148] As shown in Table 1, the experimental results on the comparative dataset demonstrate that the proposed method for image segmentation achieves an IoU of 0.7580 and a Dice of 0.8623, outperforming Unet (IoU=0.7378, Dice=0.8492), Unet++ (IoU=0.7461, Dice=0.8546), and AttUnet (IoU=0.7499, Dice=0.8571) in terms of segmentation accuracy. Specifically, compared to the original Unet, this method improves IoU by 2.74% and Dice by 1.54%; compared to Unet++, it improves IoU by 1.59% and Dice by 0.90%; and compared to AttUnet, it improves IoU by 1.08% and Dice by 0.61%. Furthermore, from the perspective of model complexity, the total number of parameters in this method is 10.39M. However, due to the strategy of freezing the pre-trained backbone and training only the key incremental parameters, the number of training parameters is only 4.64M, which is significantly lower than Unet (7.84M), Unet++ (9.16M) and AttUnet (7.98M). While ensuring higher segmentation accuracy, it reduces the scale of trainable parameters and training cost, demonstrating a good accuracy-efficiency trade-off.

[0149] III. Overview

[0150] This invention proposes a lightweight automatic MRI segmentation method for breast masses based on MobileSAM. Its core innovation lies in constructing an end-to-end segmentation model that balances lightweight design with high accuracy. This model uses a lightweight MobileSAM encoding network as its backbone and introduces an adapter mechanism to achieve low-cost fine-tuning in the field of breast imaging. Under the premise of freezing the backbone parameters, only a small number of incremental parameters are optimized, which significantly reduces the number of trainable parameters and computational overhead of the model and solves the problem that general large models are difficult to deploy in resource-constrained scenarios. In the decoding branch, this embodiment of the invention designs a separable, dilated pyramid fusion module. It employs depthwise separable convolution combined with multi-dilation rate dilated convolution to construct a multi-branch pyramid structure. This achieves adaptive aggregation of cross-scale contextual information while maintaining feature resolution, significantly enhancing the feature discrimination capability for small target masses and blurred boundaries. Simultaneously, this embodiment introduces a coordinate-channel-space joint attention gating module. Guided by decoded semantic features, it performs multi-dimensional attention filtering and enhancement on fused skip connection features, effectively suppressing background interference, strengthening mass boundary location modeling, and improving the precision and robustness of the segmentation results. Through these innovative designs, this embodiment achieves end-to-end automatic segmentation without interactive prompts. While maintaining low training costs and fast inference, it significantly improves the accuracy and clinical applicability of mass segmentation in breast MRI images, providing a feasible technical solution that balances performance and efficiency for intelligent breast cancer screening in primary healthcare settings.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A lightweight MRI-based automatic segmentation method for breast masses using MobileSAM, characterized in that, The process involves acquiring a breast MRI image to be detected, inputting the breast MRI image to be detected into a pre-trained breast mass MRI image segmentation model, and outputting pixel-level breast mass segmentation results of the breast MRI image to be detected. The breast mass MRI image segmentation model includes an encoding branch and a decoding branch. The encoding branch includes a shallow convolutional feature extraction module and a MobileSAM encoding module with an adapter, connected in sequence, for multi-scale feature extraction and domain adaptation of the input breast MRI image, outputting multi-level feature maps including shallow details and deep semantics respectively. The decoding branch includes a separable hollow pyramid fusion module and a coordinate-channel-space joint attention gating module, connected in sequence, for cross-scale aggregation, attention filtering and resolution-level recovery of the multi-level feature maps output by the encoding branch, and finally outputting a pixel-level mass segmentation mask with the same resolution as the input breast MRI image as the pixel-level breast mass segmentation result.

2. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 1, characterized in that, The shallow convolutional feature extraction module includes two cascaded convolutional units, each of which includes a cascaded 3×3 convolutional layer, a BN layer, and a ReLU activation layer. The input feature map is used as the input to the first convolutional unit. Features are extracted through a 3×3 convolutional layer, normalized through a BN layer, and then non-linearly activated through a ReLU activation layer to obtain the primary features. The primary features output by the first convolutional unit are used as input to the second convolutional unit for shallow feature extraction. After performing the same operation as the first convolutional unit, the original-scale shallow feature map output by the second convolutional unit is used as the output of the shallow convolutional feature extraction module.

3. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 2, characterized in that, The MobileSAM coding module with adapter includes a shallow coding unit, three TinyViT coding units, and three Reshape convolutional units connected in sequence; the shallow coding unit includes a cascaded segmentation embedding unit and an MBConv module, and the TinyViT coding unit includes a cascaded downsampling fusion unit and a TinyViT attention module with adapter. In the MobileSAM encoding module with adapter, the original-scale shallow feature map obtained by the shallow convolutional feature module is used as the input of the shallow encoding unit for initial encoding, and the output is a first encoded feature map at a scale of 1 / 2. The output of the shallow encoding unit is used as the input of the first TinyViT encoding unit, and the output of the first TinyViT encoding unit is used as the input of the second TinyViT encoding unit and the first Reshape convolutional unit, respectively. After processing by the first Reshape convolutional unit, the output is a second encoded feature map at a scale of 1 / 4. The output of the second TinyViT coding unit is used as the input of the third TinyViT coding unit and the second Reshape convolutional unit, respectively. After processing by the second Reshape convolutional unit, a third coding feature map with a scale of 1 / 8 is output. The output of the third TinyViT coding unit is used as the input of the third Reshape convolutional unit, and after processing by the third Reshape convolutional unit, a fourth coding feature map with a scale of 1 / 16 is output. The first, second, third, and fourth encoded feature maps are used as the output of the encoding branches to form a multi-level feature map that includes shallow details and deep semantics.

4. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 3, characterized in that, The MBConv module includes a cascaded first 1×1 convolutional layer, a first GELU activation layer, a 3×3 depth convolutional layer, a second GELU activation layer, a second 1×1 convolutional layer, a descent path layer, and a third GELU activation layer. The output of the descent path layer is added to the input of the MBConv module by residual addition, and then passed through the third GELU activation layer to obtain the output of the MBConv module. The TinyViT attention module with adapter includes a window attention submodule and an MLP submodule connected in sequence. The window attention submodule includes a cascaded layer normalization layer, a window attention layer, and a descent path layer, and a first adapter connected in series after the descent path layer. The output of the first adapter is added to the input of the window attention submodule by residual addition, and this is used as the output of the window attention submodule. The MLP submodule includes a cascaded local depth convolutional layer, a layer normalization layer, an MLP layer, and a descent path layer, and a second adapter connected in parallel after the descent path layer. The output of the window attention submodule is used as the input of the local depth convolutional layer, and the output of the local depth convolutional layer is used as the input of the layer normalization layer and the second adapter, respectively. The output of the layer normalization layer is processed by the MLP layer and the descent path layer in sequence, and then added to the output of the second adapter by residual addition, to obtain the output of the MLP submodule, which is used as the overall output of the TinyViT attention module. The first adapter and the second adapter each include a lower projection layer, a first GELU activation layer, an upper projection layer, and a second GELU activation layer. The output of the second GELU activation layer is added to the input of the adapter by residual addition, and then used as the output of the first adapter and the second adapter.

5. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 4, characterized in that, The decoding branch includes four-scale separable hollow pyramid fusion modules, as well as four-scale coordinate-channel-space joint attention gating modules and four-scale decoding and reconstruction dual convolution modules that correspond one-to-one with the four-scale separable hollow pyramid fusion modules. The input to the decoding branch includes a shallow feature map, a first encoded feature map, a second encoded feature map, a third encoded feature map, and a fourth encoded feature map; The input to the original-scale separable hollow pyramid fusion module includes five levels of feature maps: shallow feature map, first coded feature map, second coded feature map, third coded feature map, and fourth coded feature map; the input to the 1 / 2-scale separable hollow pyramid fusion module includes four levels of feature maps: first coded feature map, second coded feature map, third coded feature map, and fourth coded feature map; the input to the 1 / 4-scale separable hollow pyramid fusion module includes three levels of feature maps: second coded feature map, third coded feature map, and fourth coded feature map; and the input to the 1 / 8-scale separable hollow pyramid fusion module includes two levels of feature maps: third coded feature map and fourth coded feature map. In the decoding branch, the fourth encoded feature map at a scale of 1 / 16 is used as the starting point for progressive upsampling. After bilinear upsampling, a fourth upsampled feature map at a scale of 1 / 8 is obtained. The fourth upsampled feature map and the fused jump feature output from the separable hollow pyramid fusion module at a scale of 1 / 8 are input into the coordinate-channel-space joint attention gating module at a scale of 1 / 8. The output jump feature map at a scale of 1 / 8 is then input into the decoding and reconstruction dual convolution module at a scale of 1 / 8, and after bilinear upsampling, a third upsampled feature map at a scale of 1 / 4 is obtained. The third upsampled feature map and the fused jump feature output from the 1 / 4 scale separable hollow pyramid fusion module are input together into the 1 / 4 scale coordinate-channel-space joint attention gating module. The output 1 / 4 scale jump feature map is then input into the 1 / 4 scale decoding and reconstruction dual convolution module, and then bilinear upsampling is performed to obtain the second upsampled feature map at the 1 / 2 scale. The second upsampled feature map and the fused jump feature output from the 1 / 2 scale separable hollow pyramid fusion module are input together into the 1 / 2 scale coordinate-channel-space joint attention gating module. The output 1 / 2 scale jump feature map is then input into the 1 / 2 scale decoding and reconstruction dual convolution module, and then bilinear upsampling is performed to obtain the first upsampled feature map at the original scale. The first upsampled feature map and the fused jump feature output from the original-scale separable hollow pyramid fusion module are input together into the original-scale coordinate-channel-space joint attention gating module. The output original-scale jump feature map is then input into the original-scale decoding and reconstruction dual convolution module, and after bilinear upsampling and 1×1 convolution mapping, a pixel-level mass segmentation mask with the same resolution as the input breast MRI image is output as the overall output of the decoding branch.

6. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 5, characterized in that, The processing steps of the separable dilated pyramid fusion module include: aligning the input multi-level feature maps to the target scale and concatenating them along the channel dimension to obtain a fused input feature map at that scale; performing 1×1 convolutional dimensionality reduction on the fused input feature map to obtain a dimensionality-reduced feature map; constructing multiple parallel branches, each branch using depthwise separable dilated convolution to process the dimensionality-reduced feature map, wherein the dilation rate of each branch is set incrementally according to the branch number; performing two-dimensional random dropout processing on the feature maps output by each branch, concatenating the processed branch feature maps along the channel dimension to obtain a multi-branch concatenated feature map; and performing 1×1 convolutional fusion processing on the multi-branch concatenated feature map to obtain a fused jump feature at that scale.

7. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 5, characterized in that, The coordinate-channel-space joint attention gating module includes a coordinate attention branch, a channel attention branch, and a spatial attention branch; wherein, the input of the coordinate-channel-space joint attention gating module includes upsampled feature maps of the corresponding scale and fused jump features, and the upsampled feature maps are used as guiding features to sequentially perform horizontal and vertical coordinate attention enhancement, global channel attention enhancement, and spatial position attention enhancement on the fused jump features; In the coordinate-channel-space joint attention gating module, the upsampled feature map of the corresponding scale and the fused jump feature are concatenated by channels and used as the input of the coordinate attention branch. The feature is compressed in the horizontal and vertical directions by global average pooling layers along the horizontal and vertical directions, respectively, to obtain horizontal pooling feature maps and vertical pooling feature maps. These are then concatenated in the channel dimension and processed by convolutional layers. The processed feature map is split into two independent feature maps, which are activated by Sigmoid and then added to the input feature map by residual addition to obtain the feature map enhanced by coordinate attention. The feature map enhanced by coordinate attention is used as the input of the channel attention branch. Global spatial information is compressed by global average pooling layer and global max pooling layer respectively to obtain average pooling feature map and max pooling feature map. Then, the average pooling feature map and max pooling feature map are processed by shared multilayer perceptron and added element by element. The added feature map is passed through sigmoid activation layer and then added with the residual of the input feature map of the channel attention branch to obtain the feature map enhanced by channel attention. The feature map enhanced by channel attention is used as the input of the spatial attention branch. The input feature map is compressed in the channel dimension by channel compression to obtain the spatial attention feature map. The spatial attention feature map is processed by 7×7 convolution and then Sigmoid activation is used to map the convolutional feature map into spatial attention weights. The spatial attention weights are added to the input feature map by residual addition to obtain the feature map enhanced by spatial attention. The feature map enhanced by spatial attention is activated by Sigmoid and then added to the input of the coordinate-channel-space joint attention gating module. The output is the jump feature map gated by the coordinate-channel-space joint attention gating module.

8. The lightweight automatic MRI segmentation method for breast masses based on MobileSAM according to claim 1, characterized in that, The breast mass MRI image segmentation model was trained in the following manner: Pre-annotated pixel-level breast MRI images of the lump region are used as training samples to form a training sample set, which is then input into the breast lump MRI image segmentation model. A total loss function consisting of segmentation region loss and edge prediction loss is constructed, and the model parameters of the breast lump MRI image segmentation model are optimized and updated with the goal of minimizing the total loss function, thereby training the breast lump MRI image segmentation model.

9. The lightweight MRI automatic segmentation method for breast masses based on MobileSAM according to claim 8, characterized in that, The loss of the segmented region is: ； In the formula, The loss function for segmenting regions; The region BCE loss function; The Dice loss function for the region; The region BCE loss function is: ； In the formula, This represents the total number of pixels in the image. For the first The real label of each pixel; The model predicts the first The probability that a pixel belongs to a tumor region; The Dice loss function for the region is: ； In the formula, The model predicts the first The probability that a pixel belongs to a tumor region; For the first The real label of each pixel; It is a very small constant; The edge prediction loss is: ； In the formula, The edge prediction loss function; The edge BCE loss function; Edge Dice loss function; The edge BCE loss function is: ； In the formula, For the first Predicted edge strength of each pixel; For the first The true edge label of each pixel; 。 10. The lightweight MRI automatic segmentation method for breast masses based on MobileSAM according to claim 9, characterized in that, The total loss function is: ； In the formula, This is the total loss function.

Images