Medical image tumor segmentation model based on transunet framework and construction method and segmentation method thereof

Abstract

The application discloses a medical image tumor segmentation model based on a TransUNet framework and a construction method and a segmentation method thereof, the construction method comprising constructing a segmentation model based on the TransUNet, wherein an encoder of the segmentation model comprises CNN modules and a self-defined VSS module connected in sequence, the output ends of each convolution block of the CNN modules are input into a decoder of the segmentation model through a jump connection layer, the self-defined VSS module comprises a plurality of novel conversion layers stacked in series based on a Mamba module and a focus linear attention module, the self-defined VSS module performs multi-level feature interaction and global modeling on first encoding features output by the CNN module to obtain second encoding features; and the decoder performs feature fusion and up-sampling on the second encoding features and the output of the jump connection layer to obtain a segmentation image for a sample image. The application can significantly improve the segmentation robustness and precision of breast ultrasound images with low contrast, high noise and fuzzy boundaries.

Description

Technical Field

[0001] This invention relates to the fields of computer, artificial intelligence and image processing technology, and in particular to a medical imaging tumor segmentation model based on the TransUNet framework and its construction and segmentation methods. Background Technology

[0002] Early detection of breast cancer can significantly improve the survival rate of patients. Common diagnostic methods for breast cancer include clinical palpation, breast ultrasound, and mammography. Breast ultrasound has advantages such as being radiation-free, painless for the patient, simple to operate, and fast in imaging, and is therefore widely used in the clinical diagnosis of breast cancer.

[0003] However, interpreting ultrasound images requires radiologists with extensive clinical experience, and diagnoses can vary significantly between doctors of different skill levels. This is because the image quality of breast ultrasound examinations is related to the operator's technique and experience, and there are also considerable individual differences among patients. Furthermore, ultrasound images themselves have characteristics such as low contrast and artifacts, which may lead to diagnostic errors when doctors use breast ultrasound images for diagnosis.

[0004] In recent years, computer-aided diagnostic (CAD) systems have been extensively studied. These systems can help doctors generate more reliable diagnostic results. With the continuous development of artificial intelligence and computer technology, automatic segmentation of breast ultrasound images has become a hot topic in modern medical research, and its value in practical applications is constantly being validated.

[0005] Currently, there are two main methods for ultrasound image segmentation: traditional algorithms and deep learning-based methods. Traditional algorithms primarily rely on the image's grayscale characteristics, edge information, texture features, and morphological characteristics. Examples include thresholding, region growing, clustering, and edge detection. These traditional methods typically require precise parameter tuning, and they exhibit limitations when dealing with common interferences in ultrasound images, such as artifacts and noise.

[0006] Deep learning-based methods dominate image segmentation because, unlike traditional methods, they can efficiently and automatically extract features. U-net8 is widely used for medical image segmentation due to its efficiency. However, as a representative of traditional neural networks (CNNs), U-net primarily focuses on local information and lacks the ability to capture global information. In contrast, graph neural networks (GNNs) and Transformers have certain advantages in capturing global information, but their computational complexity is higher.

[0007] At present, the deep learning-based breast image segmentation technology has the following shortcomings: (1) Due to the interference of noise and artifacts during the image acquisition process, the overall image contrast is reduced and the edge of the target area is blurred, which affects the clarity and reliability of the segmentation; (2) Breast tissue itself has high heterogeneity, including glands, fat and fibrous tissue, and the morphology and density of the mass are significantly different from those of the surrounding tissue, making it difficult for traditional segmentation algorithms to accurately distinguish between normal tissue and lesion areas; (3) Breast lesions are diverse in shape, size, boundary and location. The shape characteristics of benign and malignant masses are very different, and the boundaries are often irregular or blurred, which further increases the complexity of the segmentation task and the uncertainty of the results.

[0008] The disclosure of the above background technical content is only for the purpose of assisting in understanding the concept and technical solution of this application, and does not necessarily provide technical instruction. Summary of the Invention

[0009] The purpose of this invention is to provide a medical imaging tumor segmentation model based on the TransUNet framework, as well as its construction and segmentation methods, which can significantly improve the robustness and accuracy of segmentation of breast ultrasound images with low contrast, high noise, and blurred boundaries.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A method for constructing a medical image tumor segmentation model based on the TransUNet framework includes the following steps:

[0012] A segmentation model is constructed based on TransUNet. The segmentation model includes an encoder, a decoder, and a skip connection layer, which is used for skip connections between the encoder and the decoder.

[0013] The encoder is constructed by constructing a CNN module and a custom VSS module connected in sequence, wherein the CNN module is configured to perform convolution operations and downsampling on the input sample image to obtain a first encoded feature, and the output of each convolution block of the CNN module is input to the decoder through the skip connection layer;

[0014] The custom VSS module is constructed as follows: the Mamba module is used to replace the Transformer layer in TransUNet and a focused linear attention module is introduced to construct a novel transformation layer. The Mamba module is used to extract and enhance features in the multidimensional latent space of its input. The focused linear attention module is used to segment the input image and perform attention calculation in each window. By stacking and concatenating multiple novel transformation layers, the first encoded features are subjected to multi-level feature interaction and global modeling to obtain the second encoded features.

[0015] The decoder is constructed by introducing an extended residual block and an upsampling module to perform feature fusion and upsampling on the second encoded features and the output of the skip connection layer to obtain a segmented image for the sample image.

[0016] Furthermore, following any one or a combination of the aforementioned technical solutions, the output of the novel conversion layer in the custom VSS module is configured to be input to the next novel conversion layer, and the output of the last novel conversion layer is connected to the input of the decoder.

[0017] For each of the novel conversion layers, its input is processed by layer normalization and then input into the Mamba module. The Mamba module dynamically focuses on the tumor region to adaptively process tumors of different sizes and shapes.

[0018] The input of the novel conversion layer and the output of the Mamba module are fused to obtain the first fused feature;

[0019] The focused linear attention module performs image segmentation and attention calculation on the first fused feature;

[0020] The output of the focused linear attention module is processed by layer normalization and then fused with the input of the novel conversion layer and the first fusion feature to obtain a second fusion feature. The second fusion feature is configured as the output of the novel conversion layer.

[0021] Furthermore, following any one or a combination of the aforementioned technical solutions, the number of layers in the novel conversion layer is one of the training parameters of the segmentation model, and the number of layers is determined through model training; and / or,

[0022] No. The output of the novel conversion layer described in the layer for:

[0023] ;

[0024] in, LN ( ) indicates layer normalization operation. FLA( ) represents the output of the linear attention module, and MB() represents the output of the Mamba module. Indicates the first The output of the novel conversion layer described above, Indicates feature fusion; and / or,

[0025] The input of the novel conversion layer described in the first layer Represented as:

[0026] ;

[0027] in, This indicates that the input image is transformed into 1 to 1 through reshaping. n A flattened two-dimensional image block, This indicates that the size of each image patch is... , The number of image patches, For block embedding, the linear projection matrix, Indicates the number of channels. Let represent the embedding matrix, and D represent the spatial dimension.

[0028] Furthermore, based on any or a combination of the aforementioned technical solutions, the CNN module includes a convolutional module, the skip connection layer includes a HAM module, and the decoder further includes a cascaded feature fusion module and a dilated residual block (DRB) module.

[0029] The output of the convolution module is connected to the output of the HAM module;

[0030] The output of the HAM module is connected to the input of the feature fusion module, and the input of the feature fusion module is also connected to the output of the decoder.

[0031] Furthermore, based on any or a combination of the aforementioned technical solutions, the CNN module includes multiple convolutional modules, the skip connection layer further includes multiple HAM modules, and the decoder further includes multiple sets of cascaded feature fusion modules and DRB modules;

[0032] One of the convolutional modules corresponds to a set of feature fusion modules and DRB modules, and the output of the convolutional module and the input of the feature fusion module are connected in a skip connection through the HAM module.

[0033] Furthermore, following any one or a combination of the aforementioned technical solutions, the number of the skip connection layer is one of the training parameters of the segmentation model, and the number of the skip connection layer is determined through model training.

[0034] Furthermore, following any or a combination of the aforementioned technical solutions, the HAM module includes a position attention module, a channel attention module, and a compressed excitation attention module. The input of the HAM module is input to the position attention module and the channel attention module in parallel, and the output of the position attention module and the channel attention module is input to the compressed excitation attention module after feature fusion.

[0035] The location attention module captures spatial dependencies by evaluating the similarity between any two location features in its input feature map.

[0036] The channel attention module performs weighted fusion of the output features of each channel and then fuses them with the input feature map of the channel attention module to extract relevant features of multiple channels;

[0037] The compressed excitation attention module performs global average pooling on its input feature map and then further updates the weights of each channel through multiple fully connected layers to adaptively emphasize important features and enhance the ability to accurately distinguish different image regions.

[0038] Furthermore, following any one or a combination of the aforementioned technical solutions, the positional attention module generates a first feature map, a second feature map, and a third feature map based on its input feature map through a convolutional layer. The second and third feature maps are then matrix multiplied and processed through a softmax layer to obtain a spatial attention map. The mutual influence between channels is calculated to obtain the attention weights for each channel. The first feature map and the spatial attention map are fused based on these channel attention weights. The fused result is then weighted by learning parameters and added to the input feature map of the positional attention module to obtain the positional attention feature map; and / or,

[0039] The output of each channel in the channel attention module is obtained by weighted fusion of the features of all channels, and then combined with the input feature map of the channel attention module; and / or,

[0040] The compressed stimulus attention module includes global average pooling, two cascaded gated fully connected layers, and a recalibration module. The learned channel weights are applied to the input feature map of the compressed stimulus attention module. Global average pooling compresses spatial information into channel descriptors, and the two cascaded gated fully connected layers learn the dependencies between channels to generate channel weights. The recalibration module applies the learned channel weights to the input feature map of the compressed stimulus attention module to obtain its output features.

[0041] Furthermore, following any one or a combination of the aforementioned technical solutions, the Mamba module includes a deep convolutional block, an activation function layer, a two-dimensional selective state space layer, and a layer normalization module connected in sequence.

[0042] Furthermore, following any one or a combination of the aforementioned technical solutions, the method further includes training the segmentation model using the following loss function:

[0043] ;

[0044] in, Indicates the predicted segmentation region. Represents the actual target segmentation region. This represents the binary cross-entropy loss used to represent the difference between the predicted segmented region and the actual target segmented region. This represents the loss that maximizes the overlap between the predicted segmented region and the actual target segmented region.

[0045] According to another aspect of the present invention, a medical image tumor segmentation model based on the TransUNet framework is provided, which is constructed using the TransUNet-based medical image tumor segmentation model construction method described above, or a combination of multiple technical solutions; the medical image tumor segmentation model is configured to segment tumor regions in medical images.

[0046] Acquire medical images to be segmented and input them into the medical image tumor segmentation model;

[0047] The medical imaging tumor segmentation model predicts the tumor region to be segmented and outputs a tumor segmentation image.

[0048] According to another aspect of the present invention, a medical image tumor segmentation method based on the TransUNet framework is provided, comprising the following steps: constructing a medical image tumor segmentation model based on the medical image tumor segmentation model construction method based on the TransUNet framework as described in any of the above embodiments;

[0049] Acquire the medical image to be segmented, and input the medical image into the medical image tumor segmentation model to output a tumor segmentation image.

[0050] The beneficial effects of the technical solution provided by this invention are as follows:

[0051] a. Due to the high noise and low contrast in breast ultrasound images, image segmentation faces great challenges. This invention introduces a custom VSS module on the TransUNet basic model. This module integrates a focused linear attention (FLA) mechanism on the basis of the Mamba block. By segmenting the input image and performing attention calculation in each window, this mechanism can more effectively simulate global information, thereby more accurately identifying details in ultrasound images and significantly improving the segmentation effect.

[0052] b. In ultrasound images, the blurry boundaries and artifacts of tumors pose challenges to accurate segmentation. This invention introduces a Dilated Residual Block (DRB) in the decoder to replace traditional convolution to address the problem of blurred boundaries of the segmentation target. The DRB module uses dilated convolution technology to extract richer feature information and significantly enhance feature expression capabilities, thereby improving the robustness and accuracy of segmentation of breast ultrasound images with low contrast, high noise, and blurred boundaries.

[0053] c. This invention introduces a hybrid attention module in the skip connection layer, which integrates spatial and channel attention mechanisms as well as other attention mechanisms. By embedding the HAM module into the skip connections of the encoder and decoder, it effectively filters redundant features, enhances the feature transfer quality between the encoder and decoder, and can specifically address the problems of low contrast and loss of detail in medical images, thereby improving segmentation accuracy. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A schematic diagram of a module of a medical imaging tumor segmentation model based on the TransUNet framework, provided as an exemplary embodiment of the present invention;

[0056] Figure 2 A schematic diagram of a HAM module provided as an exemplary embodiment of the present invention;

[0057] Figure 3 A schematic diagram of a position attention module provided as an exemplary embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the channel attention module provided as an exemplary embodiment of the present invention;

[0059] Figure 5 A schematic diagram of a compression excitation module provided as an exemplary embodiment of the present invention;

[0060] Figure 6 This is a schematic diagram of a DRB module provided as an exemplary embodiment of the present invention. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0063] In one embodiment of the present invention, a method for constructing a medical image tumor segmentation model based on the TransUNet framework is provided, see [link to relevant documentation]. Figure 1 This includes the following steps:

[0064] A segmentation model is constructed based on TransUNet. The segmentation model includes an encoder, a decoder, and a skip connection layer, which is used for skip connections between the encoder and the decoder.

[0065] The encoder is constructed by constructing a CNN module and a custom VSS module connected in sequence, wherein the CNN module is configured to perform convolution operations and downsampling on the input sample image to obtain a first encoded feature, and the output of each convolution block of the CNN module is input to the decoder through the skip connection layer;

[0066] The custom VSS module is constructed as follows: the Mamba module is used to replace the Transformer layer in TransUNet and a focused linear attention module is introduced to construct a novel transformation layer. The Mamba module is used to extract and enhance features in the multidimensional latent space of its input. The focused linear attention module is used to segment the input image and perform attention calculation in each window. By stacking and concatenating multiple novel transformation layers, the first encoded features are subjected to multi-level feature interaction and global modeling to obtain the second encoded features.

[0067] The decoder is constructed by introducing an extended residual block and an upsampling module to perform feature fusion and upsampling on the second encoded features and the output of the skip connection layer to obtain a segmented image for the sample image.

[0068] See Figure 1 This invention optimizes the U-shaped architecture by adding a custom VSS module specifically designed for breast image segmentation. It introduces the concept of state-space modeling to efficiently capture the long-range dependencies of complex tissue structures in images, addressing the problem of insufficient global information in medical images. Furthermore, by introducing a Hybrid Attention Module (HAM), spatial and channel attention mechanisms, as well as a Squeeze-and-Excitation (SE) mechanism, are integrated. By embedding the HAM module into skip connections, redundant features are effectively filtered, enhancing the feature transfer quality between the encoder and decoder. This specifically addresses low contrast and detail loss in medical images, thereby improving segmentation accuracy. The segmentation model employs a hybrid encoder architecture and skip connections, combining the advantages of the aforementioned modules to progressively extract and compress key features. Simultaneously, this application introduces a custom residual structure in the decoder of the segmentation model network, replacing the original simple convolution operation. This aims to better preserve the complex features of breast tissue, further improving the accuracy of tumor region segmentation.

[0069] This application employs a dual-encoder architecture comprising a CNN module and a custom VSS module. In this architecture, the first encoder's CNN module is built based on ResNet V2, consisting of multiple convolutional blocks and their variants, along with embedding layers. These modules integrate convolutional operations with downsampling. Each convolutional block halves the size of the input feature map while doubling its feature capacity, thereby optimizing feature representation and processing efficiency. The convolutional blocks are used for initial feature extraction and downsampling. The embedding layers divide the image into blocks and map them to sequence vectors with positional encoding.

[0070] Accordingly, the number of skip connection layers is multi-layered, and each skip connection layer includes a HAM module. The decoder includes multiple sets of cascaded feature fusion modules and DRB modules. One convolutional module corresponds to one set of feature fusion modules and DRB modules, and the output of the convolutional module and the input of the feature fusion module are skip-connected through the HAM module. The number of skip connection layers is one of the training parameters of the segmentation model, and the number of skip connection layers is determined through model training.

[0071] The output of the CNN module is configured as the input to the second encoder, i.e., the custom VSS module. This application implements a trainable linear projection layer for image patching. x p The image is transformed into a D-dimensional latent space. Before passing through the projection layer, the input image is positionally encoded, and the image patch embeddings are enhanced to incorporate spatial information. Specifically, the enhanced embedding sequence, which is the output of the CNN module, is the input of the first novel transformation layer. Represented as:

[0072] ;

[0073] Among them, the input image By reshaping, it is transformed into a series of flattened two-dimensional image patches, where, This indicates that the size of each image patch is... , The number of image patches, , Let be a linear projection matrix for block embedding, which is used to map image blocks to the embedding space. Indicates the number of channels. It is represented as an embedding matrix and is used to preserve spatial relationships, where D represents the spatial dimension.

[0074] By dividing the input image into local blocks, each block is transformed into a vector representation that preserves spatial relationships, thus providing a visual conceptual representation with semantic and positional information for subsequent processing. The obtained block embeddings contain both identity and layout information, which can be used to guide the subsequent decoding process.

[0075] See Figure 1 A custom VSS module is used as the second part of the encoder to provide the necessary dimensional adjustments for subsequent processing stages.

[0076] The custom VSS module contains multiple novel transformation layers. The output of the last novel transformation layer is connected to the input of the decoder, and the outputs of intermediate novel transformation layers are configured to be input to the next novel transformation layer. The number of layers in series is one of the training parameters of the segmentation model, and the number of layers is determined through model training. In this application, through training, for a segmentation model suitable for breast tumors, the preferred number of layers is determined to be 12 layers.

[0077] For each of the novel transformation layers, its input, after layer normalization, is fed into the Mamba module. The Mamba module dynamically focuses on the tumor region to adaptively handle tumors of different sizes and shapes. The Mamba module comprises a depthwise convolutional block (DwConv), an activation function layer (SiLu), a two-dimensional selective state space layer (2D-ss), and a layer normalization module, connected in sequence. The input of the novel transformation layer and the output of the Mamba module are fused to obtain a first fused feature.

[0078] The Focused Linear Attention (FLA) module performs image segmentation and attention calculation on the first fused feature; the output of the Focused Linear Attention module is processed by layer normalization and then fused with the input of the novel transformation layer and the first fused feature to obtain a second fused feature, which is configured as the output of the novel transformation layer.

[0079] See Figure 1 The novel transformation layer is constructed by combining the focused linear attention module with the Mamba module (a two-dimensional spatial attention module) and adopting a multi-layer stacked structure. The segmentation model can achieve complex visual reasoning through multiple rounds of spatial attention and focused linear attention.

[0080] Among them, the The output of the novel conversion layer described in the layer for:

[0081] ;

[0082] in, LN ( ) indicates layer normalization operation. FLA ( ) represents the output of the linear attention module, and MB() represents the output of the Mamba module. Indicates the first The output of the novel conversion layer described above, This indicates feature fusion.

[0083] This application replaces the original Transformer layer in TransUNet with the Mamba module and combines it with a focused linear attention mechanism to achieve global information modeling with linear computational complexity. FLA enhances the focus on important features by adjusting the direction of query (Q) and key (K) features, avoiding the feature convergence problem in traditional linear attention, thereby improving the ability to capture the global context of the image and the relationships between pixels.

[0084] The skip connection layer based on the HAM module proposed in this application aims to enhance the model's ability to selectively extract features from the input image by simultaneously focusing on spatial and channel dimensions, thereby effectively filtering out redundant information and improving segmentation accuracy. As mentioned above, the skip connection layer used in this application is multi-layered. The HAM module of each skip connection layer includes a positional attention module, a channel attention module, and a compressed activation attention module. The input of the HAM module is input to the positional attention module and the channel attention module in parallel, and the outputs of the positional attention module and the channel attention module are input to the compressed activation attention module after feature fusion. Its workflow is as follows: Figure 2 As shown, the input feature map, i.e. the output feature of the convolutional block, first passes through a 3×3 convolutional layer, and then is input in parallel to the position attention module and the channel attention module; the resulting features are fused and then processed through a 3×3 convolutional layer and the compression activation attention module; finally, the refined features are output through a 1×1 convolutional layer.

[0085] See Figure 2 The location attention module captures spatial dependencies by evaluating the similarity between any two location features in its input feature map. The features at each location are updated through a weighted sum of all location features, with the weights depending on the feature similarity of the corresponding locations. Therefore, locations with similar features can reinforce each other, regardless of their actual distance on the feature map. The location attention module generates a first feature map, a second feature map, and a third feature map based on its input feature map using convolutional layers. It performs matrix multiplication on the second and third feature maps, then processes them through a softmax layer to obtain a spatial attention map. It then calculates the mutual influence between channels to obtain the attention weights for each channel. Based on these channel attention weights, it fuses the first feature map and the spatial attention map, and finally, after weighting with learned parameters, adds the fused result to the input feature map of the location attention module to obtain the location attention feature map.

[0086] Specifically, see Figure 3 The input feature map for the position attention module is Where C, H, and W represent the number of channels, height, and width, respectively. The input feature map A is first processed by a convolutional layer to generate three feature maps B, C, and D, all with the same dimensions. Subsequently, feature map B and feature map C are reshaped into... ,in, The total number of pixels is given. A matrix multiplication is performed between feature map B (the transpose of the second feature map) and feature map C (the third feature map), followed by processing through a softmax layer to obtain the spatial attention map. Specifically, by calculating the degree of mutual influence between channels, the obtained attention weights are fused with feature map D, then weighted by learnable parameters, and finally added to the input feature map A to generate the output feature. .

[0087] The Channel Attention (CAM) module performs weighted fusion of the output features of each channel and then fuses them with the input feature map of the Channel Attention module to extract relevant features from multiple channels. The output of each channel in the Channel Attention module is obtained by weighted fusion of the features of all channels and then combining it with its original input feature map, i.e., the input feature map of the Channel Attention module. See details. Figure 4 In the channel attention module, the final output of each channel is obtained by weighted fusion of features from all channels, combined with its original input feature map. This mechanism enables the CAM module to effectively extract channel-related features. By introducing channel attention maps for feature ensemble, rich channel context can be maintained while efficiently processing channel information, thereby significantly improving the quality of the final feature representation.

[0088] like Figure 5 As shown, the compression excitation module enhances the model's ability to process channel features by recalibrating the importance of each channel. The compression excitation attention module performs global average pooling on its input feature map and then further updates the weights of each channel through multiple fully connected layers to adaptively emphasize important features and enhance the ability to accurately distinguish different image regions. The compression excitation attention module includes global average pooling, two cascaded gated fully connected layers, and a recalibration module. It applies the learned channel weights to the input feature map of the compression excitation module. The global average pooling is used to compress spatial information into channel descriptors, and the two cascaded gated fully connected layers are used to learn the dependencies between channels and generate channel weights. The recalibration module applies the learned channel weights to the input feature map of the compression excitation attention module to obtain its output features.

[0089] The compression activation module performs global average pooling on its input feature map, compressing the spatial information of each channel into a scalar representation. Then, through a gating mechanism containing two fully connected layers, it learns and generates weights for each channel. Finally, these weights are used to recalibrate the input feature map of the compression activation module at the channel level, highlighting information-rich channels and suppressing redundant channels. At the core of the compression activation module is a learnable feature selection mechanism that enables the network to adaptively emphasize important features, improving the discriminative power of feature representations and thus enhancing the network's ability to accurately distinguish different image regions. As part of the hybrid attention module, the SE module complements the spatial attention and channel attention mechanisms through refined modeling of channel relationships, jointly optimizing the overall quality of features.

[0090] Furthermore, due to the inherent irregularities of breast ultrasound images, detailed information is easily lost. This invention proposes a hybrid attention mechanism to effectively address this issue. To this end, an innovative hybrid attention module is introduced through a skip connection layer. This module integrates spatial attention (PAM) and channel attention (CAM) mechanisms, focusing on analyzing the spatial and channel relationships of the input feature map. Moreover, through an ensemble (Squeeze-and-Excitation, SE) mechanism, the HAM module enhances its ability to capture details in the channel dimension, significantly improving the model's accuracy in image segmentation tasks.

[0091] This invention employs skip connections to bridge the semantic gap between the encoder and decoder. Addressing the semantic information misalignment and feature redundancy issues inherent in traditional skip connections during feature propagation, this method innovatively introduces, for example... Figure 2 The hybrid attention module shown is embedded in the skip connection layer.

[0092] This design integrates a hybrid attention module, enabling the encoder and decoder to transfer more information-rich features, thus constructing a complete feature enhancement system within the neural network. Specifically, the SE module recalibrates channel functions, finely adjusting the weights of each channel to improve its feature utility; the PAM module enhances the network's perception of spatial relationships within feature maps, strengthening its understanding of contextual information; and the CAM module optimizes channel activation states, highlighting the most information-rich channel features. The synergistic effect of SE, PAM, and CAM achieves a balanced optimization of channel and spatial features, jointly improving the model's accuracy and efficiency in image segmentation tasks.

[0093] See Figure 1 and Figure 6This invention replaces traditional convolutional operations with DRB in the decoder to fuse features from skip connections and perform upsampling. The DRB module, through a combination of multiple convolutions, enhances feature representation capabilities while maintaining a lightweight model. The DRB module integrates dilated convolutional layers to expand the receptive field, 3×3 convolutional layers to extract local features, and depthwise separable convolutions to reduce parameter count and computational cost. Combined with residual connections, it further enhances feature representation capabilities and alleviates the gradient vanishing problem.

[0094] This invention employs a hybrid loss function combining binary cross-entropy loss and Dice loss to comprehensively optimize segmentation accuracy. This design fully leverages the advantages of both loss functions: binary cross-entropy loss effectively corrects the error between predicted values ​​and true labels from a probabilistic perspective; while Dice loss focuses on maximizing the overlap between predicted and true target segmentation regions, improving overall consistency. This hybrid loss function not only focuses on pixel-level classification accuracy but also ensures the overall coherence of segmented regions and effectively alleviates class imbalance problems (e.g., when the region to be segmented occupies only a small portion of the image), enhancing the model's sensitivity to small regions while maintaining overall segmentation accuracy.

[0095] The expression for the hybrid loss function is as follows:

[0096] ;

[0097] in, Indicates the predicted segmentation region. Represents the actual target segmentation region. This refers to the binary cross-entropy loss used to represent the difference between the predicted segmented region and the true target segmented region. It is particularly good at handling class imbalance (such as when the tumor region in an image is very small), and can improve the model's focus on the target region. This represents the loss that maximizes the overlap between the predicted segmented region and the actual target segmented region.

[0098] This invention, through an innovative combination of the Mamba architecture and hybrid attention mechanisms, effectively addresses the key challenges of breast ultrasound image segmentation, demonstrating significant advantages in improving segmentation accuracy and robustness. Its core application potential lies in becoming a key tool for enhancing the accuracy, standardization, and efficiency of breast cancer diagnosis. It can be directly used for clinical auxiliary diagnosis and provide technical support for multiple aspects such as medical research and health management, possessing high translational value and broad social significance. Future development should focus on validation, compliance, and productization for clinical application.

[0099] In summary, this invention presents a segmentation network with three core improvements based on the TransUNet framework. Encoder enhancement: A novel Mamba layer integrating FLA is introduced to improve global dependency modeling capabilities while maintaining linear computational complexity. Attention-enhanced skip connections: Spatial, channel, and SE attention mechanisms are fused through the HAM module to enhance attention to irregular boundaries and subtle structures of breast tumors. Decoder optimization: The DRB module replaces traditional convolutions, better preserving detailed information and expanding the receptive field during feature fusion and upsampling. The segmentation model constructed using this invention was validated on two publicly available breast ultrasound datasets (Dataset B and BUSI). Experiments show that it outperforms current mainstream segmentation models (such as U-Net, U-Net++, and Swin-Unet) in multiple metrics, including Dice coefficient, IoU, recall, precision, and accuracy. Ablation experiments further confirm the effectiveness of each module.

[0100] In one embodiment of the present invention, a medical image tumor segmentation model based on the TransUNet framework is provided, which is constructed using the medical image tumor segmentation model construction method based on the TransUNet framework as described in any of the above embodiments; the medical image tumor segmentation model is configured to segment tumor regions in medical images: acquiring medical images to be segmented and inputting them into the medical image tumor segmentation model as described above; the medical image tumor segmentation model predicts the tumor regions to be segmented and outputs tumor segmentation images.

[0101] In one embodiment of the present invention, a medical image tumor segmentation method based on the TransUNet framework is provided, comprising the following steps: constructing a medical image tumor segmentation model based on the medical image tumor segmentation model construction method based on the TransUNet framework as described in any of the above embodiments, acquiring the medical image to be segmented, and inputting the medical image into the medical image tumor segmentation model to output a tumor segmentation image.

[0102] It should be noted that the embodiments of the medical image tumor segmentation model and the medical image tumor segmentation method based on the TransUNet framework described above are based on the same inventive concept as the embodiments of the medical image tumor segmentation model construction method based on the TransUNet framework. All contents of the embodiments of the medical image tumor segmentation model construction method based on the TransUNet framework are incorporated into the embodiments of the medical image tumor segmentation model and the medical image tumor segmentation method based on the TransUNet framework by reference.

[0103] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0104] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for constructing a medical image tumor segmentation model based on the TransUNet framework, characterized in that, Includes the following steps: A segmentation model is constructed based on TransUNet. The segmentation model includes an encoder, a decoder, and a skip connection layer, which is used for skip connections between the encoder and the decoder. The encoder is constructed by constructing a CNN module and a custom VSS module connected in sequence, wherein the CNN module is configured to perform convolution operations and downsampling on the input sample image to obtain a first encoded feature, and the output of each convolution block of the CNN module is input to the decoder through the skip connection layer; The custom VSS module is constructed as follows: A Mamba module replaces the Transformer layer in TransUNet, and a focused linear attention module is introduced to construct a novel transformation layer. The Mamba module is used for feature extraction and enhancement of its input in a multidimensional latent space. The focused linear attention module is used for segmenting the input image and performing attention calculations in each window. Multiple novel transformation layers are stacked and concatenated to perform multi-level feature interaction and global modeling on the first encoded feature to obtain a second encoded feature. The output of the novel transformation layer in the custom VSS module is configured as the input to the next novel transformation layer, and the output of the last novel transformation layer is connected to the input of the decoder. For each novel transformation layer, its input is normalized and then input to the Mamba module. The Mamba module dynamically focuses on the tumor region to adaptively handle tumors of different sizes and shapes. The input of the novel transformation layer and the output of the Mamba module are fused to obtain a first fused feature. The focused linear attention module performs image segmentation and attention calculations on the first fused feature. The output of the focused linear attention module is normalized and then fused with the input of the novel transformation layer and the first fused feature to obtain a second fused feature, which is configured as the output of the novel transformation layer. The decoder is constructed by introducing an extended residual block and an upsampling module to perform feature fusion and upsampling on the second encoded features and the output of the skip connection layer to obtain a segmented image for the sample image.

2. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 1, characterized in that, The number of concatenated layers in the novel conversion layer is one of the training parameters of the segmentation model, and the number of layers is determined through model training; and / or, No. The output of the novel conversion layer described in the layer for: ； in, LN ( ) indicates layer normalization operation. FLA ( ) represents the output of the linear attention module, and MB() represents the output of the Mamba module. Indicates the first The output of the novel conversion layer described above, Indicates feature fusion; and / or, The input of the novel conversion layer described in the first layer Represented as: ； in, This indicates that the input image is transformed into 1 to 1 through reshaping. n A flattened two-dimensional image block, This indicates that the size of each image patch is... , The number of image patches, For block embedding, the linear projection matrix, Indicates the number of channels. Let represent the embedding matrix, and D represent the spatial dimension.

3. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 1, characterized in that, The CNN module includes a convolutional module, the skip connection layer includes a HAM module, and the decoder further includes a cascaded feature fusion module and a DRB module; The output of the convolution module is connected to the output of the HAM module; The output of the HAM module is connected to the input of the feature fusion module, and the input of the feature fusion module is also connected to the output of the decoder.

4. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 3, characterized in that, The CNN module includes multiple convolutional modules, the skip connection layer includes multiple HAM modules, and the decoder includes multiple sets of cascaded feature fusion modules and DRB modules; One of the convolutional modules corresponds to a set of feature fusion modules and DRB modules, and the output of the convolutional module and the input of the feature fusion module are connected in a skip connection through the HAM module.

5. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 4, characterized in that, The number of skip connection layers is one of the training parameters of the segmentation model, and the number of skip connection layers is determined through model training.

6. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 3, characterized in that, The HAM module includes a position attention module, a channel attention module, and a compressed stimulus attention module. The input of the HAM module is input to the position attention module and the channel attention module in parallel. The output of the position attention module and the channel attention module is input to the compressed stimulus attention module after feature fusion. The location attention module captures spatial dependencies by evaluating the similarity between any two location features in its input feature map. The channel attention module performs weighted fusion of the output features of each channel and then fuses them with the input feature map of the channel attention module to extract relevant features of multiple channels; The compressed excitation attention module performs global average pooling on its input feature map and then updates the weights of each channel through multiple fully connected layers to adaptively emphasize important features and enhance the ability to distinguish different image regions.

7. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 6, characterized in that, The positional attention module generates a first feature map, a second feature map, and a third feature map based on its input feature map through a convolutional layer. The second and third feature maps are then multiplied by a matrix and processed by a softmax layer to obtain a spatial attention map. The attention weights of each channel are obtained by calculating the degree of mutual influence between each channel. The first feature map and the spatial attention map are fused based on the attention weights of each channel. The fused calculation result is then weighted by learning parameters and added to the input feature map of the positional attention module to obtain the positional attention feature map. And / or, The output of each channel in the channel attention module is obtained by weighted fusion of the features of all channels, and then combined with the input feature map of the channel attention module; and / or, The compressed activation attention module includes global average pooling, two cascaded gated fully connected layers, and a recalibration module. The learned channel weights are applied to the input feature map of the compressed activation attention module. The global average pooling is used to compress spatial information into channel descriptors, and the two cascaded gated fully connected layers are used to learn the dependencies between channels and generate channel weights. The recalibration module applies the learned channel weights to the input feature map of the compressed excitation attention module to obtain its output features.

8. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 1, characterized in that, The Mamba module includes a deep convolutional block, an activation function layer, a two-dimensional selective state space layer, and a layer normalization module connected in sequence.

9. The method for constructing a medical image tumor segmentation model based on the TransUNet framework according to claim 1, characterized in that, The segmentation model is also trained using the following loss function: ； in, Indicates the predicted segmentation region. Represents the actual target segmentation region. This represents the binary cross-entropy loss used to represent the difference between the predicted segmented region and the actual target segmented region. This represents the loss that maximizes the overlap between the predicted segmented region and the actual target segmented region.

10. A medical image tumor segmentation model based on the TransUNet framework, characterized in that, The medical image tumor segmentation model is constructed using the TransUNet framework-based model construction method as described in any one of claims 1 to 9; the medical image tumor segmentation model is configured to segment tumor regions in medical images. Acquire medical images to be segmented and input them into the medical image tumor segmentation model; The medical imaging tumor segmentation model predicts the tumor region to be segmented and outputs a tumor segmentation image.

11. A medical image tumor segmentation method based on the TransUNet framework, characterized in that, Includes the following steps: A medical image tumor segmentation model is constructed based on the TransUNet framework construction method as described in any one of claims 1 to 9. Acquire the medical image to be segmented, and input the medical image into the medical image tumor segmentation model to output a tumor segmentation image.

Images