A breast cancer ultrasound image automatic segmentation method, system and storage medium

Abstract

This invention discloses an automatic segmentation method, system, and storage medium for breast cancer ultrasound images. The method includes: acquiring a breast ultrasound image dataset, performing standardized preprocessing, multi-expert annotation integration, and data augmentation to obtain high-quality data; constructing a basic Attention U-Net model and two improved models, CSWin-Unet and MRCST-Net, respectively, to achieve gradient optimization of segmentation performance through attention gating mechanism, cross-shaped window attention, and residual convolution-Transformer parallel module; using a unified training framework and four indicators—mean intersection-over-union (MIoU), accuracy (Acc), Kappa coefficient, and Dice coefficient—and selecting the optimal model by averaging the results after validation; and performing end-to-end tumor region segmentation on the input breast ultrasound image based on the optimal model. This invention can help improve the efficiency and accuracy of early breast cancer diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and computer-aided diagnosis technology, specifically to an automatic segmentation method, system, and storage medium for breast cancer ultrasound images. Background Technology

[0002] Breast cancer is one of the leading malignant tumors threatening women's health, and early and accurate diagnosis is crucial for developing treatment plans and improving patient prognosis. Ultrasound imaging, with its advantages of being non-invasive, real-time, low-cost, and free of ionizing radiation, has become a core tool for breast cancer screening and diagnosis. However, ultrasound images have drawbacks such as speckle noise and tissue artifacts, and malignant tumors are often irregular in shape, have blurred boundaries, and low contrast with surrounding tissues, posing significant challenges to diagnosis and making the results highly dependent on physician experience and subjective judgment.

[0003] Deep learning-based medical image segmentation methods can assist physicians in improving diagnostic results. U-Net and its variants have become the mainstream methods for medical image segmentation due to their excellent feature extraction and segmentation performance. However, existing methods still suffer from problems such as low localization accuracy, poor adaptability to small / irregular lesions, and insufficient model robustness and generalization, making it difficult to meet the needs of precision medicine.

[0004] Existing technology CN116433586A discloses a breast ultrasound tomography segmentation method, proposing a multi-scale attention strategy that combines a convolutional pooling pyramid based on void space with a feature enhancement model; and employing a hybrid training loss function that integrates contrast and uncertainty to improve image recognition and mid-level feature extraction capabilities. However, existing methods based on Transformer architectures such as CrossFormer are mostly based on fixed loss functions and personalized designs, making it difficult to adapt to the clinical needs of different ultrasound equipment and different lesion characteristics. Their generalization performance and inference speed still need to be improved. Summary of the Invention

[0005] The research objective of this invention is to propose an automatic segmentation method, system, and storage medium for breast cancer ultrasound images, aiming to solve the technical problems of current breast cancer ultrasound image segmentation methods, such as inaccurate localization of irregular and poorly defined lesions, as well as insufficient model robustness and generalization ability. This invention will achieve high-precision, high-efficiency, and high-reliability automatic segmentation of tumor regions in breast ultrasound images, providing technical support for the early and accurate diagnosis of breast cancer.

[0006] To achieve the above objectives, this invention proposes an automatic segmentation method for breast cancer ultrasound images, which includes the following steps:

[0007] Step 1: Collect a set of breast ultrasound images, perform standardized preprocessing, integrate multiple physician annotations, and perform data augmentation on the images to construct high-quality samples for model training;

[0008] Step 2: Under a unified training framework, construct and jointly train a set of neural network models, which includes a first model, a second model, and a third model. The first model is a U-Net architecture integrating an attention gating module, used to focus on key features of the tumor region through an attention mechanism. The second model is a U-Net architecture incorporating a cross-shaped window Transformer module, used to collaboratively extract local details and global contextual features through a cross-shaped window attention mechanism. The third model is a U-Net architecture integrating residual convolution and a parallel Swing Transformer module, used to balance local texture and global semantic information through parallel feature extraction paths.

[0009] Step 3: Use a multi-dimensional evaluation system based on 5-fold cross-validation to evaluate the set of models trained in Step 2, and select the optimal segmentation model based on the evaluation results.

[0010] Step 4: Perform end-to-end tumor region segmentation on the input breast ultrasound image using the optimal segmentation model, and output visualization results and quantitative reports.

[0011] Optionally, step 1 specifically includes:

[0012] Step 11: Acquire breast ultrasound images, including benign, normal, and malignant images;

[0013] Step 12: Multiple manually annotated ultrasound images of the same type are fused using a pixel-level voting fusion algorithm to generate a single baseline ground truth image, in order to ensure the consistency and accuracy of the annotations.

[0014] Step 13: Perform pixel value normalization and denoising on all images, and perform data augmentation by at least one of random flipping, rotation, scaling, and adding noise.

[0015] Optionally, the first model in step 2 includes an encoder consisting of multiple sequentially connected convolutional layers, activation functions, dropout layers, and max pooling layers; the decoder consists of multiple sequentially connected deconvolutional layers, skip connection layers, convolutional layers, and normalization layers; wherein, the attention gate AG receives the encoded signal and the decoded signal respectively, and generates weights through the attention mechanism to achieve adaptive weighting of the encoded signal.

[0016] Furthermore, the second model in step 2 employs a cross-shaped window self-attention mechanism to divide the input feature map into equal-width, non-overlapping horizontal and vertical strips; within each strip, long-range dependencies are captured in parallel through horizontal and vertical strip self-attention; self-attention calculation is performed after grouping the feature map, thereby reducing computational complexity while ensuring modeling capability.

[0017] In step 2, local detail features such as edges and textures of the image are extracted by residual convolution, and long-range dependency features of the tumor region are extracted by modeling global context relationships using Swing Transformer. An adaptive weighted summation fusion strategy is used to fuse the output features of each sub-module to achieve a dynamic balance between local detail features and global semantic features.

[0018] Optionally, the training framework in step 2 employs the AdamW optimizer and applies a cosine annealing learning rate decreasing strategy to adjust the learning rate; simultaneously, a hybrid loss function is used as the training objective, which is a weighted sum of cross-entropy loss and Dice loss, and its expression is:

[0019]

[0020] Here, Loss is a hybrid loss function, CrossEntropyLoss is the cross-entropy loss used to optimize pixel-level classification accuracy, and DiceLoss is the Dice loss used to optimize the overlap of segmented regions.

[0021] Optionally, the multi-dimensional evaluation system in step 3 includes the mean intersection-over-union ratio (MIoU), accuracy (Acc), Kappa coefficient, and Dice coefficient. The selection of the optimal model must meet the following core performance index thresholds: mean intersection-over-union ratio (MIoU) ≥ 0.8106, accuracy (Acc) ≥ 0.9591, Kappa coefficient ≥ 0.7767, and Dice coefficient ≥ 0.8883. By setting the above key parameters, the accuracy of this method in breast cancer image segmentation and the reliability of its clinical application are guaranteed.

[0022] Optionally, the segmentation results output in step 4 include a visual segmentation map superimposed on the original image, and a quantitative analysis report containing the area and maximum diameter of the tumor region.

[0023] In addition, to achieve the above objectives, the present invention also proposes an automatic segmentation system for breast cancer ultrasound images. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the automatic segmentation method for breast cancer ultrasound images as described in any of the above claims.

[0024] On the other hand, to achieve the above objectives, the present invention also proposes a storage medium comprising a computer program that, when executed by a processor, implements the automatic segmentation method for breast cancer ultrasound images as described in any of the preceding claims.

[0025] Beneficial effects:

[0026] (1) This invention constructs a "model family" comprising a basic model and two innovative extended models, and employs 5-fold cross-validation and a unified training framework for systematic comparison and screening to ensure the robustness of the final solution. Simultaneously, the data preprocessing stage utilizes multi-expert annotation fusion, standardization, and strong data augmentation strategies to significantly improve the quality and diversity of the training data. These measures synergistically enhance the model's adaptability to different ultrasound imaging devices, different patient images, and various complex cases, enabling it to perform stably in multiple clinical scenarios and possess strong generalization ability.

[0027] (2) This invention is end-to-end clinically oriented, pursuing high algorithm accuracy while also considering inference efficiency and output practicality. The optimal model's segmentation inference time per image is ≤0.5 seconds, fully meeting the efficiency requirements of large-scale clinical screening and real-time diagnosis. Its output is not a single segmentation mask, but provides a quantitative analysis report including a visualized segmentation overlay, morphological parameters such as tumor area and maximum diameter, and a complete auxiliary diagnostic information package based on a reliability confidence score according to predicted probability. This design can significantly reduce the workload of physicians, improve the objectivity and consistency of diagnosis, and seamlessly integrate into clinical workflows.

[0028] (3) This invention constructs a complete systematic solution, covering the entire process from data preparation, multi-model innovative design, fair evaluation to optimal model application. The modular design concept, especially the hierarchical architecture of the three models, facilitates subsequent optimization and upgrades for specific clinical needs. For example, performance can be further improved on the existing framework by introducing more advanced attention mechanisms or feature fusion modules, demonstrating good scalability and potential for technological iteration.

[0029] (4) This invention provides a high-precision, robust, efficient and clinically applicable automatic segmentation method for breast cancer ultrasound images. Through systematic model architecture innovation and rigorous evaluation process, it effectively solves the core defects of existing technologies such as insufficient segmentation accuracy, weak generalization ability and poor clinical adaptability, and provides reliable technical support for early accurate diagnosis and quantitative analysis of breast cancer. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the following are only some drawings of the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the basic architecture of the U-Net model in the prior art, used for comparison and reference with the solution of this invention.

[0032] Figure 2 This is an overall schematic diagram, and the invention creates one implementation of an automatic segmentation method for breast cancer ultrasound images.

[0033] Figure 3 This is a schematic diagram of three types of tissue samples (benign, normal, and malignant) in breast ultrasound images, illustrating the composition of the dataset samples.

[0034] Figure 4 This diagram illustrates the annotation integration strategy used to generate baseline ground truth values ​​by fusing annotations from multiple physicians, demonstrating the annotation integration strategy in the data preprocessing stage.

[0035] Figure 5 This is a detailed architecture diagram of the Attention U-Net model, showing the overall structure of the encoder, decoder, and attention gate (AG) module.

[0036] Figure 6 This diagram shows the internal structure of the attention gate AG module in Attention U-Net, illustrating the computational logic of the soft attention mechanism.

[0037] Figure 7 This is a schematic diagram of the overall architecture of the CSWin-Unet model, demonstrating the integration of U-Net and the CSWin Transformer module.

[0038] Figure 8 This diagram illustrates the internal structure and data processing flow of the CSWin Transformer module, explaining the execution flow of cross-shaped window attention and dynamic window adjustment.

[0039] Figure 9 This is a schematic diagram of the core feature flow of the CSWin-Unet model, illustrating the process of extracting local details and global context features in combination.

[0040] Figure 10 This is a schematic diagram of the core architecture of the MRCST-Net model, showing the integration relationship between the RCST composite module and the multi-scale attention fusion module.

[0041] Figure 11 The flowchart of the automatic segmentation method for breast cancer ultrasound images in Example 1 outlines the complete steps from data preparation to result output.

[0042] Figure 12 The GradCAM visualization shows the segmentation results of the CSWin-Unet model at different training rounds, illustrating the changing trends of the prediction mask and heatmap during the training process.

[0043] Figure 13 The following are performance monitoring curves for the CSWin-Unet model training process: 13A shows the trend of model loss with training rounds, 13B shows the trend of model accuracy with training rounds, and 13C shows the trend of the model's average intersection-over-union ratio with training rounds.

[0044] Figure 14 The image shows the results of tumor region segmentation and post-processing optimization using the CSWin-Unet model, comparing the original image, initial prediction, and morphologically optimized segmentation results.

[0045] Figure 15 This image visualizes the tumor region segmentation results of the CSWin-Unet model, presenting the complete output results including the original image, the segmentation overlay image, and the pseudo-color mask image.

[0046] Figure 16 This is a system architecture block diagram for automatic segmentation of ultrasound images of breast cancer according to the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the above-mentioned drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0049] Figure 1The diagram shows the overall architecture of the U-Net model, clearly illustrating the entire process of feature extraction, feature fusion, and segmentation prediction from the input layer to the output layer. It includes the connection relationships and data flow of core components such as the encoder, decoder, attention gate (AG) module, and skip connections. Based on a U-shaped structure, this architecture achieves accurate segmentation of tumor regions in breast ultrasound images through "encoding-decoding-feature fusion." The functions of each core component are as follows: The input layer primarily receives standardized preprocessed 500×500×3 breast ultrasound images, providing the model with input data in a unified format. The descent path on the left is the encoder, containing four coding units. Each layer consists of a 3×3 convolutional layer → a Dropout layer with a dropout probability of 0.3 → a 2×2 max pooling layer with a stride of 2, finally connected to a non-pooling coding layer. By progressively downsampling to expand the receptive field, multi-scale image features from low-dimensional texture to high-dimensional semantics are extracted. The attention-gated (AG) module is located in the skip connection path between the encoder and decoder. Its input consists of the encoder's unpooled features and the output features of the previous layer in the decoder. Attention weights are generated to adaptively weight the encoder features, enhancing key tumor region features and suppressing background noise. The ascending path on the right leads to the decoder, which contains four decoding units, corresponding one-to-one with the encoder. Each layer consists of a 2×2 deconvolution layer → skip connection layer → 3×3 convolution layer → batch normalization (BN) layer. Deconvolution restores the image spatial resolution and is simultaneously concatenated with the attention-gated features from the encoder. The output layer uses a 1×1 convolution layer to map the multi-channel feature map to a single-channel segmentation map. A sigmoid activation function is used to obtain pixel-level predicted probabilities with a threshold of 0.5. The final output is a binary segmentation result of the tumor region (probability > 0.5) and normal tissue (probability < 0.5). The overall architecture, through the combination of skip connections and the attention mechanism, not only compensates for feature loss during downsampling but also improves the discriminative power of tumor region features, providing core support for subsequent accurate segmentation. However, traditional U-Net has shortcomings such as insufficient global context information capture and poor stability in segmenting irregularly shaped tumors.

[0050] Furthermore, the technical problem this invention aims to solve is how to improve the segmentation accuracy and model generalization ability of breast cancer ultrasound images, especially for tumor regions with blurred boundaries and irregular shapes. To address this problem, this invention provides an automatic segmentation method for breast cancer ultrasound images. Its core concept lies in: integrating a set of deep learning models with multiple differentiated feature extraction mechanisms and establishing a unified evaluation framework to select the best from these models, thereby constructing a robust and accurate segmentation model. Specifically, the method mainly includes two stages: model pool construction and optimal application.

[0051] In the first stage, the breast ultrasound image dataset was first standardized and preprocessed for enhancement. Then, a model pool containing at least three heterogeneous networks was constructed. These networks are all based on an encoder-decoder framework, but each integrates different core feature extraction modules: the first network utilizes an attention gating mechanism to achieve feature reweighting in skip connections; the second network introduces a cross-shaped window attention Transformer in the encoder to simultaneously capture long-range dependencies; and the third network employs a composite module that combines residual convolution and Transformer in parallel to extract local and global features in a balanced manner. Subsequently, the model pool was jointly trained and rigorously evaluated using 5-fold cross-validation under a unified training configuration.

[0052] In the second stage, the best-performing model is selected from the trained model pool based on multidimensional evaluation metrics (including mean intersection-over-union ratio MIoU, accuracy Acc, Kappa coefficient, and Dice coefficient) and used to perform end-to-end automatic segmentation on new breast ultrasound images, ultimately outputting a visualized segmentation map and a quantitative morphological analysis report.

[0053] Further, see Figure 2 This invention provides a flowchart of an embodiment of an automatic segmentation method for breast cancer ultrasound images, the execution process of which mainly includes the following four steps:

[0054] Step 1 - Data Preparation and Standardization.

[0055] In practice, an initial dataset is constructed by collecting breast ultrasound images covering benign, normal, and malignant categories. To improve annotation quality, for images with multiple physician annotations, a pixel-level voting fusion algorithm is used to generate a single baseline ground truth image. Furthermore, pixel value normalization and denoising are performed on all images, and data augmentation techniques such as random geometric transformations (flipping, rotation, scaling) and noise addition are applied to expand the data scale and enhance diversity, ultimately forming a high-quality training sample set.

[0056] in, Figure 3 The image features of benign, normal, and malignant samples are presented. The images show that malignant tumor images exhibit blurred boundaries and irregular shapes, increasing the difficulty of segmentation. And as... Figure 4 The image shown is a schematic diagram of a single baseline ground truth generated after multiple physicians annotated and fused the same image.

[0057] Furthermore, normalizing pixel values ​​maps grayscale values ​​to the [0,1] range, eliminating grayscale differences caused by different devices; and denoising is achieved by removing original noise through median filtering while preserving lesion boundary information; at the same time, data augmentation through random flipping, rotation, scaling, and Gaussian noise addition can expand the sample size to 3 times the original, improving the model's generalization performance.

[0058] Step 2 - Construction and collaborative training of heterogeneous model pools.

[0059] The model pool comprises three semantic segmentation networks based on an encoder-decoder U-shaped structure: the first model (an Attention U-Net model based on an Attention-Gated U-Net), the second model (a CSWin-Unet model based on a Cross-Window Attention U-Net), and the third model (an MRCST-Net model based on a Residual Convolutional-Transformer Parallel U-Net). All three share the same input specifications (e.g., a 500×500 pixel RGB image) and basic data flow paradigm, but differ fundamentally in the core feature transformation module of the encoder.

[0060] In the first model, the attention-gated AG module is embedded in the skip connection path of U-Net. It simultaneously receives encoder and decoder features and generates weights through a soft attention mechanism to adaptively weight the encoder features. For example... Figure 6 As shown, this module is the core of the Attention U-Net model. The AG module receives the output feature map of the current layer from the encoder. and the output feature map from the previous layer of the decoder Attention weights are generated through multi-layer convolution and activation functions, and the encoder features are adaptively weighted. The calculation process satisfies the formula:

[0061]

[0062] in, These are the weight parameters of the convolutional layer. As the bias parameter, the Sigmoid function maps the attention coefficient to the [0,1] interval to enhance the features of the tumor region and suppress background noise. The output in formula (1) is the output feature of the attention gate AG module, which is used to achieve precise focusing of key features of the tumor region and suppression of background noise.

[0063] The Attention-Gate (AG) module enables adaptive feature selection and reweighting, allowing the network to autonomously decide which parts of the encoder feature map to "focus" on when fusing encoder and decoder features. Furthermore, in medical image segmentation, tumor regions typically occupy only a small portion of the image. The AG module automatically suppresses background noise unrelated to the tumor and feature responses from non-critical regions, while significantly enhancing features relevant to the tumor region. This essentially provides the model with a "visual focus" in the feature space, enabling it to more accurately pinpoint local lesions. Consequently, this process effectively suppresses background noise unrelated to the tumor and focuses on the critical regions of the lesion.

[0064] The CSWin Transformer module in the second model is deeply integrated after each layer of the U-Net encoder to process the multi-scale feature maps output by the encoder, injecting global context into the encoder features. Its workflow includes: splitting the input feature map into non-overlapping local windows; adaptively adjusting the window size based on the variance of the spatial feature values ​​within the window; calculating cross-shaped attention weights along the horizontal and vertical directions within each window; and performing channel fusion after grouping and parallelly calculating attention for the feature maps. In other words, for each layer of the encoder's output feature map, the CSWin Transformer module expands the receptive field of that layer's features through its four-step process: feature map splitting → dynamic window adjustment → cross-shaped attention calculation → parallel group attention fusion. It not only focuses on information within local windows but also simultaneously captures long-range dependencies of the feature map in the horizontal and vertical directions through cross-shaped attention. Its output is an enhanced version of the features that incorporates global context information. The summary is as follows:

[0065] ① Feature map splitting: The input feature map is split into non-overlapping local windows (e.g., 16×16 window size), with each window corresponding to a local region of the image;

[0066] ② Dynamic window adjustment: The strategy adaptively adjusts the window size (8×8-32×32) based on the variance of the feature values ​​within each window. Small windows are used for regions with rich details, such as tumor edges, to preserve fine features, while large windows are used for homogeneous background regions to improve computational efficiency.

[0067] ③ Cross-shaped attention calculation: Within each window, The mechanism calculates attention weights in parallel along the horizontal and vertical directions. This "cross-shaped" design enables it to simultaneously capture long-range dependencies in both length and width dimensions of the image, effectively modeling the overall contour and spatial distribution of the tumor.

[0068] ④ Parallel grouped attention fusion: The feature map is divided into G groups according to the channel dimension (preferably divided into 4 equal groups). Each group performs the above cross-shaped attention calculation independently. Finally, the results are concatenated and channel fusion is performed through 1×1 convolution to balance computational efficiency and feature diversity.

[0069] This module efficiently introduces global context modeling capabilities into convolutional neural networks, overcoming the limitations of local receptive fields in traditional convolutional operations. Furthermore, for breast cancer tumors, especially malignant lesions, which often have irregular invasive boundaries, understanding their overall contours and spatial relationships with surrounding tissues requires global information. The CSWin Transformer, through its cross-shaped window attention mechanism, enables the model to directly establish the correlation between features at any two locations in an image (even those far apart), thereby effectively modeling the long-range spatial dependencies and global semantic context of tumors. This is crucial for segmenting tumors with blurred boundaries.

[0070] The RCST composite module in the third model contains two parallel processing paths:

[0071] The residual convolutional submodule consists of multiple convolutional layers with skip connections, focusing on extracting local detail features of the image, such as edges and textures (the output is denoted as Y). local );

[0072] The Swin Transformer submodule: Based on a window-moving attention mechanism, it excels at modeling global contextual relationships and long-range semantic dependencies in images (output denoted as Y). global );

[0073] and parallel processing of input feature maps The superscript H represents the height of the input feature map, W represents the width of the input feature map, and C represents the number of channels in the input feature map. Given the set of real numbers, this formula defines the input feature map as a set containing real number elements. Three-dimensional data blocks; the outputs of the two sub-modules are passed through a pair of learnable adaptive weight parameters. and The weighted summation and fusion are specifically expressed as follows:

[0074] (2)

[0075] (3)

[0076] (4)

[0077] in, This represents the residual convolution submodule, used to extract local detail features of an image; This represents the Swing Transformer submodule, which is used to model the global context of an image; and These are the outputs of the two sub-modules, respectively. and It is a learnable adaptive weighting parameter used to dynamically balance the importance of local and global features.

[0078] The RCST composite module constructs a feature extraction unit that can synchronously and balancedly extract local details and global semantics, and dynamically fuses the two through a data-driven approach. It addresses the feature extraction problem of dynamically balancing local and global features. Medical image analysis requires both convolutional networks to capture local texture details to define precise boundaries and Transformers to understand global structure to confirm lesion extent. The RCST module does not simply concatenate or integrate the two in a primary-secondary manner, but rather allows them to work in parallel, each performing its specific function. Finally, through learnable... and By fusing features, the network can adaptively determine the contribution ratio of local and global features in the final output based on the specific circumstances of the current features, thereby achieving a more robust and accurate feature representation.

[0079] Furthermore, the first model—the Attention U-Net model—innovatively introduces an Attention Gate (AG) module into the skip connection path of the standard U-Net to achieve adaptive feature selection and enhancement. The model fully retains the main structure of the standard U-Net's encoder (downsampling) and decoder (upsampling). The encoder extracts multi-scale features through convolution and pooling, while the decoder gradually restores resolution through deconvolution.

[0080] Specifically, the encoder comprises multiple sequentially connected convolutional layers, activation functions, dropout layers, and max pooling layers. Preferably, as follows: Figure 5 As shown, the encoder contains four coding units. Each layer consists of two 3×3 convolutional layers (each convolution followed by a ReLU activation function), a Dropout layer (with a dropout probability of 0.3), and a 2×2 max-pooling layer (with a stride of 2) to progressively downsample and extract multi-scale features. The output of the last coding unit is connected to a non-pooling coding layer to preserve high-dimensional semantic features. The encoder feature extraction process satisfies the following formula:

[0081] (5)

[0082] Wherein, the input x is the original normalized image or the output feature map of the previous encoder layer. For learnable weight parameters of convolutional layers, For bias parameters, Represents the ReLU activation function. For max pooling, used to expand the receptive field and preserve key features, in formula (5) It is the output feature map of the encoder.

[0083] The decoder is symmetrical to the encoder and includes multiple layers of sequentially connected deconvolutional layers, skip connection layers, convolutional layers, and normalization layers. Preferably, the decoder structure includes four decoding units, corresponding one-to-one with the encoder units. Each layer consists of a 2×2 deconvolutional layer, a skip connection layer, two 3×3 convolutional layers, and a BN layer. The decoder establishes skip connections with the encoder through an attention gate module, and the decoding process satisfies the formula:

[0084] (6)

[0085] (7)

[0086] Here, the input x consists of the output features of the previous layer decoder and the encoder features passed through the skip connections. The encoder layer's unpooled output features are represented by `upsample` (deconvolution upsampling operation), `concatenate` (skip connection operation), `encoding` (processing result of the unpooled encoder layer), `w` (weight), `b` (bias), and `wx+b` (a basic feature transformation performed on the output features of the previous decoder layer to prepare for upsampling). In formula (6), `output` is the encoder's output feature map. In formula (7), ... Represents the ReLU activation function. For learnable weight parameters of convolutional layers, This is the bias parameter.

[0087] The AG (Attention Graph) module intervenes when encoder feature maps need to be skipped to the corresponding decoder layer for fusion; these features are not directly passed. Instead, they are first fed into the AG module. The AG module then simultaneously receives the feature map output from the current encoder layer and the feature map from the previous layer of the decoder. It utilizes the more semantic contextual information provided by the decoder to generate a spatially adaptive attention weight map for the encoder features.

[0088] The output layer of this model uses a 1×1 convolutional layer to map the multi-channel feature map output by the decoder into a single-channel segmentation map. The pixel-level prediction probability is obtained through the Sigmoid activation function. The probability threshold is set to 0.5. Pixels with a probability greater than the threshold are identified as tumor regions, and those with a probability less than the threshold are identified as normal tissue.

[0089] The model works by having the encoder extract multi-scale features through convolution and pooling; the attention gate (AG) module receives features from the encoder and upper-layer features from the decoder, generates spatial attention weights, and adaptively weights the transmitted encoder features to enhance tumor region features and suppress background noise; the decoder restores image resolution and outputs a segmentation map through deconvolution and skip connections. The encoder features, weighted and enhanced by the AG module (suppressing the background and focusing on suspected tumor regions), are then concatenated with the decoder features. This integration does not alter the main data flow path but adds a "smart regulator" to the crucial "feature bridge" (skip connection), enabling the model to automatically focus on key regions during feature fusion, thereby improving the segmentation accuracy of local tumor details.

[0090] The second model, the CSWin-Unet model, is a CSWin-Unet model that integrates the U-Net structure with the CSWin Transformer module. Its core is to embed the CSWin Transformer module after the downsampling unit of each stage of the U-Net encoder, and then connect it in series with the convolutional layers. Through a cross-shaped window self-attention mechanism and a hierarchical window strategy, it achieves the collaborative extraction of local details and global contextual features. The model maintains the U-Net encoder-decoder equivalence structure. The encoder part is still responsible for feature extraction and downsampling.

[0091] The specific construction and working mechanism of the CSWin-Unet model are as follows: Figure 7-9 As shown, the overview is as follows:

[0092] The input layer performs channel normalization on the image, providing a stable and de-biased input for the encoder. Preferably, a pre-processed 500×500×3 breast ultrasound image is received, and channel dimension normalization is used to eliminate grayscale differences between channels.

[0093] The fusion of the encoder and CSWin Transformer involves the following: after each convolutional unit of the encoder outputs a feature map, it is immediately input into a corresponding CSWin Transformer module. Therefore, the CSWin Transformer processes multi-scale features from shallow to deep layers. The encoder module provides multi-scale feature maps to the CSWin Transformer, containing four encoding units that progressively downsample to produce feature maps with increasing channel counts (64, 128, 256, 512) and decreasing spatial resolution. Preferably, each layer structure is "two 3×3 convolutions → BN layer (batch normalization layer) → 2×2 max pooling with a stride of 2," with each encoding unit outputting feature maps with 64, 128, 256, and 512 channels respectively. The output of each layer is immediately input into the corresponding CSWin Transformer module. This means that the CSWin Transformer processes not single-scale features, but a multi-scale feature set ranging from detailed shallow features to semantically abstract deep features, thereby achieving the fusion of local convolutional features and global attention features. The CSWin Transformer module performs cross-shaped window attention computation on the received feature maps at a specific scale, enabling it to model the long-range spatial dependencies between features at that scale. After processing, the output global context-enhanced features are passed to the next encoder layer for deeper abstraction, and also saved as skip features for later fusion with the decoder. This integration of the encoder and CSWin Transformer is equivalent to adding a "global information enhancement stage" after each level of feature extraction in the encoder. It allows the model to retain the ability of CNNs to extract local details while fusing a global perspective at each layer, which is particularly beneficial for understanding the overall contour and spatial distribution of malignant tumors with blurred boundaries and irregular shapes.

[0094] The decoder module combines the globally enhanced features from each layer of the CSWin Transformer module with its own upsampled features through skip connections, gradually reconstructing a high-resolution segmentation map that combines detail and overall integrity. It contains four decoding units, corresponding one-to-one with the encoder units. Each layer's structure is "2×2 deconvolution → skip connection → two 3×3 convolutions → BN layer." This achieves local-global coordination; the decoder itself processes upsampled features through convolution, which can be seen as local smoothing and integration, and then combines this with the encoder's global features, achieving the final level of coordinated modeling of local details and global context.

[0095] The output layer of this model is consistent with Attention U-Net, outputting a single-channel segmentation map. The final tumor region is obtained by segmentation through probability thresholds, and the boundary probability map of the tumor region is also output to help doctors determine the extent of tumor invasion.

[0096] To more clearly illustrate the data processing flow of the CSWin-Unet model, its main computational steps can be represented as follows:

[0097] Let the encoder be the first The feature map output by the layer is , For the set of real numbers, superscript For the first The height of the layer output feature map, For the first The width of the layer output feature map. For the first The layer outputs the number of channels in the feature map; this feature map is then input into the corresponding CSWin Transformer module. The overall computation process of this module can be summarized as follows:

[0098] (8)

[0099] Where CSWin-T(·) represents the processing function of the CSWin Transformer module, This represents the feature map of the input to the i-th layer of the encoder; This means splitting the input feature map into a series of non-overlapping local windows; This represents a dynamic window resizing strategy that adaptively adjusts the window size based on the variance of the spatial eigenvalues ​​within the window. Indicates the first The cross-shaped window attention mechanism performed by the group features calculates attention weights in the horizontal and vertical directions to capture long-range dependencies in the image. This indicates that the splicing operation is performed along the channel dimension; G represents the total number of attention groups; g represents the index of the current group, used to traverse all groups from 1 to G; A 1×1 convolution is used to perform channel fusion on the concatenated features after parallel grouping and computation, outputting an enhanced feature map. .

[0100] Finally, the enhanced feature map The image is passed to the decoder via skip connections and fused with the upsampled features from the decoder to restore image details and generate the final segmentation result.

[0101] The third model, the MRCST-Net model, integrates the RCST module and a multi-scale attention fusion component. Specifically, the RCST composite module is deeply replaced by the core feature extraction unit of each level of the U-Net encoder. Furthermore, the model integrates a multi-scale attention fusion module for feature aggregation.

[0102] The RCST module is configured with a parallel configuration of residual convolution and a Swing Transformer sub-module. The core of this method lies in achieving feature balance between local image details and global context through the RCST module, followed by explicit and adaptive fusion of multi-scale image features using a multi-scale visual fusion model, and finally, skip connections in the decoder to achieve accurate image reconstruction. The computational processes of these two core components are represented as follows:

[0103] Multi-scale attention fusion module: This module integrates multi-scale feature maps output from different levels of the encoder. The fusion process begins by unifying all feature maps to the same resolution using interpolation. ,in To unify the height of the feature map, To unify the width of the feature maps, channel attention and spatial attention are then applied in parallel, followed by feature aggregation. The computation process can be represented as follows:

[0104] (9)

[0105] (10)

[0106] (11)

[0107] in, These are feature maps output from different levels of the encoder, with subscripts... Indicates hierarchy; yes After bilinear interpolation resize operation, the result is consistent with the target resolution. A unified feature map of the k-th layer; Yes Channel attention and spatial attention were applied, and the k-th layer feature map was weighted. This represents a bilinear interpolation operation used to unify the spatial resolution of feature maps. and These are channel attention mechanism and spatial attention mechanism, respectively. This indicates element-wise multiplication, used to apply the calculated attention weights to the original features; This indicates splicing along the channel dimension; A 1×1 convolution is used to compress the channels of the concatenated feature maps, generating the final fused features. .

[0108] The core architecture of the MRCST-Net model is as follows: Figure 10 As shown, the specific construction is as follows:

[0109] Input layer: Receives standardized breast ultrasound images of the same specifications as CSWin-Unet to ensure input consistency.

[0110] The encoder module comprises four coding units, each with an RCST composite module at its core, relying on the output feature map of the previous layer for deep abstraction. The RCST module employs a parallel structure of "residual convolution + Swing Transformer," achieving a dynamic balance between local and global features through adaptive weighted fusion of the two outputs, ensuring gradient backpropagation and avoiding gradient vanishing. Thus, each layer of the encoder is no longer simply a stack of convolutions, but is replaced by an RCST composite module. This module internally runs residual convolution and Swing Transformer sub-modules in parallel. When input features enter a layer of the encoder, they are processed by the RCST module of that layer. The module extracts local detail features and global semantic features in parallel, and dynamically fuses them through learnable weights, outputting a hybrid feature that balances local and global information. This feature is then downsampled through pooling and input to the next layer's RCST module. The integration of the RCST module is an upgrade to the encoder's basic "cells," enabling it to produce a feature representation that balances local and global information in a single processing step.

[0111] Multi-scale attention fusion module: Integrates multi-scale features output by the encoder, first unifies the feature resolution through bilinear interpolation or max pooling, then applies channel attention (SE module) and spatial attention in parallel, and finally compresses the channels through 1×1 convolution to output a feature map that fuses multi-scale information, taking into account both feature integrity and information losslessness.

[0112] Decoder module: It contains four decoding units, each with a structure of "2×2 deconvolution → skip connection → RCST composite module → BN layer"; the core is to inject multi-scale attention fusion features at each step, and combine them with the RCST module to refine features, thereby enhancing reconstruction accuracy and detail restoration capabilities.

[0113] Output layer: The process of “1×1 convolution → Sigmoid activation → morphological post-processing” is adopted. 1×1 convolution and Sigmoid activation generate pixel-level probability maps. Erosion-dilation operation eliminates false positives and smooths the boundaries, improving the practicality of segmentation results.

[0114] Furthermore, to ensure fairness in the comparison and efficient collaborative training, the three models are placed under a completely consistent training framework.

[0115] Specifically, a unified training framework is set up to jointly train the Attention U-Net, CSWin-Unet, and MRCST-Net models. The AdamW optimizer is preferably used as the unified training framework because it can automatically adjust the learning rate and performs stably in sparse gradient or noisy conditions. Training parameters are set with an initial learning rate of 0.001, a commonly used and moderate starting point in deep learning. A cosine annealing decay strategy is used to smoothly reduce the learning rate from the initial value to a minimum of 1×10⁻⁶. -5 This approach facilitates rapid convergence and allows for precise model tuning later in training. Setting the batch size to 8 strikes a balance between limited GPU memory and training stability. Training for 20 epochs provides ample learning time for the model.

[0116] Introducing the hybrid loss function Loss, the expression is:

[0117] (12)

[0118] CrossEntropyLoss is used to optimize the type imbalance problem, forcing the model to focus on the minority of tumors that are difficult to classify; DiceLoss is used to improve the overlap of tumor region segmentation, directly optimizing the overlap between the predicted region and the real region. The weighted fusion uses a ratio of 0.6:0.4, which indicates that the scheme focuses more on solving type imbalance while taking into account the shape accuracy of the segmented region.

[0119] Furthermore, the training optimization strategy for the Attention U-Net model includes:

[0120] (1) The Xavier initialization method, an algorithm that adaptively adjusts the initial weight values ​​based on the number of input and output neurons, is used to initialize the weights of the convolutional network and ensure gradient stability in the initial stage. Its principle is to adjust the initial weight values ​​according to the number of input and output neurons, ensuring that the weights maintain a reasonable amplitude during forward propagation and remain stable during backward propagation. This method can accelerate the early convergence speed of the neural network, avoid gradient vanishing or exploding problems caused by improper weights, and lay the foundation for network optimization.

[0121] (2) A dynamic adjustment strategy is adopted in the Dropout layer: the Dropout rate is set to 0.3 in the early stage of training (1-10 epochs, i.e., training rounds) and to 0.1 in the middle stage of training (11-20 epochs). Dropout, as a regularization method, suppresses overfitting by randomly deactivating neurons. In the early stage of training, a high Dropout rate is used to explore features to avoid overfitting to noisy data and improve the model's generalization performance and learning efficiency; in the later stage of training, a low Dropout rate is used to converge the model, reducing the constraints of the regularization term, making the model closer to the optimal solution, and effectively achieving a balance between fitting and generalization.

[0122] (3) A gradient clipping algorithm is used to limit the gradient norm to within 5, effectively solving the gradient explosion problem during training. Its core principle is to scale the gradient during backpropagation, constraining it to a preset threshold. This method can suppress gradient explosion in complex models such as deep neural networks and recurrent neural networks, preventing training instability or failure, and ensuring stable network training.

[0123] Furthermore, the parameter optimization of the CSWin Transformer module includes: introducing layer normalization during attention calculation to stabilize the input feature distribution and accelerate model convergence.

[0124] Furthermore, the multi-scale attention fusion module optimization of the MRCST-Net model includes:

[0125] (1) The compression coefficient of the channel attention module is set to 16. The channel weight adaptive learning is achieved by “global average pooling → 1×1 convolution dimensionality reduction → ReLU activation → 1×1 convolution dimensionality increase → Sigmoid activation”.

[0126] (2) The spatial attention module uses a 3×3 convolutional layer with 64 kernels to balance feature extraction capability and computational efficiency.

[0127] Step 3 - Model evaluation and selection based on cross-validation.

[0128] To fully validate the model's generalization ability and performance reliability, a 5-fold cross-validation strategy was adopted. The breast ultrasound image dataset was randomly and uniformly divided into 5 mutually exclusive subsets, with the training set comprising 80% and the test set 20% in each subset. Four subsets were alternately selected as the training set and one subset as the independent test set, repeating this process for 5 rounds to ensure that each subset was used as the test set only once. The average of the 5 rounds of multi-dimensional evaluation metrics was taken as the final model performance index, effectively avoiding the randomness of a single partition while ensuring the consistency of the distribution of training and validation data. An early stopping strategy was implemented during each fold of training; training was automatically stopped when the test set loss did not decrease for 3 consecutive epochs to avoid overfitting. The loss curves of the training and test sets and various evaluation metrics were monitored in real time during training to ensure the model's convergence stability.

[0129] A multi-dimensional evaluation indicator system quantifies the quality of model segmentation from different perspectives using four core indicators, avoiding the one-sidedness of a single indicator. Details are as follows:

[0130] ① Mean Intersection over Union (MIoU): Measures the overall accuracy of segmented regions at the pixel level and is a core metric in semantic segmentation. It considers both the correctness and completeness of predictions and is highly sensitive to segmentation errors (false positives, false negatives). It is the arithmetic mean of the ratios of the intersection and union of each type of prediction result with the true value, calculated using the following formula:

[0131] (13)

[0132] Where C represents the type number, i.e., tumor versus normal tissue, TP c FP represents the number of true positive pixels in class c. c For the number of false positive pixels, FN c This represents the number of false negative pixels.

[0133] ② Accuracy (Acc): Measures the proportion of all pixels correctly classified. When background pixels constitute the majority, the type imbalance is extreme, and this metric can easily be inflated; therefore, it needs to be analyzed in conjunction with other metrics. The proportion of correctly predicted pixels out of the total number of pixels is calculated using the following formula:

[0134] (14)

[0135] Wherein, TN represents the number of true negative pixels, TP represents the number of true positive pixels, FP represents the number of false positive pixels, and FN represents the number of false negative pixels.

[0136] ③ Kappa coefficient: Used to assess the degree of consistency between model predictions and ground truth annotations, excluding the influence of random agreement. In medical image analysis, this is an important indicator because it reflects the consistency and reliability between model results and doctor annotations. The consistency test index based on the confusion matrix is ​​calculated using the following formula:

[0137] (15)

[0138] in, For the overall classification accuracy, The desired classification accuracy is [value].

[0139] ④ Dice coefficient: Directly corresponding to DiceLoss in the loss function, it specifically measures the accuracy of the predicted tumor region, i.e., the degree of overlap between the predicted tumor region and the actual tumor region. The calculation formula is:

[0140] (16)

[0141] Where TP represents the number of true positive pixels, FP represents the number of false positive pixels, and FN represents the number of false negative pixels.

[0142] Furthermore, based on the evaluation results, the optimal model is selected from the models trained in step 2, and the optimal model is used to segment the tumor region of the input breast ultrasound image, and the segmentation results are output.

[0143] The model selection requires the evaluation indicators to meet the following criteria: MIoU≥0.8106, Acc≥0.9591, Kappa coefficient≥0.7767, Dice coefficient≥0.8883, and segmentation accuracy ≥90% for irregularly shaped and poorly defined tumors.

[0144] After selecting the optimal model, the breast ultrasound image to be segmented is input into the optimal model for segmentation inference, and the segmentation result is output. Specifically, the breast ultrasound image to be diagnosed clinically is input into the optimal model for forward inference, and the model can output a pixel-level tumor probability map in a short time (e.g., within 0.5 seconds). The system then performs post-processing on the probability map and generates two results: one is a visualization of the segmented region superimposed on the original image in pseudo-color (e.g., green); the other is a report that automatically calculates and lists the quantitative parameters of the tumor region, including at least the area and maximum diameter, providing intuitive and quantitative evidence for clinical diagnosis.

[0145] Furthermore, to better illustrate the automatic segmentation method for breast cancer ultrasound images of the present invention, specific examples are provided below, covering the complete process from data preparation, model construction and training, evaluation and selection to clinical application. A detailed flowchart is shown below. Figure 11 .

[0146] Example 1

[0147] Step 1: Data preparation and preprocessing.

[0148] First, a complete breast ultrasound image dataset was constructed. This invention uses clinical case data collected in 2018, including 780 breast ultrasound images from 600 patients aged 25-75 years. The image size is 500×500 pixels, comprising three categories of samples: 220 normal, 280 benign, and 280 malignant. See the example below for details. Figure 3 The characteristics of normal breast ultrasound images are: homogeneous glandular structure, no obvious space-occupying lesions, no prominent edges or irregular shapes, totaling 220 cases, accounting for 28.2%, which serve as the reference standard for no tumors. Benign lesions show clear space-occupying lesions on ultrasound images, mostly round or oval in shape, with clear boundaries, significant contrast with surrounding normal tissue, and homogeneous internal echoes, totaling 280 images, accounting for 35.9%, with clear edge features, facilitating feature extraction. Malignant lesions (breast cancer) show complex morphology and blurred boundaries on ultrasound images, often exhibiting invasive growth, unclear boundaries with surrounding tissue, heterogeneous internal echoes, and often accompanied by speckled noise or artifacts, totaling 280 images, accounting for 35.9%. Among these, 100 cases have blurred boundaries and complex shapes, representing a challenge for segmentation tasks, requiring global scene modeling and multi-scale feature fusion for accurate segmentation. Through standardized preprocessing and data augmentation, a high-quality dataset is constructed to provide data support for the training, validation, and testing of three types of models: Attention U-Net, CSWin-Unet, and MRCST-Net.

[0149] Secondly, the acquired raw data is normalized as follows:

[0150] (1) Labeling integration: For 16 breast ultrasound images manually labeled by 3 or more physicians, the pixel-based majority voting method was used to complete the labeling integration and generate reference labels with multi-physician consensus. After verification by physicians, the labeling consistency reached 96.8%, providing accurate gold standard labels for model training.

[0151] (2) Data augmentation effect: By randomly flipping, rotating, scaling and adding Gaussian noise, the dataset size was expanded from 780 images to 2340 images. The type distribution of the augmented dataset remained balanced, with benign: normal: malignant ≈4:3:4.

[0152] (3) Dataset partitioning: Five-fold cross-validation is used, with the training set accounting for 80% and the test set accounting for 20% in each fold, to ensure the consistency of the distribution of training and validation data.

[0153] Step 2, Model Building.

[0154] Based on the PyTorch deep learning framework, we completed the construction of three models: Attention U-Net, CSWin-Unet, and MRCST-Net. These three models adopt a unified input specification and basic architectural paradigm, and are differentiated and improved for the task of tumor segmentation in breast ultrasound images. The specific construction method is as follows:

[0155] The Attention U-Net model is based on U-Net and introduces an attention-gated AG module to achieve precise focusing of tumor features. Its parameters and structure are strictly matched to clinical segmentation requirements. The encoder channels are 64, 128, 256, and 512 respectively; the decoder channels are 256, 128, 64, and 32 respectively; the attention-gated convolution kernel size is 3×3; and the input image size is 500×500×3. The encoder consists of 5 encoding blocks. The first 4 blocks are downsampled sequentially, and the last layer is unpooled to preserve high-dimensional semantics. Each unit consists of a 3×3 convolution, ReLU activation, Dropout, a second 3×3 convolution, and a 2×2 max pooling. The features before pooling are output as skip connections. The attention-gated AG module receives the unpooled features from the encoder and the features from the upper layer of the decoder. After convolution alignment, size matching, weight generation, and feature weighting, it achieves key feature enhancement and noise suppression. The decoder consists of 4 decoding blocks. It upsamples through 2×2 deconvolution and then concatenates and fuses the attention-gated weighted encoding features to restore image resolution layer by layer. The output layer uses 1×1 convolution combined with Sigmoid activation to output a pixel-level segmentation probability map, thus completing the localization of the tumor region.

[0156] The CSWin-Unet model deeply integrates U-Net and CSWin Transformer, enhancing global feature modeling capabilities through cross-shaped window attention. The core parameters—encoder channel count and Attention U-Net—are consistent. The CSWinTransformer has an initial window size of 16×16 and a dynamic window range of 8×8 to 32×32; it has 4 groups and 8 attention heads, employing a four-layer encoder-four-layer decoder structure. The basic encoder outputs multi-scale feature maps through four convolutional units, directly feeding them into the CSWin Transformer module. This module first splits the feature maps into non-overlapping local windows, adaptively adjusting the window size based on spatial feature variance. Then, it performs cross-shaped window attention in the horizontal and vertical directions, performing parallel computation in groups before channel fusion via 1×1 convolution, simultaneously capturing local details and global long-range dependencies. The decoder fuses the enhanced encoded features through deconvolutional upsampling and skip connections, accurately reconstructing tumor boundaries and contours. The output layer outputs a single-channel segmentation probability map, while also providing boundary probability information to aid in determining the extent of tumor invasion.

[0157] The MRCST-Net model, with its core RCST composite module and multi-scale attention fusion, achieves a balanced extraction of local details and global semantics. Core parameters include: RCST module residual convolution (64-256 channels); Swing Transformer window size (4×4); channel attention compression factor of 16; and spatial attention using 3×3 convolution with 64 channels. In terms of implementation, the RCST composite module employs a parallel structure of residual convolution and Swing Transformer to extract local texture details and global semantic information respectively, achieving dynamic feature balance through adaptive weighted fusion. Each layer of the encoder is based on the RCST module, ensuring efficient feature extraction and stable gradient backpropagation. The multi-scale attention fusion module unifies the resolution of features from each layer of the encoder, applies channel and spatial attention in parallel, and concatenates and fuses them to output complete multi-scale features. The decoder injects fused features during upsampling, further refining features in conjunction with the RCST module to improve the restoration of segmentation details. The output layer undergoes 1×1 convolution, sigmoid activation, and morphological post-processing to remove false positives and smooth boundaries, outputting stable and reliable segmentation results.

[0158] Step 3, training configuration.

[0159] To ensure fairness and stability during training of the three models, the AdamW optimizer was used uniformly, with a weight decay coefficient of 1×10⁻⁶. -5 The initial learning rate is 0.001, and with the cosine annealing learning rate scheduler, the minimum learning rate is 1×10. -6 This achieves a smooth decay of the learning rate. The loss function is a weighted combination of binary cross-entropy loss and Dice loss, as shown in the formula: This effectively alleviates the problem of imbalanced positive and negative samples in breast ultrasound images. The batch size was set to 8, and gradient clipping (maximum norm set to 1.0) was used for stable training. An early stopping strategy was also implemented: training was terminated if the Dice coefficient on the test set showed no improvement for five consecutive rounds to prevent overfitting. Accuracy, mean intersection-over-union ratio, and Dice coefficient were used as core evaluation metrics to comprehensively assess the model's segmentation performance.

[0160] Step 4, Model Evaluation and Selection.

[0161] This invention employs a 5-fold cross-validation method to evaluate model performance. The dataset is randomly divided into 5 mutually exclusive subsets, with 4 subsets selected sequentially as the training set and 1 subset as the test set. This training and testing process is repeated 5 times. For each round of testing, four core metrics are calculated: Mean Intersection over Union (MIoU), Accuracy, Kappa coefficient, and Dice coefficient. The average of the 5 rounds of experiments is taken as the final performance metric of the model. These metrics measure model performance from different perspectives, including segmentation overlap, pixel accuracy, consistency, and similarity. The comparison of the final key performance metrics is shown in Table 1.

[0162]

[0163] Table 1 shows the values ​​of the three models, Attention U-Net, MRCST-Net, and CSWin-Unet, on the four core metrics: MIoU, Acc, Kappa, and Dice. Among them, the CSWin-Unet model achieved the maximum on all metrics, obtaining the best result. Therefore, CSWin-Unet was selected as the optimal model for final deployment.

[0164] To verify the training effectiveness and feature extraction capability of the CSWin-Unet model, this invention performs a visual analysis of the model output at different training epochs, as shown in the following figures. Figure 12 As shown in the figure, each row corresponds to a training epoch (8, 13, 17), and from left to right, they are the original labeled mask, the model predicted mask, and the gradient activation heatmap. As can be seen from the figure, with the increase in training epochs, the contour fit between the model predicted mask and the original labeled mask continuously improves, and the focus area of ​​the gradient activation heatmap gradually shifts from the background to the actual tumor region. This fully verifies that the CSWin-Unet model can effectively learn the features of the tumor region and possesses excellent feature extraction and segmentation capabilities.

[0165] Figure 13 The training performance monitoring curves for the CSWin-Unet model include the trends of three key metrics: model loss, model accuracy, and mean intersection-over-union (MIU). Figure 13 A represents the model loss curve, with the blue curve representing the training set loss value and the orange curve representing the test set loss value.

[0166] Depend on Figure 13 As can be seen from A, the loss value of the model decreased rapidly in the early stage of training (Epoch 0-4), and then gradually converged and stabilized at a low level, indicating that the model can effectively learn the feature patterns of the tumor region without obvious overfitting. Figure 13 B represents the model accuracy curve. As the number of training rounds increases, the model's accuracy on the test set continues to climb and stabilizes above 0.98, demonstrating the extremely high stability of the CSWin-Unet model on pixel-level classification tasks. Figure 13 C represents the model's average intersection-over-union (MIoU) curve. The average MIoU on the test set eventually stabilized above 0.49. Combined with the data in Table 1, this demonstrates that the model achieved optimal convergence and segmentation accuracy in complex breast ultrasound image segmentation tasks.

[0167] Step 5: Separate the application from the output.

[0168] The application process of integrating the trained, optimal CSWin-Unet model into a clinical auxiliary diagnostic system is as follows:

[0169] Input: The system receives the breast ultrasound image to be analyzed, using 500×500 pixels.

[0170] Segmentation Inference: After undergoing the same standardized preprocessing, the images are input into the CSWin-Unet model. The model completes forward propagation within ≤0.5 seconds and outputs a pixel-level tumor probability map.

[0171] Results Generation and Output: The system automatically generates two parts of results for doctors' diagnostic reference:

[0172] Visualized segmentation map: The tumor region predicted by the model is segmented using a green pseudo-color mask.

[0173] Quantitative analysis report: Automatically calculates morphological parameters of the tumor region, including area, maximum diameter, average gray value, etc., and generates a structured report.

[0174] Figure 14 This diagram illustrates tumor region segmentation and quantitative morphological analysis. It showcases the segmentation performance of the CSWin-Unet model in different clinical scenarios and the post-processing optimization process, including comparative validation of three typical samples: normal breast tissue, benign tumors, and malignant tumors. As shown, each row from left to right represents: the original ultrasound image, the initial prediction result of the CSWin-Unet model, and the post-processing optimization result. The diagram demonstrates that the original ultrasound image clearly presents the original morphological features of the lesion; the model prediction results in the middle column show the CSWin-Unet model's localization performance in the initial inference stage, accurately identifying the tumor region and distinguishing it from surrounding glandular tissue; the post-processing optimization results in the right column show the final output after morphological operations such as denoising and boundary smoothing.

[0175] As can be seen from the comparison, the end-to-end segmentation process proposed in this invention effectively eliminates minor noise in the initial prediction, further smooths and optimizes the tumor boundary, resulting in post-processing optimization results that are superior to the initial prediction results in terms of contour integrity and edge smoothness, especially achieving precise repair of the invasive boundary of malignant tumors. Figure 14The superiority of the CSWin-Unet model combined with post-processing strategies was intuitively verified, providing clinicians with clear, accurate, and artifact-free standardized tumor region masks. Based on this optimized segmentation mask, the system can automatically extract and calculate morphological parameters such as tumor pixel area, actual physical area converted based on image scale, maximum diameter, and minimum diameter, generating a structured quantitative analysis report containing tumor area and maximum diameter. This provides objective and repeatable quantitative morphological parameters to support clinicians in assessing lesion size, determining tumor growth morphology, and assisting in the differentiation between benign and malignant tumors.

[0176] Figure 15 This is a visualization of the tumor region segmentation effect of the CSWin-Unet model. The figure shows the end-to-end segmentation effect of the CSWin-Unet model on the tumor region in breast ultrasound images, covering two typical clinical samples: benign tumors and malignant tumors. Each row from left to right is: the original ultrasound image, the overlay of the prediction results, and the pseudo-color segmentation mask.

[0177] The first column shows the original breast ultrasound images acquired clinically, clearly presenting the original morphology, boundary features, and tissue echoes of the lesions. The second column is an overlay of the prediction results. This invention uses green pseudo-color to overlay and mark the tumor areas predicted by the model on the original ultrasound images, intuitively showing the location, extent, and outline of the tumor. The third column is an independent pseudo-color segmentation mask image, with red as the background and green as the tumor area, clearly separating the pure segmentation results output by the model, making it easy for doctors to quickly read the complete morphology of the tumor.

[0178] As shown in the figure, the CSWin-Unet model can achieve complete and accurate contour wrapping for benign tumors with clear boundaries and regular shapes, without under-segmentation or over-segmentation. For malignant tumors with blurred boundaries, invasive growth, and complex shapes, it can still closely fit the actual lesion contour, accurately distinguishing tumors from normal glandular tissue, while avoiding false positives in normal tissue segmentation. The overall segmentation results are highly consistent with clinical annotations, fully validating the high segmentation accuracy, strong adaptability, and clinical applicability of the CSWin-Unet model in complex breast ultrasound image scenarios, providing reliable visualization support for subsequent quantitative tumor analysis and diagnosis.

[0179] Example 2 - Automatic segmentation system and storage medium for breast cancer ultrasound images

[0180] The present invention also provides an automatic segmentation system for ultrasound images of breast cancer, the architecture of which is as follows: Figure 16As shown, the system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the automatic segmentation method for breast cancer ultrasound images according to any one of the claims of this invention: the system receives the breast ultrasound image to be analyzed, inputs it into the optimal CSWin-Unet model after standardized preprocessing, completes forward inference and post-processing optimization within ≤0.5 seconds, and finally outputs a superimposed visualization of the original image and the segmented region, and a quantitative analysis report including the tumor area and maximum diameter, providing support for clinical diagnosis. This invention also provides a storage medium storing a computer program. When the computer program is executed by the processor, it implements the automatic segmentation method for breast cancer ultrasound images according to this invention. This computer program can be stored in various storage media such as hard disks, solid-state drives, and USB flash drives, enabling cross-device deployment and application of the method.

[0181] Furthermore, the processor is used to call the optimal CSWin-Unet model program stored in the memory to perform end-to-end segmentation inference and post-processing optimization on the input 500×500 pixel breast ultrasound image; the memory is used to store model weights, original image data and final segmentation results; the system finally outputs two types of standardized results: one is a visualization of the overlay of the original image and the segmented region, and the other is a quantitative analysis report containing the tumor area and maximum diameter, thereby realizing the automatic segmentation method for breast cancer ultrasound images described above.

[0182] The above embodiments demonstrate that, through systematic multi-level model design and verification, the present invention has successfully achieved high-precision, high-efficiency, and high-robust segmentation of breast ultrasound images, especially complex malignant tumors, and has significant clinical application value.

[0183] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. An automatic segmentation method for breast cancer ultrasound images, characterized in that, The method includes the following steps: Step 1: Obtain a breast ultrasound image dataset and preprocess the dataset to construct high-quality samples for model training. Step 2: Under a unified training framework, construct and jointly train a set of neural network models, which includes a first model, a second model, and a third model. The first model is a U-Net architecture integrating an attention gating module, used to focus on key features of the tumor region through an attention mechanism. The second model is a U-Net architecture incorporating a cross-shaped window Transformer module, used to collaboratively extract local details and global contextual features through a cross-shaped window attention mechanism. The third model is a U-Net architecture integrating residual convolution and a parallel Swing Transformer module, used to balance local texture and global semantic information through parallel feature extraction paths. Step 3: Use a multi-dimensional evaluation system based on 5-fold cross-validation to evaluate the set of models trained in Step 2, and select the optimal segmentation model based on the evaluation results. Step 4: Perform end-to-end tumor region segmentation on the input breast ultrasound image using the optimal segmentation model, and output visualization results and quantitative reports.

2. The method according to claim 1, characterized in that, Step 1 specifically includes: Step 11: Obtain breast ultrasound images covering benign, normal, and malignant tissue types to achieve comprehensive coverage of lesion samples with different morphologies and characteristics; Step 12: Multiple manually annotated ultrasound images of the same type are fused using a pixel-level voting fusion algorithm to generate a single baseline ground truth image, in order to ensure the consistency and accuracy of the annotations. Step 13: Perform pixel value normalization and denoising on all images, and perform data augmentation by at least one of random flipping, rotation, scaling and adding noise.

3. The method according to claim 1 or 2, characterized in that, In step 2, the attention gating module in the first model takes as input the first feature map output by the encoder and the second feature map from the previous layer of the decoder. The attention gating module generates spatially adaptive weights for the first feature map, and the weighted output feature is defined by the following formula: in, These are the weight parameters of the convolutional layer. For bias parameters, Represents the ReLU activation function. The encoder's unpooled output features The output features of the previous layer of the decoder. The function maps the attention coefficients to the interval [0,1]. This refers to the output characteristics of the attention-gated AG module.

4. The method according to claim 1, characterized in that, In step 2, the cross-shaped window Transformer module in the second model processes the feature map output from the i-th layer of the encoder. The process is defined by the following formula: in, This represents the processing functions of the CSWin Transformer module; This represents the feature map output by the i-th layer of the encoder. For the set of real numbers, superscript The height of the output feature map of the i-th layer. Let be the width of the output feature map of the i-th layer. The number of channels in the output feature map of the i-th layer; This means splitting the input feature map into a series of non-overlapping local windows; This represents a dynamic window resizing strategy that adaptively adjusts the window size based on the variance of the spatial eigenvalues ​​within the window. Indicates the first The cross-shaped window attention mechanism performed by the group features calculates attention weights in the horizontal and vertical directions to capture long-range dependencies in the image. This indicates a splicing operation along the channel dimension; This represents the total number of attention groups; This represents the index of the current group, used to iterate from 1 to... All groups; for Convolution is used to fuse concatenated features after parallel grouping and processing, outputting an enhanced feature map. .

5. The method according to claim 1, characterized in that, In step 2, the residual convolution and the Swing Transformer parallel module in the third model process the input feature map. Simultaneously, local features are extracted through the residual convolution submodule. And extracting global features through the Swing Transformer submodule. And through learnable weighted parameters and Perform fusion and output features .

6. The method according to claim 1, characterized in that, The training framework in step 2 employs the AdamW optimizer and applies a cosine annealing learning rate decreasing strategy to adjust the learning rate. Simultaneously, a hybrid loss function is used as the training objective. This hybrid loss function is a weighted sum of cross-entropy loss and Dice loss, and its expression is: Where Loss is a hybrid loss function, CrossEntropyLoss is the cross-entropy loss used to optimize pixel-level classification accuracy, and DiceLoss is the Dice loss used to optimize the overlap of segmented regions.

7. The method according to claim 1 or 6, characterized in that, The multi-dimensional evaluation system in step 3 includes the mean intersection-over-union ratio (MIoU), accuracy (Acc), Kappa coefficient, and Dice coefficient. The selection of the optimal model must meet the following core performance index thresholds: mean intersection-over-union ratio (MIoU) ≥ 0.8106, accuracy (Acc) ≥ 0.9591, Kappa coefficient ≥ 0.7767, and Dice coefficient ≥ 0.8883.

8. The method according to claim 1, characterized in that, The segmentation results output in step 4 include a visual segmentation map superimposed on the original image, as well as a quantitative analysis report containing the area and maximum diameter of the tumor region.

9. An automatic segmentation system for breast cancer ultrasound images, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor; when executed by the processor, the computer program implements the automatic segmentation method for breast cancer ultrasound images as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium includes a computer program that, when executed by a processor, implements the automatic segmentation method for breast cancer ultrasound images as described in any one of claims 1 to 8.

Images