A method and system for segmenting ultrasound images of breast cancer

Abstract

This invention provides a method and system for breast cancer ultrasound image segmentation, belonging to the field of image processing technology. The method includes: acquiring a raw breast ultrasound image; performing data augmentation processing on the raw breast ultrasound image; constructing a breast tumor segmentation model, wherein the breast tumor segmentation model includes an MGM encoder module, a multi-scale cross-attention module, an edge feature refinement module, and an EMCAD decoder module connected in sequence; inputting the data-augmented raw breast ultrasound image into the breast tumor segmentation model to train the breast tumor segmentation model; acquiring a real-time breast ultrasound image; inputting the real-time breast ultrasound image into the trained breast tumor segmentation model, and outputting the final segmented image. This invention relies on a multi-module collaborative architecture, integrates multi-scale semantic features, and accurately refines tumor edges, significantly improving the accuracy of tumor segmentation and edge localization.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for segmenting ultrasound images of breast cancer. Background Technology

[0002] Breast cancer poses a serious threat to women's health, and early diagnosis and precise intervention are key to reducing mortality. Ultrasound imaging, with its advantages of being non-invasive and low-cost, has become an important means of screening and diagnosing breast lesions. However, breast ultrasound images have inherent limitations, increasing the difficulty of tumor segmentation. Traditional manual segmentation methods are time-consuming, labor-intensive, and subject to subjective bias, making it difficult to meet the needs of large-scale clinical screening. Therefore, efficient and accurate automatic segmentation algorithms have become a research hotspot in the field of medical imaging.

[0003] Deep learning methods have been widely used in breast ultrasound image segmentation. Both convolutional neural network-based and Transformer-based methods have their advantages and disadvantages, and both are powerful segmentation tools in this field. Meanwhile, attention mechanisms have become a core method for improving segmentation accuracy and robustness. Related technologies integrate encoder features through different approaches, contributing to the accurate segmentation of breast ultrasound lesions.

[0004] However, existing methods have significant limitations. Attention-related models have low efficiency in aggregating multi-scale features and insufficient refinement of semantic information. Meanwhile, Transformer-related methods still have room for improvement in addressing the problem of blurred boundaries in ultrasound images. Furthermore, edge-aware methods often ignore the long-range dependencies between the object's interior and its boundaries, failing to adequately solve the problems of speckle noise and blurred boundaries in ultrasound images. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a breast cancer ultrasound image segmentation method that can solve the problems of low aggregation efficiency of attention-related models for multi-scale features and insufficient semantic information refinement in existing technologies. Meanwhile, Transformer-related methods still have room for improvement in addressing the problem of blurred boundaries in ultrasound images. Furthermore, edge-aware methods often ignore the long-range dependencies between the object's interior and its boundaries, failing to adequately solve the technical problems of speckle noise and blurred boundaries in ultrasound images.

[0006] A first aspect of this invention provides a method for segmenting ultrasound images of breast cancer, comprising:

[0007] S1: Acquire raw breast ultrasound images.

[0008] S2: Perform data enhancement processing on the original breast ultrasound images.

[0009] S3: Construct a breast tumor segmentation model, which includes an MGM encoder module, a multi-scale cross-attention module, an edge feature refinement module, and an EMCAD decoder module connected in sequence.

[0010] S4: Input the original breast ultrasound image after data augmentation into the breast tumor segmentation model to train the breast tumor segmentation model.

[0011] S5: Acquire real-time breast ultrasound images.

[0012] S6: Input real-time breast ultrasound images into the trained breast tumor segmentation model and output the final segmented image.

[0013] A second aspect of the present invention provides a breast cancer ultrasound image segmentation system, comprising: a processor and a memory.

[0014] The memory stores programs or instructions that can run on a processor, which, when executed by the processor, implement the steps of the breast cancer ultrasound image segmentation method of the first aspect.

[0015] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the breast cancer ultrasound image segmentation method of the first aspect.

[0016] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0017] In this embodiment of the invention, a multi-scale cross-attention module efficiently aggregates multi-scale features output by the encoder, suppresses redundant channel information, and reconstructs spatial information to model global dependencies. Simultaneously, a progressive design achieves deep integration of multi-scale features and precise refinement of semantic features, fundamentally improving the utilization efficiency of multi-scale features. Furthermore, an edge feature refinement module accurately extracts and enhances global and local boundary features, models long-range dependencies between the object's interior and boundaries, optimizes boundary feature representation while resisting speckle noise interference, effectively overcoming the limitations of single edge information and improving boundary ambiguity. Attached Figure Description

[0018] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0019] Figure 1This is a flowchart illustrating a breast cancer ultrasound image segmentation method provided in an embodiment of the present invention.

[0020] Figure 2 This is an overview diagram of the breast tumor segmentation model provided in an embodiment of the present invention.

[0021] Figure 3 This is an overview diagram of the multi-scale cross-attention module provided in the embodiments of the present invention.

[0022] Figure 4 This is an overview diagram of the edge feature refinement module provided in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of a breast cancer ultrasound image segmentation system provided in an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] The breast cancer ultrasound image segmentation method provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0026] Reference manual attached Figure 1 The diagram illustrates a flowchart of a breast cancer ultrasound image segmentation method provided by an embodiment of the present invention.

[0027] This invention provides a method for segmenting ultrasound images of breast cancer, which may include the following steps:

[0028] S1: Acquire raw breast ultrasound images.

[0029] S2: Perform data enhancement processing on the original breast ultrasound image to obtain an enhanced image.

[0030] In one possible implementation, S2 specifically refers to:

[0031] The original breast ultrasound image was sequentially subjected to gamma transform contrast adjustment, brightness shift adjustment, and Gaussian blur processing to obtain an enhanced image.

[0032] For example, gamma values ​​of 0.6, 0.8, and 0.9, offset values ​​of +20, +50, and -10, and blur kernels of 3×3, 5×5, and 7×7 are used to obtain a total of 9 enhanced images.

[0033] It should be noted that performing gamma transform contrast adjustment, brightness offset adjustment, and Gaussian blur processing sequentially on the original breast ultrasound image to obtain an enhanced image can specifically improve the problems of uneven contrast and brightness deviation in ultrasound images. At the same time, it can effectively suppress the inherent speckle noise of the image, highlight the boundary difference between the tumor area and normal tissue, provide high-quality image input for subsequent feature extraction and tumor segmentation, and improve the accuracy and stability of the entire segmentation process.

[0034] Reference manual attached Figure 2 The diagram illustrates a flowchart of a breast cancer ultrasound image segmentation method provided by an embodiment of the present invention.

[0035] exist Figure 2 In this process, the input is a breast ultrasound image, which is first processed by Canny edge detection to generate a primary boundary feature map B1. The Canny edge detection algorithm is used to extract the primary boundary features. The image is simultaneously input into the MGM encoder, which generates E1-E4 (four-level feature maps) at layers 1, 2, 3, and 4. These features are processed in two paths: one path enters the MSCA (Multi-scale Cross Attention module, containing Efficient Channel Attention ECA, SPP pyramid pooling unit, and Cross Attention unit) to obtain feature e4. The other path, along with the primary boundary feature map B1 (also known as the edge enhancement image in the diagram), is fed into the EFRM (Edge Feature Refinement module, containing AGB gated attention unit and Conv convolution unit) to obtain boundary features b4 and boundary feature map bf (also known as the boundary segmentation image in the diagram). After e4 and b4 are concatenated into C via the concatenation channel, they are input into the first, second, third, and fourth layers (D1-D4) of the EMCAD decoder. The training is optimized by combining boundary loss, and finally the tumor segmentation mask is output.

[0036] S3: Construct a breast tumor segmentation model, which includes an MGM encoder module, a multi-scale cross-attention module, an edge feature refinement module, and an EMCAD decoder module connected in sequence.

[0037] The MGM encoder module (Multi-scale Gradient-guided Mix Encoder) is a backbone feature extraction encoder used to extract four levels of multi-scale feature maps (E1, E2, E3, E4) from breast ultrasound images, ranging from shallow to deep and from detail to semantics. This backbone encoding network provides the foundational features for subsequent MSCA multi-scale cross-attention and EFRM edge feature refinement. In this embodiment, a breast tumor segmentation model is constructed, forming a complete chain of multi-scale feature extraction, cross-scale semantic aggregation, edge feature refinement, and accurate decoding and segmentation. Each module has a clear division of labor and complements each other, which not only strengthens the feature expression of global semantics and local edges, but also effectively solves the pain points of blurred ultrasound image boundaries and noise interference, significantly improving the accuracy and robustness of breast tumor segmentation.

[0038] S4: Input the original breast ultrasound image after data augmentation into the breast tumor segmentation model to train the breast tumor segmentation model.

[0039] In one possible implementation, S4 specifically involves: inputting the original breast ultrasound image after data augmentation into the breast tumor segmentation model, training the breast tumor segmentation model until the value of the hybrid loss function of the breast tumor segmentation model is less than the preset loss function value.

[0040] Furthermore, the hybrid loss functions specifically include: the binary cross-entropy loss function and the Dice loss function.

[0041] Specifically, the formula for the hybrid loss function is as follows:

[0042] ;

[0043] in, L BCE Represents the binary cross-entropy loss. N Indicates the total number of pixels. The summation symbol is used to represent the summation symbol. i Indicates the first i 1 pixel, Y Indicates the true label, This represents the model's predicted value. Y i Represents the actual tumor pixels, This represents the loss term for the actual tumor pixels. This represents the loss term for the real background pixels, and log represents the natural logarithm function. L Dice ∩ represents the Dice loss, ∩ represents the intersection symbol, and ∪ represents the union symbol. L HybridThis represents the mixed loss function.

[0044] In this embodiment of the invention, the augmented breast ultrasound image is input into the tumor segmentation model for training. This ensures that the training data is of uniform specifications, prompts the various modules of the model to collaboratively optimize parameters, learn the semantic and edge feature patterns of breast tumors, effectively reduces the interference caused by image noise and boundary blurring, improves the model's accuracy in identifying and segmenting tumor regions, and enhances the model's generalization ability.

[0045] S5: Acquire real-time breast ultrasound images.

[0046] Reference manual attached Figure 3 The diagram shows an overview of the multi-scale cross-attention module provided in an embodiment of the present invention.

[0047] exist Figure 3 In the middle, the module input is Four levels of feature maps, let's start with Each process is handled by an ECA (Efficient Channel Attention Unit). It also needs to be processed by SPP (Symbol Pooling Unit), and then these features and The ECA processing results are then fed into the Cross Attention unit. ECA processing results ( ) as Q (query vector), SPP processing results ( ) as K (key vector), The vector itself is V (value vector). Q is multiplied by K and then subjected to Softmax to obtain the attention weights, which are then multiplied by V to obtain the attention-weighted features. ).final and After two splicing steps ("C" in the diagram), the integrated output features are obtained. .

[0048] Reference manual attached Figure 4 The diagram shows an overview of the edge feature refining module provided in an embodiment of the present invention.

[0049] exist Figure 4 In the upper-level process, the input includes B1 (primary boundary feature map). (Encoder layer feature map) and the output of Conv1x1 (1×1 convolution), these signals are processed sequentially through multiple layers of AGB, each AGB layer combines the corresponding layer features (E1 to E4), and the final processed features are concatenated ("C" in the figure) to output bf and b4. The lower layer is the specific logic of AGB: input G (gating signal) and X (feature signal), the two are processed and added, and then the attention weights are obtained by passing through ReLU (linear rectified activation function) and Sigmoid (sigmoid activation function), and then multiplied by X to output attention-weighted feature A.

[0050] S6: Input real-time breast ultrasound images into the trained breast tumor segmentation model and output the final segmented image.

[0051] In one possible implementation, S6 specifically includes sub-steps S601 to S605:

[0052] S601: Through the MGM encoder module, multi-scale features of the original breast ultrasound image after data enhancement are extracted layer by layer to obtain multi-level multi-scale feature maps.

[0053] It should be noted that by sampling and extracting multi-scale features layer by layer through the MGM encoder module, it is possible to capture different scale information of breast ultrasound images from superficial details to deep semantics, covering the feature expression of tumors of different sizes. This provides a high-quality, hierarchical feature foundation for feature aggregation in the subsequent multi-scale cross-attention module and boundary extraction in the edge feature refinement module, thereby improving the feature perception capability of the overall segmentation model.

[0054] For example, taking a breast ultrasound image with an input size of 256×256×3 (3 being the number of RGB channels), the output features maps at four levels are: Level 1: 64×64, 120 channels; Level 2: 32×32, 240 channels; Level 3: 16×16, 480 channels; Level 4: 8×8, 960 channels. The number of Transformer Block stacks (i.e., the depth of each stage) in the MGM module is [8,8,12,8]; the hidden layer dimension expansion ratio of the MLP (feedforward network) in each stage of the MGM module is [2,2,4,4]; and the probability of Drop Path (random depth) in the MGM module is 0.5. Simultaneously, the feature maps at all four levels of the encoder are output to the EFRM and MSCA, while the feature maps at levels 1-3 are also used as skip connection inputs to the decoder.

[0055] S602: The multi-scale cross attention module processes the multi-scale feature maps at each level to obtain the semantic information map. The multi-scale cross attention module specifically includes: efficient channel attention unit (ECA), pyramid pooling unit (SPP), cross attention unit (Cross Attention, CA) and channel splicing adjustment unit.

[0056] Among them, the Multi-Scale Cross Attention Block (MSCA) is a core component in the breast tumor segmentation model used for cross-scale feature interaction and semantic enhancement. Specifically, it refers to a neural network module that integrates multi-scale feature extraction and cross attention mechanisms.

[0057] In one possible implementation, S602 specifically includes sub-steps S6021 to S6029:

[0058] S6021: Enhances multi-scale feature maps at various levels through efficient channel attention units.

[0059] It should be noted that by enhancing the multi-scale feature maps at each level through efficient channel attention units, weights can be adaptively assigned to different channel features, strengthening key semantic channel information related to breast tumors, suppressing interference from background noise and redundant channels, increasing the proportion of effective information in the feature maps, and laying a precise channel feature foundation for subsequent cross-attention cross-scale feature aggregation.

[0060] S6022: The enhanced first-level feature map is processed by three-way pooling through the pyramid pooling unit.

[0061] It should be noted that by performing three-way pooling on the enhanced first-level feature map through pyramid pooling units, local details and global contextual information of different receptive fields can be obtained, adapting to the feature expression of breast tumor lesions of different sizes, enriching the semantic hierarchy of the feature map, and providing multi-dimensional feature support for subsequent cross-scale feature interaction.

[0062] S6023: The enhanced second-level feature map is subjected to two-way pooling processing through the pyramid pooling unit.

[0063] It should be noted that by performing two-way pooling on the enhanced second-level feature map through pyramid pooling units, it is possible to adapt to the semantic depth of the features at this level, accurately capture the local details and mid-range contextual information at the corresponding scale, enhance the semantic expressive power of the features while avoiding redundant computation, and provide more adaptable feature support for cross-scale feature interaction.

[0064] S6024: The enhanced third-level feature map is processed by one-way pooling through the pyramid pooling unit.

[0065] It should be noted that performing one-way pooling on the enhanced third-level feature map through the pyramid pooling unit is in line with the characteristics of deeper semantics and wider receptive field of this level of features. This can maximize the preservation of global semantic information of deep tumors, avoid feature redundancy and information loss caused by multi-way pooling, and provide accurate and complete deep semantic support for cross-scale feature aggregation.

[0066] S6025: Using bilinear interpolation, each pooled feature map is downsampled to the size of the enhanced fourth-level feature map to obtain the first pooled feature map, the second pooled feature map, and the third pooled feature map.

[0067] It should be noted that by downsampling the feature maps of each pooling level to the fourth level size through bilinear interpolation, the size specifications of feature maps at different scales can be accurately unified, eliminating the scale barrier of cross-scale feature interaction. At the same time, bilinear interpolation can preserve the tumor semantics and detailed information in the feature maps to the maximum extent, avoid feature distortion during the downsampling process, and lay a foundation for consistent specifications for subsequent feature aggregation.

[0068] S6026: Cross-attention is calculated using the cross-attention unit based on the first pooling feature map, the second pooling feature map, the third pooling feature map, and the enhanced fourth-level feature map.

[0069] Specifically, the formula for calculating cross-attention is:

[0070] ;

[0071] in, Attention Indicates cross attention. Q Represents the query vector. K Indicates key, V Represents value, K T express K transpose, d k The dimension represents the common features of the query vector and the key, and softmax represents the activation function.

[0072] It should be noted that by calculating cross-attention between the multi-scale pooling feature map and the fourth-level enhanced feature map through cross-attention units, semantic associations between features of different scales can be established, cross-scale dependencies of breast tumor features can be captured, the expression of key tumor semantic information can be strengthened, background noise interference can be suppressed, and high-quality feature support can be provided for subsequent semantic information map generation.

[0073] S6027: Generate first attention-weighted features, second attention-weighted features, and third attention-weighted features based on the cross-attention unit.

[0074] It should be noted that generating three types of attention-weighted features based on cross-attention units can accurately map attention weights to corresponding level features, amplify key semantic information of breast tumors, highlight the advantages of features at different scales, provide differentiated high-quality features for subsequent channel splicing and adjustment, and improve the accuracy of semantic information graphs.

[0075] S6028: Through the channel splicing and adjustment unit, the first attention-weighted features, the second attention-weighted features, the third attention-weighted features, and the enhanced fourth-level feature map are spliced ​​and adjusted in the channel dimension to output refined semantic features.

[0076] It should be noted that by integrating and adapting the channel dimensions of multiple features through the channel splicing and adjustment unit, semantic information of breast tumors at different scales can be aggregated, eliminating dimensional differences between features and forming refined semantic features with uniform specifications. This provides high-quality and consistent input for subsequent modules, improving the semantic modeling efficiency and accuracy of the overall model.

[0077] S6029: Obtain a semantic information graph based on the refined semantic features.

[0078] It should be noted that the semantic information map obtained from the refined semantic features can accurately present the multi-scale semantic contour of breast tumors, filter out irrelevant background interference, provide a clear semantic reference for the subsequent edge feature refinement module, ensure the complementary integration of semantic and boundary features, and help improve the accuracy and completeness of the final tumor segmentation.

[0079] Specifically, the breast tumor segmentation model adopts a U-Net-like architecture, with the core difference being the introduction of the MSCA and EFRM modules in the bottleneck layer. First, the encoder extracts four sets of multi-scale features (F1, F2, F3, F4) from the input. Then, the MSCA module generates refined semantic features e4, and the EFRM module generates refined boundary features b4. Finally, in the decoder, the semantically richest feature e4 and the most refined boundary feature b4 are concatenated on the channel and fed into the decoder layer D4. F1, F2, and F3 serve as skip connections corresponding to the decoder layers D1, D2, and D3. This allows the decoder to start decoding from features richer in both semantic and boundary information, guided by shallow information, which directs skip connections to enhance decoding, thereby improving segmentation accuracy.

[0080] For example, the MSCA module takes four layers of feature maps from the encoder as input. First, it processes the data using ECA modules with parameters k_size of 5, 7, 9, and 11 respectively to enhance key channels and suppress redundant channels. Then, the first layer of feature maps after ECA enhancement undergoes three-way pooling (2x2, 4x4, 8x8), the second layer undergoes two-way pooling (2x2, 4x4), and the third layer undergoes one-way pooling (2x2). The processed features are then downsampled to 8×8 (the same size as the encoder's four-layer feature map) using bilinear interpolation. Finally, the three pooled feature maps (denoted as K and V) are used as keys (K) and values ​​(V) in Cross Attention, and the fourth feature map is used as the query vector in Cross Attention to calculate cross-attention.

[0081] In this embodiment of the invention, relying on the multi-unit collaborative effect of the multi-scale cross-attention module, key features are enhanced through efficient channel attention, multi-receptive field information is obtained through pyramid pooling, and cross-scale associations are established through cross-attention. Then, features are adjusted and integrated through channel splicing. This allows for the accurate extraction and refinement of multi-scale semantic information of breast tumors, effectively suppressing background noise and redundant interference, providing high-quality semantic references for subsequent boundary feature processing, and improving the feature utilization efficiency and semantic modeling accuracy of the overall segmentation model. The three enhanced feature maps obtained through cross-attention calculation are then spliced ​​with a fourth feature map on the channel to obtain the final output. Additionally, the semantic information map (8×8×960) output by MSCA and the 4-channel boundary information map (8×8×4) output by EFRM are spliced ​​to obtain a spliced ​​feature map (8×8×964), which serves as the starting point for the decoder input.

[0082] S603: The edge feature refinement module processes the multi-scale feature maps at various levels to obtain the boundary information map. The edge feature refinement module specifically includes: Canny edge detection algorithm, convolution unit, gated attention unit, bilinear interpolation unit and channel splicing unit.

[0083] The Edge Feature Refine Module (EFRM) is a key component in the breast tumor segmentation model used for edge feature extraction, enhancement, and refinement. Specifically, it refers to a neural network module that integrates multi-scale edge perception and cross-module feature feedback.

[0084] In one possible implementation, S603 specifically includes sub-steps S6031 to S6035:

[0085] S6031: The original breast ultrasound image after data enhancement is processed using the Canny edge detection algorithm to obtain a primary boundary feature map.

[0086] It should be noted that by using the Canny edge detection algorithm to process the augmented breast ultrasound images, the initial edge contours of tumor tissue can be accurately captured, the inherent speckle noise interference of ultrasound images can be effectively suppressed, and a clear primary boundary feature map can be generated, providing a reliable and accurate basis for the subsequent refinement and enhancement of boundary features.

[0087] S6032: Through convolutional units, multi-scale feature maps at various levels are compressed to generate multiple single-channel edge confidence maps corresponding to the multi-scale feature maps at various levels.

[0088] Specifically, the formula for calculating multiple single-channel edge confidence maps is as follows:

[0089] ;

[0090] in, This represents the edge confidence map of the first single channel. This represents the edge confidence map of the second single channel. This represents the edge confidence map of the third single-channel image. This represents the fourth single-channel edge confidence map. R Represents the real number field. H Indicates the height of the multi-scale feature map. W This represents the width of the multi-scale feature map.

[0091] Specifically, attention gate units (AGB) are used to adaptively adjust the weights of Canny edge features, incorporating boundary details while preventing edge information from obscuring semantic features. Specifically, F1', F2', F3', and F4' are used as gate signals G to focus on the boundary regions of the corresponding levels. The primary contour B1 is used as the feature signal X, which also serves as a boundary prior for this boundary refinement module, analyzing and activating contextual boundary information to guide the module to focus on true boundary information rather than noise.

[0092] It should be noted that by compressing the multi-scale feature maps at each level through convolutional units, the feature dimension can be reduced to reduce the subsequent computational load. At the same time, it focuses on and enhances the tumor edge feature response at different scales. The generated single-channel edge confidence map can provide accurate and focused edge feature input for the gating attention unit, improving the targeting and efficiency of boundary information extraction.

[0093] S6033: Through the gated attention unit, multiple boundary information corresponding to each single-channel edge confidence map is generated using each single-channel edge confidence map as a gate signal and the primary boundary feature map as a feature signal.

[0094] ;

[0095] in, A Represents boundary information, σ 2 represents the Sigmoid activation function. σ 1 represents the ReLU activation function. Represents the weight matrix of the convolutional unit. G Indicates a gating signal. X Represents the characteristic signal, Representing characteristic signals X The convolution weights, Indicates gating signal G The convolution weights.

[0096] It should be noted that by using a gated attention unit with a single-channel edge confidence map as the gating signal and a primary boundary feature map as the feature signal, it can adaptively filter and activate real tumor boundary features, suppress background pseudo-edge interference, and accurately enhance effective boundary information at different scales, providing highly reliable support for subsequent boundary feature integration.

[0097] S6034: By using bilinear interpolation units, all boundary information is unified to the same preset size.

[0098] It should be noted that by using bilinear interpolation units to unify the boundary information to the same preset size, the scale difference of boundary features at different levels can be eliminated, breaking down the barrier to cross-scale feature integration. At the same time, the detailed information of the tumor boundary is preserved to the maximum extent, avoiding feature distortion caused by size adjustment, and providing consistent, accurate and reliable basic data for subsequent boundary information fusion processing.

[0099] S6035: By using the channel splicing unit, the boundary information after being standardized in size is merged to obtain the boundary information map.

[0100] It should be noted that by merging boundary information of uniform size through channel splicing units, effective tumor boundary features at different scales can be integrated, making up for the limitations of single-scale boundary extraction, strengthening the continuity and integrity of the boundary, and gathering multi-dimensional boundary details to provide a comprehensive and accurate reference for subsequent semantic and boundary feature fusion, thereby improving the boundary localization accuracy of the final tumor segmentation.

[0101] In this embodiment of the invention, relying on the multi-unit collaboration of the edge feature refinement module, Canny is used to extract the initial boundary, convolutional units compress and enhance features, gated attention adaptively filters effective boundaries, bilinear interpolation unifies the scale to avoid distortion, and channel stitching integrates multi-dimensional information. This can filter out false edges in the background of ultrasound images, accurately preserve tumor boundary details, improve the integrity and accuracy of the boundary information map, and provide a highly reliable reference for subsequent semantic and boundary feature fusion.

[0102] For example, the first step is the generation of the primary boundary feature map. The existing Canny edge detection algorithm is used to generate a primary boundary feature map from the input 256×256×3 breast ultrasound image. The Canny algorithm parameters are set as follows: Gaussian filter kernel size 3*3, low threshold: 10, high threshold: 100. Subsequently, this primary boundary feature map is downsampled through a 1*1 convolution to match the feature map size of each of the four encoder layers, while maintaining a channel count of 1, resulting in a primary boundary feature map adapted for the subsequent AGB module. The second step is the specific implementation of the gated attention module (AGB): the AGB module takes the downsampled primary boundary feature map and the four encoder layers as input to generate four boundary information points. The specific process is as follows: The feature maps of encoder layers 1 (64×64×120), 2 (32×32×240), 3 (16×16×480), and 4 (8×8×960) are first processed through 1×1 convolutional layers to halve the number of channels in each layer (e.g., layer 1 from 120 to 60, layer 2 from 240 to 120), and then compressed to 1 channel. The four layer maps compressed to 1 channel are then used as G, the primary boundary feature map as X, and the output is A.

[0103] S604: Concatenate the semantic information map and the boundary information map through channels to obtain the concatenated feature map.

[0104] In this embodiment of the invention, relying on the multi-unit collaborative operation of the edge feature refinement module, the initial boundary is extracted through Canny detection, the convolutional unit compresses and enhances the features, and the gating attention filters the effective boundary signals. Then, the size is unified by bilinear interpolation and the multi-scale information is integrated by channel splicing. This can accurately capture the true boundary contour of breast tumors, effectively suppress background pseudo-edges and noise interference, enhance the continuity and integrity of the boundary, provide a high-quality reference for the subsequent fusion of semantics and boundary features, and significantly improve the boundary localization accuracy of the final tumor segmentation.

[0105] S605: Decodes the stitched feature map using the EMCAD decoder and outputs the final segmented image.

[0106] In one possible implementation, S605 specifically includes sub-steps S6051 to S6054:

[0107] S6051: By initializing the EMCAD decoder, the stitched feature map is input into the fourth layer of the decoder to build the decoding infrastructure.

[0108] It should be noted that by initializing the EMCAD decoder and inputting the stitched feature map into its fourth layer to build the basic architecture, it can prioritize the acceptance of high-dimensional features that integrate semantic and boundary information, laying a stable and high-quality decoding starting point, avoiding shallow features from interfering with deep feature parsing, providing reliable support for subsequent layer-by-layer upsampling and feature refinement, and improving the accuracy of tumor segmentation.

[0109] S6052: Through skip connection fusion technology, the third-level feature map is connected to the third layer of the decoder in sequence, the second-level feature map is connected to the second layer of the decoder, and the first-level feature map is connected to the first layer of the decoder.

[0110] It should be noted that by using the skip connection fusion technique to connect the feature maps of each level of the encoder to the corresponding layers of the decoder, it can effectively compensate for the feature loss of the decoder upsampling layer by layer, accurately transmit the shallow detailed features and deep semantic features of the encoder to the decoder, enhance the expressive power of each layer of features, and improve the detail integrity and boundary accuracy of the tumor segmentation results.

[0111] S6053: Upsamples the feature maps of each level to the size of the original breast ultrasound image by using bilinear interpolation upsampling technology with a segmentation head.

[0112] It should be noted that by using bilinear interpolation upsampling technology in the segmentation head to restore the feature maps of each level to the original image size, the spatial location information of the feature maps can be accurately restored, the tumor boundary and detailed features can be preserved to the maximum extent, feature distortion during the upsampling process can be avoided, and the output features can correspond one-to-one with the pixel positions of the original ultrasound image, laying a key foundation for the final generation of a high-precision tumor segmentation mask.

[0113] S6054: Output the final segmented image based on the upsampled hierarchical feature map.

[0114] It should be noted that outputting the final segmented image based on the upsampled hierarchical feature map can fully integrate the advantages of the previous multi-scale semantic and boundary features, accurately restore the true shape and location of the tumor in the original ultrasound image, eliminate information loss in the feature processing process, and ensure that the regional integrity and boundary accuracy of the segmentation result meet the application requirements, thereby improving the practicality and reliability of the overall technical solution.

[0115] For example, using existing EMCAD decoder technology, the input starting point is the concatenated feature map of MSCA and EFRM (8×8×964), which is then skipped to the encoder's layer 1 (64×64×120), layer 2 (32×32×240), and layer 3 (16×16×480) feature maps. Finally, the segmentation head uses bilinear interpolation to upsample to the original size (256*256) to generate a 256×256 binary segmentation map (pixel value 0 represents the background, and 1 represents the tumor region).

[0116] In one possible implementation, the process after S6 includes:

[0117] The model performance is comprehensively evaluated using the following metrics: Precision, Recall, Overall Accuracy (OA), Kappa coefficient, Mean Intersection over Union (mIoU), and Average Symmetric Surface Distance (ASSD). OA measures the proportion of correctly classified pixels out of the total pixels; Kappa coefficient assesses the consistency between the predicted result and the ground truth label; Precision represents the proportion of true positives out of all predicted positives; Recall reflects the proportion of true positives out of all actual positives; mIoU measures the overlap between the prediction and the ground truth; and ASSD represents the average symmetric distance between the predicted boundary and the ground truth boundary. The values ​​of Precision, Recall, OA, Kappa, and mIoU range from 0 to 1, with higher values ​​indicating better performance. ASSD is always greater than 0, with lower values ​​indicating better performance.

[0118] In this embodiment of the invention, real-time breast ultrasound images are input into a trained breast tumor segmentation model and the final segmented image is output. This fully leverages the advantages of multi-scale semantic extraction and boundary refinement in model integration, quickly and accurately restoring the true shape and location of the tumor, effectively improving the regional integrity and boundary accuracy of the segmentation results, meeting the application needs of real-time clinical diagnosis, and reducing the errors and time consumption of manual segmentation.

[0119] The breast cancer ultrasound image segmentation method provided in this application can be executed by a breast cancer ultrasound image segmentation device. This application uses a breast cancer ultrasound image segmentation device to execute the breast cancer ultrasound image segmentation method as an example to illustrate the breast cancer ultrasound image segmentation device provided in this application.

[0120] Reference manual attached Figure 5 The diagram shows a schematic of the structure of a breast cancer ultrasound image segmentation system provided in an embodiment of the present invention.

[0121] This invention provides a breast cancer ultrasound image segmentation system 20, including: a processor 201 and a memory 202;

[0122] The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the breast cancer ultrasound image segmentation method described above and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0123] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0124] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0125] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0126] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] This invention provides a readable storage medium that stores a program or instructions on the storage medium. When the program or instructions are executed by a processor, they implement the steps of the breast cancer ultrasound image segmentation method described above and achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for segmenting ultrasound images of breast cancer, characterized in that, include: S1: Acquire raw breast ultrasound images; S2: Perform data enhancement processing on the original breast ultrasound image to obtain an enhanced image; S3: Construct a breast tumor segmentation model, wherein the breast tumor segmentation model includes an MGM encoder module, a multi-scale cross-attention module, an edge feature refinement module, and an EMCAD decoder module connected in sequence; S4: Input the original breast ultrasound image after data augmentation into the breast tumor segmentation model to train the breast tumor segmentation model; S5: Acquire real-time breast ultrasound images; S6: Input the real-time breast ultrasound image into the trained breast tumor segmentation model and output the final segmentation image; S6 specifically includes: S601: Through the MGM encoder module, multi-scale features of the original breast ultrasound image after data enhancement are extracted layer by layer to obtain multi-level multi-scale feature maps. S602: The multi-scale feature maps of each level are processed by the multi-scale cross-attention module to obtain a semantic information map. The multi-scale cross-attention module specifically includes: an efficient channel attention unit, a pyramid pooling unit, a cross-attention unit, and a channel splicing and adjustment unit. S603: The edge feature refinement module processes the multi-scale feature maps of each level to obtain a boundary information map. The edge feature refinement module specifically includes: Canny edge detection algorithm, convolution unit, gated attention unit, bilinear interpolation unit and channel splicing unit. S604: Concatenate the semantic information map and the boundary information map through channels to obtain a concatenated feature map; S605: The stitched feature map is decoded by the EMCAD decoder to output the final segmented image.

2. The breast cancer ultrasound image segmentation method according to claim 1, characterized in that, Specifically, S2 involves sequentially performing gamma transformation contrast adjustment, brightness shift adjustment, and Gaussian blur processing on the original breast ultrasound image to obtain an enhanced image.

3. The breast cancer ultrasound image segmentation method according to claim 1, characterized in that, S4 specifically involves: inputting the original breast ultrasound image after data augmentation into the breast tumor segmentation model, training the breast tumor segmentation model until the value of the hybrid loss function of the breast tumor segmentation model is less than the preset loss function value.

4. The breast cancer ultrasound image segmentation method according to claim 3, characterized in that, The hybrid loss function specifically includes: the binary cross-entropy loss function and the Dice loss function.

5. The breast cancer ultrasound image segmentation method according to claim 1, characterized in that, The hierarchical multi-scale feature map specifically includes: a first-level feature map, a second-level feature map, a third-level feature map, and a fourth-level feature map; Specifically, S602 includes: S6021: Enhance the multi-scale feature maps of each level through the efficient channel attention unit; S6022: The enhanced first-level feature map is subjected to three-way pooling processing through the pyramid pooling unit; S6023: The enhanced second-level feature map is subjected to two-way pooling processing through the pyramid pooling unit; S6024: The enhanced third-level feature map is processed by one-way pooling through the pyramid pooling unit; S6025: Using the bilinear interpolation unit, each pooled layer feature map is downsampled to the size of the enhanced fourth layer feature map to obtain the first pooled feature map, the second pooled feature map, and the third pooled feature map. S6026: The cross-attention unit calculates cross-attention based on the first pooling feature map, the second pooling feature map, the third pooling feature map, and the enhanced fourth-level feature map. S6027: Generate a first attention-weighted feature, a second attention-weighted feature, and a third attention-weighted feature based on the cross-attention unit; S6028: Through the channel splicing and adjustment unit, the first attention-weighted feature, the second attention-weighted feature, the third attention-weighted feature, and the enhanced fourth-level feature map are spliced ​​and adjusted in the channel dimension to output refined semantic features; S6029: Based on the refined semantic features, the semantic information graph is obtained.

6. The breast cancer ultrasound image segmentation method according to claim 1, characterized in that, Specifically, S603 includes: S6031: The original breast ultrasound image after data enhancement is processed by the Canny edge detection algorithm to obtain a primary boundary feature map; S6032: The convolutional unit compresses the multi-scale feature maps of each level to generate multiple single-channel edge confidence maps corresponding to each multi-scale feature map of each level. S6033: Through the gated attention unit, using each of the single-channel edge confidence maps as gate signals and the primary boundary feature map as feature signals, multiple boundary information corresponding to each of the single-channel edge confidence maps is generated; S6034: The bilinear interpolation unit is used to unify all the boundary information to the same preset size; S6035: The boundary information after being unified in size is merged through the channel splicing unit to obtain the boundary information map.

7. The breast cancer ultrasound image segmentation method according to claim 1, characterized in that, Specifically, S605 includes: S6051: By initializing the EMCAD decoder, the stitched feature map is input into the fourth layer of the decoder to build the decoding infrastructure; S6052: Through skip connection fusion technology, the third-level feature map is connected to the third layer of the decoder in sequence, the second-level feature map is connected to the second layer of the decoder, and the first-level feature map is connected to the first layer of the decoder. S6053: Upsample each of the aforementioned hierarchical feature maps to the size of the original breast ultrasound image using bilinear interpolation upsampling technology with a segmentation head; S6054: Output the final segmented image based on the upsampled hierarchical feature map.

8. A breast cancer ultrasound image segmentation system, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the breast cancer ultrasound image segmentation method as described in any one of claims 1 to 7.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the breast cancer ultrasound image segmentation method as described in any one of claims 1 to 7.

Images