Breast tumor image multi-task learning method, system, device and medium

Abstract

The invention discloses a breast tumor image multi-task learning method, system, equipment and medium, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining a breast cancer ultrasonic image, inputting the breast cancer ultrasonic image into a pre-trained improved multi-task network, carrying out the feature extraction and multiple times of down-sampling operation, obtaining a plurality of different scale features, and obtaining a plurality of different scale features; the features of the minimum scale are processed through an HC module, and bottleneck features are obtained; enhancing the bottleneck feature and the feature of the penultimate scale in a decoder, and fusing the bottleneck feature and the feature of the penultimate scale to obtain an initial fusion feature and a classification result; through a top-down path, the initial fusion feature is subjected to up-sampling and then is subjected to iterative fusion with other features with different scales and enhanced features, and the final fusion feature is mapped into a single-channel segmentation probability graph. According to the multi-source input design, the classifier can comprehensively utilize information of different abstraction levels to obtain more accurate judgment, and an efficient and accurate solution is provided for automatic auxiliary processing of breast cancer by combining the cooperative processing process of two tasks.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a multi-task learning method, system, device and medium for breast tumor images. Background Technology

[0002] Breast cancer is one of the most common malignant tumors among women worldwide, and early and accurate diagnosis is crucial for improving patient survival rates. Breast ultrasound, due to its advantages of being radiation-free, low-cost, and real-time, has become the preferred imaging method for breast disease screening and diagnosis. However, the interpretation of breast ultrasound images highly depends on the professional experience of the radiologist, and the images themselves often suffer from problems such as speckle noise, artifacts, low contrast, and blurred lesion boundaries. In particular, malignant tumors often present with irregular shapes, spiculated edges, and angular protrusions, while benign tumors have relatively regular shapes. These characteristics make manual delineation and diagnosis time-consuming, laborious, and subject to subjective differences.

[0003] In recent years, deep learning techniques, represented by CNNs, especially variants based on U-Net networks, have achieved great success in medical image segmentation and classification tasks. These methods effectively fuse deep semantic information and shallow spatial information through encoder-decoder structures and skip connections. However, existing methods still face the following challenges when processing breast ultrasound images: traditional methods often treat segmentation and classification as two independent tasks, ignoring the inherent connection between them. Furthermore, simple feature-concatenated skip connections may lead to a semantic gap between the encoder and decoder, resulting in insufficient fusion of low-level spatial features and high-level semantic features, affecting segmentation details. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the prior art by providing a multi-task learning method, system, device, and medium for breast tumor images, thereby solving the problems of the prior art.

[0005] This invention specifically provides the following technical solution: a multi-task learning method for breast tumor images, comprising the following steps: Acquiring ultrasound images of breast cancer; The breast cancer ultrasound image is input into the pre-trained improved multi-task network; wherein the improved multi-task network includes an encoder-decoder network with ResNet-18 as the backbone, and a hybrid convolution HC module set at the encoder output, a triaxial coordinate and channel collaborative attention TACCSA module set in the decoder, a gated dynamic feature fusion G-DFF module and a hybrid convolution HC module. The encoder performs feature extraction and multiple downsampling operations on breast cancer ultrasound images to obtain features at different scales. The smallest scale feature is then processed by the HC module to obtain the bottleneck feature. In the decoder, the bottleneck features and the penultimate scale features are enhanced by the TACCSA module in three dimensions: channel, height, and width. The two enhanced features are then fused by the G-DFF module to obtain the initial fused features. Finally, the HC module performs attention weighting and multi-level fusion on the bottleneck features, the penultimate scale features, and the initial fused features to obtain the classification result. In the decoder, the initial fused features are upsampled and then iteratively fused with other features of different scales and enhanced by the TACCSA module through a top-down path, and the final fused features are mapped to a single-channel segmentation probability map.

[0006] Preferably, the step of extracting features from breast cancer ultrasound images using an encoder and performing multiple downsampling operations to obtain features at different scales specifically involves: Feature extraction is performed on ultrasound images of breast cancer, and the extracted features are then downsampled. The downsampled features are passed sequentially through four residual stages in the encoder: layer 1, layer 2, layer 3, and layer 4. In each stage, the spatial resolution is halved while the number of channels increases, ultimately generating features at five scales.

[0007] Preferably, the bottleneck features are obtained by processing the minimum-scale features through the HC module, specifically as follows: Features at a single scale are used as input to the HC module, and two different feature maps are generated through multiple parallel processing paths. The two feature maps are multiplied element-wise using dynamic weights, and the weighted feature maps are summed to obtain the final output of the module, which serves as the bottleneck feature.

[0008] Preferably, the bottleneck features and the penultimate scale features are enhanced using the TACCSA module in three dimensions: channel, height, and width. Specifically: The bottleneck features and the features at the penultimate scale are used as inputs to the TACCSA module, respectively. X-axis coordinate attention is used to generate a direction-aware feature map of size C×H×1 by performing one-dimensional average pooling along the width direction to preserve the positional information in the height direction. Y-axis coordinate attention uses one-dimensional average pooling along the height direction to preserve positional information in the width direction, generating a direction-aware feature map of size C×1×W. The channel attention branch first obtains the global channel description vector through global average pooling, and then explicitly models the interdependencies between channels through two layers of 1×1 convolution to generate channel attention weights. The spatial attention map of the coordinate attention branch and the channel weights of the channel attention branch are multiplied element-wise and then applied together to the original feature map to synergistically enhance the spatial location and channel information, resulting in enhanced bottleneck features and penultimate scale features.

[0009] Preferably, the two enhanced features are fused using a G-DFF module to obtain initial fused features, specifically as follows: The two enhanced features are then input into the G-DFF module. By dynamically adjusting the information flow through two parallel branches—spatial attention and channel-gated attention—in the G-DFF module, the two features are fused to obtain the initial fused features.

[0010] Preferably, the step of inputting breast cancer ultrasound images into a pre-trained improved multi-task network includes: The tasks in the improved multi-task network are divided into segmentation tasks and classification tasks; For segmentation tasks, a linear combination of Dice loss and binary cross-entropy loss is used as the segmentation loss; for classification tasks, cross-entropy loss is used as the classification loss. By dynamically weighting segmentation loss and classification loss, a dynamic weighted loss function based on task uncertainty is obtained. This dynamic weighted loss function is then used to pre-train the improved multi-task network, resulting in the pre-trained improved multi-task network. Preferably, the dynamic weighted loss function based on task uncertainty is obtained by dynamically weighting the segmentation loss and classification loss, specifically expressed as follows: ; in, To divide the loss, For classifying losses, and These are the learnable parameters associated with segmentation and classification tasks, respectively. and These are learnable parameters that are optimized through backpropagation, as part of the model, along with the network weights. This is a dynamically weighted loss function based on task uncertainty.

[0011] This invention provides a multi-task learning system for breast tumor images, comprising: The image acquisition module is used to acquire ultrasound images of breast cancer. The data input module is used to input breast cancer ultrasound images into a pre-trained improved multi-task network; wherein the improved multi-task network includes an encoder-decoder network with ResNet-18 as the backbone, and a hybrid convolution HC module set at the encoder output, a triaxial coordinate and channel collaborative attention TACCSA module, a gated dynamic feature fusion G-DFF module and a hybrid convolution HC module set in the decoder. The feature acquisition module is used to extract features from breast cancer ultrasound images through the encoder and perform multiple downsampling operations to obtain features at multiple different scales. The smallest scale features are then processed by the HC module to obtain the bottleneck features. The classification module is used in the decoder to enhance the bottleneck features and the penultimate scale features through the TACCSA module in three dimensions: channel, height, and width. The two enhanced features are then fused through the G-DFF module to obtain the initial fused features. Finally, the HC module performs attention weighting and multi-level fusion on the bottleneck features, the penultimate scale features, and the initial fused features to obtain the classification result. The mapping module is used in the decoder to iteratively fuse the initial fused features after upsampling with other features of different scales and enhanced by the TACCSA module through a top-down path, and to map the final fused features into a single-channel segmentation probability map.

[0012] The present invention provides a computer device, including a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor performs the steps of the above-described multi-task learning method for breast tumor images.

[0013] The present invention provides a storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the above-described multi-task learning method for breast tumor images.

[0014] Compared with the prior art, the present invention has the following significant advantages: This invention improves the multi-task network by enhancing bottleneck features from the encoder using the TACCSA module to highlight key information. The G-DFF module efficiently fuses the enhanced bottleneck features with features at the penultimate scale, and the HC module performs deep fusion of the fused features. This enhances the representation ability of irregular shapes and complex textures, adaptively balancing the weights of shallow fine textures and deep abstract semantic information at different spatial locations. This significantly improves the representation ability of irregular shapes and complex textures, effectively reducing the semantic gap caused by direct addition or simple concatenation, thus achieving classification. Simultaneously, by improving the classification... The initial fusion features used in this study are upsampled and used as the initial input. Through a top-down path, the upsampled initial fusion features are iteratively fused with features of different scales and enhanced by the TACCSA module. The final fusion features are then mapped to a single-channel segmentation probability map. Using the fusion features encoded during classification as the initial input for segmentation, multi-task collaborative learning of segmentation and classification is achieved. The multi-source input design enables the classifier to comprehensively utilize information at different levels of abstraction to obtain more accurate judgments. Furthermore, the combination of the collaborative processing of the two tasks provides an efficient and accurate solution for automated assisted processing of breast cancer. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the complete process of the multi-task deep learning method based on breast ultrasound images proposed in this invention; Figure 2 This is a schematic diagram of the TACCSA module in this invention; Figure 3 This is a schematic diagram of the structure of the G-DFF module in this invention; Figure 4 This is a schematic diagram of the HC module in this invention; Figure 5 This is a schematic diagram of the structure of the multi-scale feature extraction module in this invention; Figure 6 This is a schematic diagram of the segmentation results of the method of the present invention on a breast tumor sample; wherein, Figure 6 Image (a) is a sample of a breast tumor. Figure 6 (b) is a segmentation result of a breast tumor. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] The core idea of ​​this invention is to construct a multi-task network based on an encoder-decoder architecture, possessing hierarchical feature enhancement and refinement capabilities. This network uses a lightweight ResNet-18 as the encoder backbone to efficiently extract hierarchical features from images. The innovations of this invention are mainly reflected in the following three aspects:

[0018] The decoder employs a three-stage process of "enhancement-fusion-refinement": At each level of the decoder, this invention utilizes a highly structured processing flow. First, a TACCSA module is designed to pre-enhance the skip features from the encoder; second, a G-DFF module is designed to fuse the enhanced skip features with the upsampled features; finally, an HC module is designed to deeply refine the transformed features, specifically enhancing the representation capabilities for irregular shapes and complex textures. This series of operations ensures that features are optimally processed at each layer.

[0019] Multi-source input, multi-level fusion classification head: This invention designs a unique parallel classification head that does not only use features from the deepest layers of the network, but aggregates features from three sources: the higher layers of the encoder, the bottleneck layer of the network, and the shallower layers of the decoder. This multi-source input design enables the classifier to comprehensively utilize information from different levels of abstraction to obtain more accurate judgments.

[0020] Multi-task adaptive learning framework: This invention places segmentation and classification tasks within a unified framework for end-to-end training. Specifically, it employs an automatically weighted loss function, enabling the network to automatically learn the optimal weights for both tasks during training, dynamically balancing their importance and achieving mutual promotion between the two tasks. This avoids tedious manual parameter tuning and achieves effective collaborative promotion between tasks.

[0021] For ease of understanding, the English abbreviations and technical terms appearing in this invention are explained as follows: TACCSA: The full English name is tri-axis coordinate and channel synergy attention. This invention is simply referred to as the tri-axis coordinate and channel synergy attention module.

[0022] HC: The full English name is hybrid convolution, and this invention is simply referred to as the hybrid convolution module.

[0023] G-DFF: The full English name is gated dynamic feature fusion, and this invention is simply referred to as the gated dynamic feature fusion module.

[0024] BUSI stands for breast ultrasound images, and in this invention, it is simply referred to as breast ultrasound images.

[0025] CNN stands for convolutional neural network, and in this invention it is simply referred to as a convolutional neural network.

[0026] Jaccard: The full English name is Jaccard similarity coefficient, which is abbreviated as Jaccard similarity coefficient in this invention. It is used to measure the similarity between the predicted result and the ground truth annotation, with a value ranging from 0 to 1. The larger the value, the closer the segmentation result is to the ground truth region.

[0027] Dice: The full name is Dice similarity coefficient, which is used to evaluate the degree of overlap between the predicted result and the ground truth annotation. A higher value indicates a higher degree of similarity between the predicted and ground truth regions.

[0028] 95HD: The full English name is 95th percentile of the Hausdorff Distance. In this invention, it is simply referred to as the 95th percentile of the Hausdorff distance. It is used to measure the stability of the maximum distance between the predicted boundary and the true boundary. The 95th percentile is used to reduce the impact of extreme errors; the smaller the value, the closer the boundary is to the actual boundary.

[0029] ACC stands for Accuracy, which is used to measure the overall proportion of correct predictions. It ranges from 0 to 1, with a higher value indicating better overall prediction performance.

[0030] PRE stands for Precision, which is abbreviated as Precision Rate in this invention. It represents the proportion of samples that were predicted to be positive but were actually positive. The higher the value, the fewer incorrect positive examples there are in the prediction results.

[0031] REC stands for Recall, which is abbreviated as Recall Rate in this invention. It represents the proportion of samples that are actually positive but are correctly predicted as positive. The higher the value, the fewer positive examples the model misses.

[0032] like Figure 1 As shown, an embodiment of the present invention provides a multi-task learning method for breast tumor images, comprising the following steps: Step S1: Obtain ultrasound images of breast cancer.

[0033] Step S2: Input the breast cancer ultrasound image into the pre-trained improved multi-task network; wherein the improved multi-task network includes an encoder-decoder network with ResNet-18 as the backbone, and a hybrid convolution HC module set at the encoder output, a triaxial coordinate and channel collaborative attention TACCSA module set in the decoder, a gated dynamic feature fusion G-DFF module and a hybrid convolution HC module.

[0034] Network Structure: First, the original tumor images undergo a series of data augmentation techniques for preprocessing to improve the model's robustness and generalization ability. An improved multi-task network is then used to extract features from breast cancer ultrasound images, ultimately obtaining a segmentation probability map. Specifically, the improved multi-task network is as follows:

[0035] The lightweight ResNet-18 is used as the encoder-decoder network backbone.

[0036] Configure the TACCSA module, G-DFF module, and HC module in the encoder-decoder network.

[0037] The preprocessed image is fed into an encoder-decoder network with ResNet-18 as the backbone.

[0038] Step S3: The encoder performs feature extraction and multiple downsampling operations on the breast cancer ultrasound image to obtain features at different scales. The smallest scale feature is then processed by the HC module to obtain the bottleneck feature.

[0039] Feature extraction was performed on breast cancer ultrasound images, and multiple downsampling was conducted to obtain feature maps at different scales. Specifically: Feature extraction was performed on ultrasound images of breast cancer, and the extracted features were then downsampled.

[0040] The downsampled feature map is passed through four residual stages: layer 1, layer 2, layer 3, and layer 4. In each stage, the spatial resolution is halved while the number of channels increases, ultimately generating feature maps at five scales.

[0041] like Figure 5As shown, in the encoder stage, the input image first passes through layer 0, which consists of an initial convolution, normalization, activation function, and max pooling layer, downsampling the feature map to 128×128, reducing the number of channels to 64. Subsequently, the feature map passes through four residual stages: layers 1, 2, 3, and 4. With each stage, the spatial resolution is halved while the number of channels increases, ultimately generating feature maps at five scales: layer 0 (64 channels, 128×128), layer 1 (64 channels, 64×64), layer 2 (128 channels, 32×32), layer 3 (256 channels, 16×16), and layer 4 (512 channels, 8×8). Before entering the decoder, the output of layer 4 of the encoder undergoes feature refinement through an HC module, resulting in enhanced bottleneck features.

[0042] After generating feature maps at five scales, the process also includes: Features at a single scale are used as input to the HC module, and two different feature maps are generated through three parallel processing paths.

[0043] The two feature maps are multiplied element-wise using dynamic weights, and the weighted feature maps are summed to obtain the final output of the module, which serves as the bottleneck feature.

[0044] Step S4: In the decoder, the bottleneck features and the penultimate scale features are enhanced using the TACCSA module in three dimensions: channel, height, and width. The two enhanced features are then fused using the G-DFF module to obtain the initial fused features. Finally, the bottleneck features, the penultimate scale features, and the initial fused features are weighted by attention and fused at multiple levels using the HC module to obtain the classification result.

[0045] The bottleneck features and the penultimate scale features are enhanced using the TACCSA module in three dimensions: channel, height, and width. Specifically: The bottleneck features and the features at the penultimate scale are used as inputs to the TACCSA module. The X-axis coordinate attention is used to perform one-dimensional average pooling along the width direction to preserve the positional information in the height direction, generating a direction-aware feature map of size C×H×1.

[0046] The Y-axis coordinate attention method uses one-dimensional average pooling along the height direction to preserve positional information in the width direction, generating a direction-aware feature map of size C×1×W.

[0047] The channel attention branch first obtains the global channel description vector through global average pooling, and then explicitly models the interdependencies between channels through two layers of 1×1 convolution to generate channel attention weights.

[0048] The spatial attention map of the coordinate attention branch and the channel weights of the channel attention branch are multiplied element-wise and then applied together to the original feature map to synergistically enhance the spatial location and channel information, resulting in enhanced bottleneck features and penultimate scale features.

[0049] The decoder stage employs a three-step process of "enhancement-fusion-refinement," gradually restoring resolution from the bottom up. In the first decoding layer, the bottleneck feature is first enhanced by the TACCSA module, then upsampled by a factor of 2 and the number of channels adjusted to 256 by the upsampling module; simultaneously, the skip connection feature from the encoder (layer 3) is also enhanced by a dedicated TACCSA module. The two enhanced features are input into the G-DFF module, where the information flow is dynamically adjusted through two parallel branches: spatial attention and channel-gated attention, fusing the two features to obtain an initial fused feature of 256 channels. This feature is then further refined by convolutional block 2 and the DFDC module. Subsequently, a parallel classification head is activated, receiving information from layer 3, the bottleneck feature, and the initial fused feature. Internally, it performs attention weighting and multi-level fusion through the CA module, ultimately outputting the benign / malignant classification result.

[0050] Step S5: In the decoder, the initial fused features are upsampled and then iteratively fused with features of different scales and enhanced by the TACCSA module through a top-down path, and the final fused features are mapped to a single-channel segmentation probability map.

[0051] In subsequent decoding processes, the workflow remains consistent: the second decoding layer upsamples the initial fused features to 128 channels and feeds them together with the TACCSA-enhanced Layer 2 features into G-DFF2 for fusion, followed by convolutional block 3 and DFDC refinement to obtain the 128-channel second fused feature; the third decoding layer upsamples the second fused feature to 64 channels and feeds it together with the TACCSA-enhanced Layer 1 features into G-DFF3 for fusion, followed by convolutional block 4 and DFDC refinement to obtain the 64-channel third fused feature; the fourth decoding layer upsamples the third fused feature to 64 channels and feeds it together with the TACCSA-enhanced Layer 0 features into G-DFF4 for fusion, followed by convolutional block 5 and DFDC refinement to obtain the final fourth fused feature. Finally, the fourth fused feature is used by the segmentation head module (which includes one upsampling and a 1×1 convolution) to map the feature map into a single-channel segmentation probability map, thus completing end-to-end prediction from the input image to the segmentation mask.

[0052] Breast tumor data: The BUSI dataset contains 780 ultrasound images and corresponding segmentation maps of breast cancer from 600 female patients, including 210 cases of malignant tumors, 437 cases of benign tumors, and 133 normal controls. In this study, since the experimental task focused on the segmentation and classification of benign and malignant tumors, the 133 normal control images without lesions were removed, resulting in a final selection of 647 images containing both benign and malignant lesions. To comprehensively and fairly evaluate model performance, this invention employs a five-fold cross-validation method to randomly partition and experiment with these 647 images.

[0053] Data augmentation and input size normalization: The training phase employs image augmentation techniques, including random flipping and random rotation: First, the input image and its corresponding label image are scaled to a preset size of 256×256; then, the image is randomly flipped horizontally with a 60% probability; next, the image is randomly rotated within a range of ±30°; then, the image is randomly flipped vertically with a 60% probability; finally, tensor quantization and normalization are performed on the augmented samples, where the normalization mean is [0.330, 0.330, 0.330] and the standard deviation is [0.204, 0.204, 0.204].

[0054] The image preprocessing in the verification stage includes scaling the input image to a preset size of 256×256 and performing tensor quantization and normalization operations, where the normalization mean is [0.330, 0.330, 0.330] and the standard deviation is [0.204, 0.204, 0.204].

[0055] Attention module that combines three-axis coordinates and channels: like Figure 2 As shown, the TACCSA module aims to refine feature representations in breast ultrasound image analysis by collaboratively modeling information in three dimensions: channel, height, and width. Its framework is as follows: Figure 2As shown, this module is deployed in the decoder path to enhance features during the upsampling stage, thereby improving the model's ability to capture key diagnostic information. Specifically, TACCSA contains three parallel attention branches. The upper part is a coordinate attention branch, including X-axis coordinate attention and Y-axis coordinate attention. X-axis coordinate attention uses one-dimensional average pooling along the width direction to preserve positional information in the height direction, generating a direction-aware feature map of size C×H×1 (C represents the number of channels, H represents the height, and 1 represents a width of 1, i.e., a single-column feature map used to encode directional information only in the vertical direction). Y-axis coordinate attention uses one-dimensional average pooling along the height direction to preserve positional information in the width direction, generating a direction-aware feature map of size C×1×W (C represents the number of channels, 1 represents a height of 1, i.e., a single-row feature map, and W represents the width used to encode directional information only in the horizontal direction). These two directional features can capture long-distance spatial dependencies with direction awareness, playing a crucial role in accurately depicting lesions with varied morphologies and blurred boundaries in breast ultrasound images. Meanwhile, the lower-level channel attention branch first obtains the global channel description vector through global average pooling, and then explicitly models the interdependencies between channels through two 1×1 convolutions (equivalent to a double fully connected layer), generating channel attention weights to enhance the feature responses that are diagnostically valuable in distinguishing between benign and malignant lesions. In the collaborative reweighting stage, the spatial attention map from the coordinate attention branch and the channel weights from the channel attention branch are multiplied element-wise and then applied to the original feature map to achieve synergistic enhancement of spatial location and channel information. In this way, TACCSA retains the sensitivity of convolutional neural networks to local patterns while compensating for their shortcomings in long-distance dependency modeling. It can effectively suppress the interference of background breast tissue and artifacts, highlight key details related to tumor boundaries and internal echo patterns, thereby significantly improving the model's segmentation and classification performance on breast ultrasound datasets and demonstrating higher robustness and generalization ability in real clinical scenarios.

[0056] Gated dynamic feature fusion module: like Figure 3As shown, the G-DFF module is a dual-gated structure designed to effectively fuse shallow features from the encoder and deep features from the decoder, particularly suitable for breast ultrasound image segmentation tasks that require simultaneous preservation of boundary details and enhancement of semantic information. This module receives upsampled features *x* from the decoder and enhanced skip features *skip* from the encoder as input, and dynamically adjusts the information flow through two parallel branches: spatial attention and channel-gated attention. In the spatial attention branch, *x* and *skip* are convolved and then summed, generating a spatial attention map using a sigmoid activation function. This attention map identifies and emphasizes spatially consistent regions between the two features, such as the core location of the lesion, thereby guiding the module to strengthen responses in these regions. Simultaneously, the channel-gated attention branch concatenates *x* and *skip*, then extracts global context and salient feature information through parallel global average pooling and global max pooling branches, and combines multi-layer convolution and gating mechanisms to generate a channel attention map to characterize the correlation and importance between feature channels. Finally, the module multiplies the attention signals from both spatial correspondence and channel context dimensions, and then multiplies them element-wise with the features from the skip convolution to achieve dual modulation fusion. This is then integrated through the output convolutional layer to generate the final fused feature map. This process enables the G-DFF module to adaptively balance the weights of shallow fine textures and deep abstract semantic information at different spatial locations, effectively reducing the semantic gap caused by direct addition or simple concatenation. This significantly improves the segmentation accuracy of breast lesion boundaries and the model's robustness in complex ultrasound scenarios.

[0057] Hybrid Convolutional Module: like Figure 4 As shown, the HC module is an adaptive feature refining unit designed for accurately capturing the complex features of breast tumors, particularly suitable for handling the challenges of irregular lesion morphology and diverse internal textures. This module receives a single feature map as input and intelligently fuses two complementary feature representations through three parallel processing paths: a deformable attention path, a frequency-dynamic convolution path, and a gated generation path. In the deformable large-kernel attention path, the input features are processed to generate a feature map. This path leverages the geometric adaptability of deformable convolution and the broad receptive field of large-size convolutional kernels, enabling it to flexibly capture the non-rigid, irregular contours of tumors and their long-distance spatial dependencies. This is crucial for accurately depicting morphological features such as edge spurs and angular protrusions. Simultaneously, in the frequency dynamic convolution path, the input features are processed to generate feature maps. This approach focuses on analyzing features from the frequency domain, making it particularly sensitive to information such as image texture, periodic structure, and high-frequency details. By dynamically generating convolutional kernels, it can adaptively extract subtle texture differences related to the benign or malignant nature of lesions, effectively compensating for the shortcomings of traditional spatial convolution in detail representation.

[0058] The core adaptive fusion mechanism is driven by a gated generation path. This path first performs global average pooling on the input features, compressing spatial information into a channel descriptor to capture the global context. Subsequently, this descriptor undergoes a nonlinear transformation through a lightweight network consisting of two convolutional layers and a ReLU activation function, and finally generates a scalar gating value between 0 and 1 through a sigmoid activation function. This gate value and their complementary values Used as dynamic weights, respectively with features and Element-wise multiplication is performed. Finally, the two weighted feature maps are added together to obtain the module's final output. In this way, the HC module can intelligently determine, based on the global information of the input features, whether the current region requires more attention to its macroscopic morphology (increasing...). (weight) or microtexture (increase) (Weights). This adaptive weighted fusion of complementary features enables the model to generate more discriminative and robust feature representations for complex lesions, thereby significantly improving the accuracy of segmentation tasks.

[0059] Model training: This invention designs an end-to-end multi-task deep learning network framework and employs a five-fold cross-validation method to evaluate the model's segmentation and classification performance. The entire dataset is divided into five non-overlapping subsets. In each fold experiment, four subsets are selected as training data, and the remaining subset is used as test data. This process is repeated five times to ensure that each subset is used as a test set once. For each fold training, the model starts from scratch and is trained for a total of 100 epochs.

[0060] The tasks in the improved multi-task network are divided into segmentation and classification tasks. To optimize both tasks simultaneously, a combined loss function is defined. First, for the segmentation task, Dice loss is used. With binary cross-entropy loss A linear combination of (BCE Loss). Dice loss effectively addresses the imbalance in the number of pixels between tumor and background, directly optimizing the overlap of segmented regions; while BCE loss focuses on the classification accuracy of each pixel. Segmentation Loss The specific form is:

[0061] ; Secondly, for classification tasks, the standard cross-entropy loss is used. As the classification loss, this loss function is a commonly used metric to measure the difference between the probability distribution predicted by the model and the probability distribution of the true labels in multi-class classification problems. For a single sample, its calculation formula is as follows:

[0062] ; in, It is the total number of categories (in this invention, (corresponding to benign and malignant) It is a symbolic function (when category) The value is 1 if it is a real label, otherwise it is 0. The model predicts that the sample belongs to a category. The probability of.

[0063] Finally, to address the challenge of balancing the loss weights of segmentation and classification tasks in multi-task learning and avoiding the dominance of a single task in the optimization process, a dynamic weighted loss function based on task uncertainty is obtained by dynamically weighting the segmentation and classification losses. This method abandons the static weighting strategy that requires manual parameter tuning and introduces two learnable parameters associated with the segmentation and classification tasks. and This automatically adjusts the contribution of each task to the total loss. These two parameters intuitively represent the uncertainty of the model's prediction for each task, allowing the model to dynamically assign smaller weights to tasks with higher uncertainty (i.e., greater learning difficulty or higher loss values) and larger weights to tasks with lower uncertainty during training. Optimized total loss function. The definition is as follows:

[0064] ; in, and These are learnable parameters that are optimized along with the network weights through backpropagation as part of the model.

[0065] This invention employs the Adam optimizer to update all learnable parameters of the network (including network weights and the weights of the loss function). , The Adam optimizer combines the advantages of momentum and adaptive learning rates, achieving efficient and stable convergence. The initial learning rate is set to 1e-4, and the weight decay is also set to 1e-4 to prevent overfitting. LambdaLR is used as the learning rate scheduler. Throughout the training process, the learning rate is dynamically adjusted using a cosine annealing strategy. This strategy allows the learning rate to decrease smoothly from its initial value as the training cycle increases, helping the model converge to the optimal solution better in the later stages of training.

[0066] Model evaluation: To monitor the model's training progress, prevent overfitting, and select the optimal model for final testing, this invention employs a complete validation and model selection process. During each training cycle, the model undergoes a performance evaluation on an independent validation set after each training period. In the validation phase, the model switches to evaluation mode, without gradient calculation or parameter updates. To comprehensively evaluate the model's performance on both tasks, a weighted comprehensive performance metric, Score, is defined. This metric combines the core metrics of the segmentation task (Dice coefficient) and the core metrics of the classification task (accuracy). Its calculation formula is as follows:

[0067] ; in, and These are preset weights used to balance the importance of the two tasks. In a specific implementation of this invention, the following settings are made: and This indicates a greater focus on the performance of the segmentation task. At the end of each epoch, the current model's score on the validation set is calculated. If the current score is higher than the highest score recorded in all previous epochs, the current model is considered the optimal model, and its state (i.e., model weights) is saved. This process continues throughout the entire training epoch, and the model saved at the end is the optimal model obtained in that epoch, which will be used in the final testing phase.

[0068] Results analysis: The proposed model was compared with several existing segmentation and classification models, as shown in Table 1. Experimental results show that the proposed model achieves better performance in both segmentation and classification tasks, with the segmentation results as follows: Figure 6 As shown in the table, traditional CNN methods have limited ability to capture irregular tumor morphology and long-range dependencies, making it difficult to accurately depict complex tumor boundaries. However, as can be seen from the table, the model proposed in this invention demonstrates superiority across all core metrics. In the 95HD and Dice metrics, which measure boundary fit, this model achieves significant reductions due to the refined "enhancement-fusion-refinement" design in the decoder, indicating that the segmentation boundaries more closely resemble reality. Simultaneously, thanks to the effective fusion of multi-source features by the classification head and the attention weighting of the internal channel attention module, the classification ACC metric is also significantly improved, reducing the risk of missed diagnoses and misdiagnoses.

[0069] Table 1. Performance comparison of the proposed method with existing segmentation and classification models. Based on the same inventive concept, this invention provides a multi-task learning system for breast tumor images, including: an image acquisition module, a data input module, a feature acquisition module, a classification module, and a mapping module.

[0070] The system comprises several modules: an image acquisition module for acquiring breast cancer ultrasound images; a data input module for inputting these images into a pre-trained improved multi-task network; the improved multi-task network includes an encoder-decoder network with a ResNet-18 backbone, a hybrid convolutional (HC) module at the encoder output, a triaxial coordinate and channel collaborative attention (TACCSA) module, a gated dynamic feature fusion (G-DFF) module, and a hybrid convolutional (HC) module in the decoder; and a feature acquisition module for performing feature extraction and multiple downsampling operations on the breast cancer ultrasound images through the encoder to obtain features at multiple different scales, and processing the smallest scale features through the HC module to obtain... The bottleneck feature and classification module are used in the decoder to enhance the bottleneck feature and the penultimate scale feature through the TACCSA module in three dimensions: channel, height, and width. The two enhanced features are then fused through the G-DFF module to obtain the initial fused feature. The HC module performs attention weighting and multi-level fusion on the bottleneck feature, the penultimate scale feature, and the initial fused feature to obtain the classification result. The mapping module is used in the decoder to iteratively fuse the initial fused feature, after upsampling, with other features of different scales that have been enhanced by the TACCSA module, through a top-down path. The final fused feature is then mapped to a single-channel segmentation probability map.

[0071] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a program, and when the program is executed by the processor, the processor performs the steps of a multi-task learning method for breast tumor images.

[0072] According to the disclosed embodiments, the computer device can communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth communication, etc.) or with any device that enables the computing device to communicate with one or more other computing devices (e.g., router, demodulator, etc.).

[0073] The present invention also provides a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of a multi-task learning method for breast tumor images.

[0074] According to the disclosed embodiments, the storage medium can be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0075] The above description, in conjunction with specific preferred embodiments, provides a more detailed explanation of the present invention. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection of the present invention.

Claims

1. A breast tumor image multi-task learning method, characterized in that, The method comprises the following steps: acquiring a breast cancer ultrasound image; inputting the breast cancer ultrasound image into a pre-trained improved multi-task network; wherein the improved multi-task network comprises an encoder-decoder network with ResNet-18 as the backbone, a hybrid convolution (HC) module arranged at the output end of the encoder, a three-axis coordinate and channel collaborative spatial attention (TACCSA) module arranged in the decoder, a gated dynamic feature fusion (G-DFF) module, and a HC module; feature extraction and multiple downsampling operations are performed on the breast cancer ultrasound image by the encoder to obtain multiple features of different scales, and the feature of the smallest scale is processed by the HC module to obtain a bottleneck feature; in the decoder, the bottleneck feature and the feature of the second smallest scale are enhanced in three dimensions of channel, height, and width by the TACCSA module, and the two features after the feature enhancement are fused by the G-DFF module to obtain an initial fusion feature, and the bottleneck feature, the feature of the second smallest scale, and the initial fusion feature are subjected to attention weighting and multi-level fusion by the HC module to obtain a classification result; in the decoder, the initial fusion feature is iteratively fused with other features of different scales after being enhanced by the TACCSA module through a top-down path, and the final fusion feature is mapped into a single-channel segmentation probability map.

2. The breast tumor image multi-task learning method of claim 1, wherein, the feature extraction and multiple downsampling operations performed on the breast cancer ultrasound image by the encoder to obtain multiple features of different scales are as follows: feature extraction is performed on the breast cancer ultrasound image, and the extracted features are downsampled; the downsampled features are sequentially passed through the first layer, the second layer, the third layer, and the fourth layer of the encoder, and after passing through each stage, the spatial resolution is halved and the number of channels is increased, finally generating five scale features.

3. The breast tumor image multi-task learning method of claim 2, wherein, the feature of the smallest scale is processed by the HC module to obtain a bottleneck feature, which is as follows: a single scale feature is taken as the input of the HC module, and two different feature maps are generated through multiple parallel processing paths; the two feature maps are multiplied element by element through dynamic weights, and the weighted feature maps are added to obtain the final output of the module, which is taken as the bottleneck feature.

4. The breast tumor image multi-task learning method of claim 1, wherein, the bottleneck feature and the feature of the second smallest scale are enhanced in three dimensions of channel, height, and width by the TACCSA module, which is as follows: the bottleneck feature and the feature of the second smallest scale are taken as the input of the TACCSA module; the X-axis coordinate attention generates a direction perception feature map with a size of C×H×1 by one-dimensional average pooling along the width direction to retain the position information in the height direction; the Y-axis coordinate attention generates a direction perception feature map with a size of C×1×W by one-dimensional average pooling along the height direction to retain the position information in the width direction; the channel attention branch first obtains a global channel description vector through global average pooling, and then models the interdependence between channels through two 1×1 convolution layers to generate channel attention weights; The space attention map of the coordinate attention branch and the channel weight of the channel attention branch are multiplied element by element to act on the original feature map, the spatial position and the channel information are synergistically enhanced, and the enhanced bottleneck feature and the feature of the second last scale are obtained.

5. The breast tumor image multi-task learning method of claim 1, wherein, The two features after feature enhancement are fused through a G-DFF module to obtain initial fusion features, specifically: The two features after feature enhancement are input into the G-DFF module; Through the dynamic adjustment of information flow by the space attention and the channel gated attention in the G-DFF module, the two features are fused to obtain the initial fusion features.

6. The breast tumor image multi-task learning method of claim 1, wherein, The breast cancer ultrasound image is input into the improved multi-task network after pre-training, and the pre-training includes: The tasks in the improved multi-task network are divided into segmentation tasks and classification tasks; For the segmentation task, a linear combination of the Dice loss and the binary cross-entropy loss is used as the segmentation loss; for the classification task, the cross-entropy loss is used as the classification loss; The segmentation loss and the classification loss are dynamically weighted to obtain a dynamic weighting loss function based on task uncertainty, and the improved multi-task network is pre-trained based on the dynamic weighting loss function based on task uncertainty to obtain the pre-trained improved multi-task network.

7. The breast tumor image multi-task learning method of claim 6, wherein, The dynamic weighting loss function based on task uncertainty is obtained by dynamically weighting the segmentation loss and the classification loss, and is specifically represented as: ； wherein, is a segmentation loss, is a classification loss, and are learnable parameters associated with the segmentation and classification tasks, respectively, and are learnable parameters that are optimized through backpropagation as part of the model along with the network weights, is a dynamically weighted loss function based on task uncertainty. 8.A breast tumor image multi-task learning system, characterized by, It includes: An image acquisition module for acquiring a breast cancer ultrasound image; A data input module for inputting the breast cancer ultrasound image into the improved multi-task network after pre-training; The improved multi-task network includes an encoder-decoder network with ResNet-18 as the backbone, a hybrid convolution HC module arranged at the output end of the encoder, a three-axis coordinate and channel synergistic attention TACCSA module arranged in the decoder, a gated dynamic feature fusion G-DFF module, and a hybrid convolution HC module; A feature acquisition module for feature extraction and multiple downsampling operations on the breast cancer ultrasound image through the encoder to obtain multiple features of different scales, and processing the smallest scale feature through the HC module to obtain a bottleneck feature; A classification module for, in the decoder, enhancing the bottleneck feature and the feature of the second last scale in three dimensions of channel, height and width through the TACCSA module, and fusing the two features after feature enhancement through the G-DFF module to obtain initial fusion features, and performing attention weighting and multi-level fusion on the bottleneck feature, the feature of the second last scale and the initial fusion features through the HC module to obtain a classification result; A mapping module for, in the decoder, iteratively fusing the initial fusion features after upsampling with other features of different scales enhanced by the TACCSA module through a top-down path, and mapping the final fusion features into a single-channel segmentation probability map.

9. A computer device, comprising: A memory and a processor, the memory stores a program, and the program is executed by the processor to make the processor execute the steps of the breast tumor image multi-task learning method according to any one of claims 1-7.

10. A storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the steps of the breast tumor image multi-task learning method according to any one of claims 1 to 7.

Images