Breast cancer image segmentation method based on feature disentanglement recombination and joint learning

Abstract

This invention discloses a breast cancer image segmentation method based on feature deentanglement and recombination and joint learning, comprising the following steps: S1, acquiring breast cancer image data and dividing it into training and test sets; S2, performing preprocessing; S3, extracting image features; S4, obtaining training foreground features and test foreground features and recombining them to obtain foreground alignment features; S5, generating tumor foreground prototypes based on foreground alignment features; generating training set background prototypes based on training background features, and splicing them to form a two-dimensional prototype set; S6, performing segmentation prediction, generating a bi-branch segmentation probability map and prediction mask, constructing a joint loss function, simultaneously optimizing segmentation performance, and promoting prototype alignment; S7, performing pixel-level classification on the test set segmentation probability map, outputting binary segmentation masks for tumor regions and normal tissues, achieving accurate localization of breast cancer lesions.

Description

Technical Field

[0001] This invention relates to the field of image segmentation, and more particularly to a breast cancer image segmentation method based on feature deentanglement and recombination and joint learning. Background Technology

[0002] Currently, in the field of breast cancer image segmentation technology, existing technologies still have many key limitations, making it difficult to meet the requirements of clinical precision segmentation for lesion localization accuracy, feature discrimination, and model generalization.

[0003] In terms of feature separation and purification, existing methods mostly directly use basic convolutional networks to extract global features of breast images, without designing a dedicated feature separation mechanism for the characteristics of breast cancer lesions and surrounding tissues. For breast ultrasound images, lesion edges are blurred, internal echoes are heterogeneous, and they are easily confused with dense glandular tissue. Existing technologies lack efficient feature purification strategies, making it difficult to accurately separate the tumor foreground from the background features of normal tissue, often leading to over-segmentation of lesion areas or omission of microinvasive lesions. Furthermore, when processing breast X-ray images, they do not fully focus on the semantic differences between key pathological features such as calcification clusters and spiculated edges and glandular structures, still using a generalized fixed feature extraction mode. This results in the core features of the tumor being masked by background noise, insufficient feature discrimination, and affects segmentation accuracy.

[0004] Regarding feature interaction and alignment, existing bi-branch or multi-branch segmentation models mostly employ simple feature stitching or one-way information transmission modes, failing to establish deep semantic relationships between different data subsets. For breast cancer images acquired from multiple centers, the heterogeneity of tumor feature distribution due to differences in equipment parameters and imaging positions means that existing methods lack effective cross-subset feature alignment mechanisms. This fails to enhance the feature consistency of similar tumors, resulting in poor model adaptability to tumor morphological variations. Furthermore, most methods neglect the mining of spatial correspondences between foreground features, failing to enhance the stability of core lesion features through feature interaction, further limiting the reliability of segmentation results.

[0005] In terms of prototype generation and training strategies, existing prototype generation mechanisms have significant shortcomings. They mostly generate category prototypes through global average pooling or single-dimensional feature aggregation, failing to fully utilize the potential auxiliary information of the background region in breast images and neglecting the diversity of tumor morphology. This results in a single semantic expression of the prototype, making it difficult to cover the feature differences of different pathological types of breast cancer. During training, most methods only optimize a single segmentation loss, lacking joint optimization of the foreground and background feature discrimination and failing to establish a two-way verification mechanism. This leads to weak segmentation ability of the model for tumor boundary details, making it difficult to meet the requirements of clinical diagnosis for the reliability and precision of segmentation results. Summary of the Invention

[0006] The purpose of this invention is to address the above-mentioned problems by providing a breast cancer image segmentation method based on feature deentanglement and recombination and joint learning, which enables precise localization of breast cancer lesions and provides intuitive and reliable image segmentation results for clinical diagnosis.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: A breast cancer image segmentation method based on feature deentanglement and recombination and joint learning includes the following steps: S1. Collect breast cancer imaging data. Professional doctors annotate the tumor regions of the breast cancer imaging data at the pixel level and generate corresponding GT masks. Combine the breast cancer imaging data with the GT masks to construct a dataset and divide the dataset into training set and test set. S2. Preprocess the breast cancer image data in the training set and the test set, and simultaneously retain the GT mask corresponding to the breast cancer image data in the training set. S3. Using a ResNet-101 backbone network with shared weights, image features of breast cancer image data in the training set are extracted. Imaging features of breast cancer imaging data in the test set ; S4. Extract image features from the training set. The foreground features are separated into training foreground features using the corresponding ground truth mask. Compared with training background features Image features in the test set Two sets of identically sized feature maps are generated by copying, and then initialized to form the test foreground features. With test background features Subsequently, the training foreground features were analyzed. Test foreground features Apply intra-branch self-attention separately, and then use a cross-attention mechanism to train foreground features. Test foreground features Recombination yields foreground alignment features; S5. Masked average pooling is used to process the foreground alignment features to generate a tumor foreground prototype. Training background features are processed through a fine-grained encoder. Extract potential auxiliary information and generate a training set background prototype. The prototype of the tumor prospect Training set background prototype splicing together to form a two-dimensional prototype set ; S6, Utilizing a Two-Dimensional Prototype Set It guides the segmentation prediction of the training set and the test set respectively, calculates the similarity between features and prototypes through non-parametric metric methods, and generates the segmentation probability map and prediction mask of the training set, the segmentation probability map and prediction mask of the test set, and the cross-entropy loss based on the prediction mask of the training set, the prediction mask of the test set and the GT mask, constructs a joint loss function, and simultaneously optimizes the segmentation performance of the training set and the test set to promote prototype alignment. S7. The argmax function is used to perform pixel-level classification on the segmentation probability map of the test set, and outputs a binary segmentation mask of the tumor region and normal tissue to achieve accurate localization of breast cancer lesions.

[0008] Furthermore, in step S4, the image features in the training set are... Separate foreground features for training Compared with training background features Specifically, the following steps are included: S411, Input an image feature from the training set. and the corresponding GT mask ; S412, Through Dimension Transformation Operation GT mask Zoom to image features Same size; S413. Use Hadamard product to perform element-wise feature filtering to obtain the training foreground features. Compared with training background features The formula is as follows: ; ; in, It is a training foreground feature; These are training background features; It is a Hadama pile; It is a matrix of all 1s from the k-th breast cancer image data in the training set. It is the background area mask.

[0009] Furthermore, in step S4, the image features in the test set are... Forming test foreground features With test background features Specifically, the following steps are included: S421. Image features in the input test set Image features The feature maps were copied into two sets of equal-sized maps, which were then used as initial values ​​for the foreground features of the test set. Initial values ​​of background features of the test set ; S422. Enhance tumor semantics through attention interaction, suppress background feature interference, and output test foreground features. and test set background features .

[0010] Furthermore, in step S4, the training foreground features are... Test foreground features The restructuring process includes the following steps: S431, Training foreground features Test foreground features The self-attention mechanism is applied separately, and the formula is as follows: ; ; in, , These are training self-attention enhancement features and testing self-attention enhancement features; , , , , , These are the query, key, and value matrices of the self-attention mechanism; It is a normalization function; S432. Based on the trained self-attention enhancement features and the tested self-attention enhancement features, semantic associations are established through cross-attention, and the foreground alignment features are obtained after alignment. The formula is as follows: ; in, It is a foreground alignment feature; It is the transpose matrix of the self-attention-enhanced features trained on.

[0011] Furthermore, in step S5, the process of processing the foreground alignment features using masked average pooling specifically includes the following steps: S511, Input Foreground Alignment Features and the corresponding GT mask Through truth function Filter out the GT mask The pixel locations belonging to the tumor region are identified, and the features of the tumor region are averaged and pooled. S512. The tumor region features of multiple breast cancer image data in the test set are aggregated by mean to generate a tumor foreground prototype with generalization ability. The formula is as follows: ; in, It is a prototype of tumor prospecting. It represents the number of breast cancer imaging data in the training set.

[0012] Furthermore, in step S5, the training background features are... Extracting potential auxiliary information specifically includes the following steps: S521, Input Training Background Features and the corresponding GT mask Linear projection operation is used to train background features Divide into n non-overlapping patch blocks, each patch block having a size of [size missing]. The formula is as follows: ; in, , ; S522, Add learnable positional codes to each patch block ,Right now: ; S523. Each patch block is feature-encoded using Vision Transformer (ViT) to mine semantic relationships between patches. Features of pure background regions are then selected based on ground truth masks, and mean aggregation is performed to generate the training set background prototype, as shown in the following formula: .

[0013] Furthermore, in step S5, the tumor foreground prototype is... Training set background prototype The splicing process specifically includes the following steps: S531, Prototype of Tumor Prospect Training set background prototype The prototypes are spliced ​​and merged to form a two-dimensional prototype set, which is as follows: ; S532, By calculating the features of the pixels to be segmented and the two-dimensional prototype set , The similarity is used to determine the probability that the pixel features to be segmented belong to the tumor region or normal tissue.

[0014] Furthermore, step S6, guiding the segmentation prediction of the training set and the test set specifically includes the following steps: S611. Input the image features of breast cancer image data in the training set, the image features of breast cancer image data in the test set, and a two-dimensional prototype set. Use a metric method to calculate the feature of each pixel relative to the prototype set. , Cosine similarity; S612. The cosine similarity is converted into pixel-level segmentation probabilities using the softmax function, generating segmentation probability maps for the training set and the test set. The formula is as follows: ; ; in, and These are the pixel positions of the training and test sets, respectively. The probability of segmentation; It is the cosine similarity function; It is a similarity amplification factor; and These are the training set and the test set in pixels. The eigenvector at that location; S613. Extract the category corresponding to the maximum probability using the argmax function to generate the training set prediction mask and the test set prediction mask. The formula is as follows: ; ; in, , These are the prediction masks for the training set and the prediction masks for the test set, respectively.

[0015] Furthermore, in step S6, constructing the joint loss function specifically includes the following steps: S621. Using cross-entropy loss, calculate the pixel-level error between the predicted mask of the training set, the predicted mask of the test set, and the corresponding GT mask. S622. Calculate the training set loss and the test set loss respectively, using hyperparameters. By balancing the loss weights, we can achieve synergistic optimization of fitting ability and generalization ability, as shown in the following formula: ; ; in, and These are the training set loss and the test set loss, respectively. S623. The joint loss function is constructed, and its formula is as follows: ; in, It is the total loss of the model. It is the loss balance hyperparameter.

[0016] Furthermore, in step S6, optimizing the segmentation performance of the training set and test set specifically includes the following steps: S631, Based on the total model loss All learnable parameters of the model are optimized through stochastic gradient descent; S632. During parameter optimization, training foreground features and test foreground features establish a semantic association through a two-dimensional prototype set, so that the two revolve around the tumor foreground prototype. To form a consistent characteristic distribution; S633. Through the collaborative constraint of training set loss and test set loss, the model prioritizes optimizing the boundary pixel features between tumors and normal tissues, improving boundary segmentation accuracy and reducing the missed detection of small lesions.

[0017] Compared with the prior art, the advantages and positive effects of this invention are: 1. This invention designs an end-to-end technical architecture for breast cancer clinical image segmentation, consisting of "data preprocessing - dual-branch shared weight feature extraction - feature de-entanglement and recombination - multi-dimensional prototype generation - joint learning optimization - precise segmentation output". The entire technology is developed around the characteristics of breast tumor images, such as blurred lesion edges, uneven internal echoes, and easy confusion with dense glandular tissue, as well as the need for pixel-level precise segmentation and detection of small lesions in clinical segmentation. This architecture is fully adaptable to the segmentation scenarios of two-dimensional clinical images such as breast cancer ultrasound and X-ray, and can better address the core clinical pain points of confusion between lesions and normal tissue features and strong heterogeneity of tumor morphology in this field. Through a dual-dimensional prototype set, it solves the clinical problem of difficulty in distinguishing lesions from similar background tissues in breast cancer images. By accurately anchoring the core semantics of the tumor through the foreground prototype and clarifying the feature boundaries of normal tissue through the background prototype, it significantly improves the segmentation accuracy of tumor boundaries and the detection rate of small invasive lesions, and greatly reduces the problems of over-segmentation and missed lesion detection.

[0018] 2. This invention constructs an end-to-end training mechanism of "dual-branch synchronous segmentation prediction + joint loss function optimization". Utilizing a unified two-dimensional prototype set, it simultaneously performs pixel-level segmentation prediction on both the support set and the query set, synchronously generating segmentation probability maps and prediction masks for both branches, thus achieving synchronous supervised optimization of both branches. It designs a joint total loss function that integrates the support set loss and the query set loss, balancing the optimization weights of both branches through hyperparameters to simultaneously minimize the fitting error of the support set and the generalization error of the query set, avoiding model bias and overfitting caused by single-branch optimization in small-sample scenarios. This joint learning strategy addresses the problem of heterogeneous feature distribution in multi-center breast images caused by differences in equipment and imaging position. By backpropagating the joint loss, it simultaneously achieves cross-branch prototype semantic alignment, enabling the foreground features of the support set and query set to form a consistent feature distribution around a unified foreground prototype. This effectively solves the problem of heterogeneous feature distribution in multi-center breast images caused by differences in equipment and imaging position. The joint learning strategy simultaneously achieves a balanced optimization of model fitting ability and generalization ability, ensuring accurate learning of labeled lesion features and improving the segmentation adaptability to unknown clinical images and heterogeneous lesions. It is also more adaptable to the complex and varied breast cancer imaging data in real clinical scenarios.

[0019] 3. This invention addresses the characteristics of breast ultrasound lesions, such as blurred edges, uneven internal echoes, and easy confusion with dense glandular tissue. It customizes a foreground feature enhancement and fine-grained background coding mechanism to improve the distinction between lesions and normal tissue. Furthermore, to address the difficulty in capturing key pathological features such as calcification clusters and spiculated edges in breast X-ray images, it enhances the extraction of key local features of lesions through intra-branch self-attention, preventing core pathological features from being masked by background noise. Simultaneously, this invention pioneers a dual-branch feature de-entanglement architecture. The training set utilizes clinical pixel-level ground truth masks, and the Hadamard product precisely separates tumor foreground features from normal tissue background features, achieving semantic purification of lesion features. In an unlabeled test set scenario, this invention achieves unsupervised de-entanglement of foreground and background through feature replication initialization and cross-branch attention guidance. Combining intra-branch self-attention to enhance core tumor features and cross-branch cross-attention to achieve feature recombination effectively eliminates intra-class differences caused by tumor morphological heterogeneity. Finally, by splicing and fusing foreground and background prototypes, a fully covered semantic prototype set is formed, providing precise guidance for pixel-level segmentation. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of the overall system of the present invention; Figure 2 This is a model architecture diagram of the present invention; Figure 3 A schematic diagram of the architecture for fine-grained coding operations. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art to all other embodiments obtained without creative effort should be included within the protection scope of the present invention.

[0023] 1. Overall Model Flow This invention discloses a breast cancer image segmentation method based on feature deentanglement and recombination and joint learning, and constructs a breast cancer image segmentation model based on cross-branch feature deentanglement and recombination and joint learning for implementation. The overall process is as follows: Figure 1 As shown, it forms a complete technical process of "data preprocessing, dual-branch feature extraction, feature deentanglement and recombination, multi-dimensional prototype generation, joint learning optimization, and accurate segmentation output", with the specific steps as follows: (1) Collect breast cancer imaging data from the hospital, have professional doctors annotate the tumor regions of the images at the pixel level, generate the corresponding ground truth mask (GTmask), combine the imaging data and annotation information to construct a dataset, and divide it into training set and test set according to the data distribution characteristics to ensure the clinical validity and annotation accuracy of the data; (2) Perform unified preprocessing on breast cancer images in the training set and test set, including image size normalization, pixel value standardization and noise suppression, and simultaneously retain the doctor-annotated GT mask corresponding to the training set images to provide a reliable annotation basis for subsequent feature separation and supervised learning; (3) In the dual-branch feature extraction stage, the ResNet-101 backbone network with shared weights is used to extract the features of the training set images respectively. Image features of the test set This ensures the consistency and generalization ability of the two-branch feature extraction, while also guaranteeing the adaptability of the feature extraction process to clinical imaging data. (4) In the feature deentanglement and recombination stage, the training set features are separated into tumor foreground features through the GT mask labeled by the doctor. Characteristics of normal tissue background The test set features do not require ground truth masks; two sets of equal-sized feature maps are generated by copying to initialize the foreground features of the test set. and background features Subsequently, intra-branch self-attention is applied to the bi-branch foreground features to enhance the focus of tumor-related task features. Then, the bi-branch foreground features are reorganized through the cross-attention mechanism to reduce intra-class differences of tumors of different morphologies and improve the inter-class distinction between tumors and normal tissues. (5) In the multi-dimensional prototype generation stage, masked average pooling is used to generate tumor foreground prototypes from the recombined bi-branch foreground features. The training set background features are processed by a fine-grained encoder to extract latent auxiliary information and generate a background prototype. Ultimately, these are pieced together to form a two-dimensional prototype set of "foreground-background". Make full use of the precise semantic information provided by the doctor's annotation mask; (6) In the joint learning optimization stage, the prototype set is used to guide the segmentation prediction of the training set and the test set respectively. The similarity between the feature and the prototype is calculated by non-parametric measurement method to generate a bi-branch segmentation probability map. Based on the prediction mask and the doctor-annotated GT mask, the cross-entropy loss is calculated, the joint loss function is constructed, and the bi-branch segmentation performance is optimized in parallel to promote prototype alignment and feature de-entanglement, and ensure that the model learns the segmentation rules that fit the clinical reality. (7) Finally, the argmax function is used to perform pixel-level classification of the segmentation probability map of the test set, and outputs the binary segmentation mask of the tumor area and normal tissue to achieve accurate localization of breast cancer lesions and provide intuitive and reliable image segmentation results for clinical diagnosis.

[0024] 2. Core Module Description like Figure 1 As shown, the model system implemented in this invention mainly includes a data layer, a feature processing layer, a model optimization layer, and an output layer; the data layer is equipped with a data acquisition module, a data preprocessing module, a bi-branch feature extraction module, a feature de-entanglement and recombination module, a multi-dimensional prototype generation module, a joint learning optimization module, a precise segmentation output module, and a clinical auxiliary result output module; the above modules work together to implement the above steps; The working process of the key modules in this invention is described in detail below: (1) Feature deentanglement and recombination module like Figure 2 As shown, to address the problem of insufficient segmentation accuracy caused by semantic confusion between tumor regions (foreground) and normal glands (background) features in breast cancer images, this module achieves effective separation and semantic alignment of foreground and background features through training set feature separation, unsupervised deentanglement of test set features, and cross-branch feature recombination. This strengthens the intra-class consistency and inter-class discriminative power of tumor features, providing a high-quality feature foundation for subsequent prototype generation.

[0025] Training set feature separation: Utilizing pixel-level tumor ground truth masks annotated by doctors in the training set, foreground and background separation is performed on the training set features extracted by the backbone network ResNet-101 to ensure the semantic purity of the supporting features. Input: Training set features extracted by ResNet-101 ,in, The number of feature channels, For the feature map height and width, the corresponding doctor-annotated ground truth mask for the supporting image. , Original image dimensions. Dimensional transformation operation. GT mask Scale to the training set features Same size To ensure the feasibility of element-wise operations, the Hadamard product is used to perform element-wise feature selection, yielding foreground and background features of the training set, as shown in the following formula: ; ; in, These are foreground features of the training set; These are background features of the training set; It is the Hadamard product, which performs element-wise multiplication of features and masks; It is a matrix of all 1s from the k-th training image, used to generate the background mask. That is, the background area mask.

[0026] Unsupervised deentanglement of test set features: For scenarios where the test set lacks doctor-annotated ground truth masks, unsupervised deentanglement of foreground and background features is achieved through feature duplication initialization and cross-branch interaction guidance, preventing irrelevant background features from interfering with tumor segmentation. The input is the test set features extracted by the backbone network. , test set features The feature maps were copied into two sets of equal-sized maps, which were then used as initial values ​​for the foreground features of the test set. Initial values ​​of background features of the test set By leveraging the semantic correlation between the foreground features of the training set and the foreground features of the test set during subsequent cross-branch feature recombination, the tumor semantics of the foreground features of the test set are indirectly enhanced through attention interactions, suppressing interference from background features. Ultimately, this achieves dynamic deentanglement of the test set features and outputs the foreground features of the test set. and test set background features .

[0027] Cross-branch feature reorganization: Intra-branch self-attention is used to enhance the task relevance of bi-branch foreground features, and cross-attention is then used to achieve semantic alignment of cross-branch foreground features. This reduces intra-class differences caused by tumor morphological variations in different training and testing sets, improving feature robustness. (The last sentence appears to be incomplete and possibly refers to a separate process for training set foreground features.) and test set foreground features Self-attention mechanisms are applied separately to focus on the core semantic features of the tumor and suppress local noise, as shown in the following formula: ; ; in, , These are foreground self-attention enhancement features for the training and testing sets, respectively. , , , , , It is a query, key, and value matrix based on a self-attention mechanism; This is a normalization function that makes the sum of attention weights equal to 1, focusing on important feature regions. Based on the bi-branch foreground features enhanced by self-attention, semantic associations are established through cross-attention, aligning the tumor feature distributions between the training and test sets, and reducing feature shifts caused by morphological variations. The formula is as follows: ; in, It is a foreground alignment feature after cross-branch recombination; It is the transpose of the foreground self-attention features in the training set, used to calculate the semantic relevance of the bi-branch features.

[0028] (2) Multi-dimensional prototype generation module To address the issues in existing breast cancer image segmentation, such as insufficient semantic representation of single-category prototypes and ineffective utilization of background information leading to blurred tumor boundaries and missed detection of small lesions, this module constructs a dual-dimensional prototype set based on high-quality features after feature deentanglement and recombination. This set includes cross-branch aligned foreground prototypes and fine-grained background prototypes, comprehensively covering the core semantics of tumors and the background semantics of normal tissues, providing accurate and comprehensive semantic guidance for pixel-level segmentation.

[0029] Cross-branch aligned foreground prototype generation: Based on the cross-branch foreground alignment features after feature reorganization, masked average pooling is performed using the ground truth mask annotated by the doctor to extract the core semantic features of the tumor and generate a high-purity foreground prototype, ensuring accurate localization of the tumor region. The input is the cross-branch foreground alignment features output by the feature reorganization module. and Through truth functions Pixels belonging to the tumor region within the ground truth mask are selected, and the features of this region are subjected to average pooling to avoid background noise interference. The tumor region features from multiple supporting images are then aggregated by mean to generate a tumor foreground prototype with generalization capabilities, as shown in the following formula: ; in, It is a prototype of tumor prospecting. The denominator is the number of training set images, and the total number of pixels in the tumor region in all training set images. This ensures that the pooling result is the mean of the tumor region features, thus improving the representativeness of the prototype.

[0030] Fine-grained training set background prototype generation: such as Figure 3 As shown, to address the semantic correlation between normal background tissues such as glands and fat and tumor regions in breast cancer images, fine-grained encoding is used to mine potential auxiliary information in the training set background, generating multi-dimensional background prototypes. This avoids semantic ambiguity caused by single background prototypes and helps distinguish tumors from similar background tissues. The input feature deentanglement module outputs the training set background features. and Linear projection operation is used to transform the background features of the training set. Divide into n non-overlapping patches, each patch having a size of [size missing]. The formula is as follows: ; in, , Then, learnable positional codes are added to each patch. Used to capture spatial location information of the background area, avoiding patch semantic confusion, i.e. Then, ViT is used to encode features for each patch, mining the semantic relationships between patches and strengthening the overall semantic expression of the background organization. Based on the ground truth mask, patch features of pure background regions are selected, and mean aggregation is performed to generate the training set background prototype, as shown in the following formula: ; in, It is the prototype of the training set background.

[0031] Dual-dimensional prototype set fusion: The cross-branch aligned foreground prototype and the fine-grained supporting background prototype are spliced ​​and fused to form a complete multi-dimensional prototype set, achieving full coverage of tumor semantics and normal tissue semantics, providing comprehensive and accurate semantic guidance for subsequent segmentation. The prototype set is as follows: ; in, Focusing on the core semantics of tumors, Covering normal organizational semantics, By calculating the features of the pixels to be segmented and the prototype set , The similarity is used to determine the probability that a pixel belongs to a tumor region or normal tissue, thereby improving the accuracy of boundary segmentation and the detection rate of small lesions.

[0032] (3) Joint Learning Optimization Module To address the issues of insufficient model generalization ability, low tumor boundary segmentation accuracy, and poor adaptability to heterogeneous data distribution caused by optimizing only a single branch in existing breast cancer segmentation, this module improves the segmentation accuracy of the training set and the generalization ability of the test set simultaneously through dual-branch joint learning. This ensures that the model can achieve accurate segmentation of breast cancer images with different shapes and distributions under sufficient data conditions.

[0033] Dual-branch simultaneous segmentation prediction: Utilizing the foreground and background prototype sets output by the multi-dimensional prototype generation module, pixel-level segmentation prediction is performed simultaneously on both the training and test sets, generating a dual-branch segmentation probability map to provide a foundation for subsequent collaborative optimization. The input backbone network extracts features from the training set. Test set features and multidimensional prototype set A metric method is used to calculate the features of each pixel in the prototype set. , The cosine similarity measures the semantic association between pixels belonging to tumor or normal tissue. The similarity is converted into pixel-level segmentation probabilities using the softmax function, generating a bi-branch segmentation probability map to determine the category probability of each pixel. The formula is as follows: ; ; in, and At pixel positions in the training and test sets, respectively. The segmentation probabilities correspond to those belonging to normal tissue. and belongs to tumors The probability of; It is a cosine similarity function that calculates the semantic matching degree between pixel features and prototype vectors. The closer the value is to 1, the more consistent the semantics. It is a similarity amplification factor used to enhance the discriminative power of probability distributions; and The training set and the test set are in pixels The feature vector at that location. Further, the class corresponding to the maximum probability is extracted using the argmax function, generating a two-branch prediction mask: ; ; in, and It is the prediction mask for the training set and the test set.

[0034] Joint Loss Function Optimization: A joint loss function is constructed that integrates the training set loss and the test set loss. The weights of the two branches are balanced through hyperparameter optimization to simultaneously minimize the training set segmentation error and the test set generalization error, avoiding model bias caused by single-branch optimization. Cross-entropy loss is used to calculate the pixel-level error between the predicted mask and the doctor-annotated ground truth mask. The training set loss and the test set loss are calculated separately. Hyperparameter optimization is then applied. By balancing the weights of the two-branch loss, a synergistic optimization of fitting ability and generalization ability is achieved, as shown in the following formula: ; ; in, and This represents the loss from the training and test sets. Further, we construct the joint total loss function: ; in, It is the total loss of the model. It is a loss balancing hyperparameter that achieves a balanced optimization of fitting and generalization.

[0035] Cross-branch prototype alignment: Through backpropagation of the joint loss function, the tumor foreground prototype semantics of the training and test sets are semantically aligned, reducing the difference in cross-branch feature distribution and improving the model's adaptability to heterogeneous data distribution scenarios. Based on the total loss... All learnable parameters of the model are optimized using stochastic gradient descent. During parameter optimization, foreground features from the training set and the test set are semantically associated through a prototype set, ensuring they resonate with each other. A consistent feature distribution is formed. Through the collaborative constraint of bi-branch loss, the model prioritizes optimizing the boundary pixel features between tumors and normal tissues, improving boundary segmentation accuracy and reducing the missed detection of small lesions.

[0036] The present invention has the following beneficial effects: 1. This invention designs an end-to-end technical architecture for breast cancer clinical image segmentation, consisting of "data preprocessing - dual-branch shared weight feature extraction - feature de-entanglement and recombination - multi-dimensional prototype generation - joint learning optimization - precise segmentation output". The entire technology is developed around the characteristics of breast tumor images, such as blurred lesion edges, uneven internal echoes, and easy confusion with dense glandular tissue, as well as the need for pixel-level precise segmentation and detection of small lesions in clinical segmentation. This architecture is fully adaptable to the segmentation scenarios of two-dimensional clinical images such as breast cancer ultrasound and X-ray, and can better address the core clinical pain points of confusion between lesions and normal tissue features and strong heterogeneity of tumor morphology in this field. Through a dual-dimensional prototype set, it solves the clinical problem of difficulty in distinguishing lesions from similar background tissues in breast cancer images. By accurately anchoring the core semantics of the tumor through the foreground prototype and clarifying the feature boundaries of normal tissue through the background prototype, it significantly improves the segmentation accuracy of tumor boundaries and the detection rate of small invasive lesions, and greatly reduces the problems of over-segmentation and missed lesion detection.

[0037] 2. This invention constructs an end-to-end training mechanism of "dual-branch synchronous segmentation prediction + joint loss function optimization". Utilizing a unified two-dimensional prototype set, it simultaneously performs pixel-level segmentation prediction on both the support set and the query set, synchronously generating segmentation probability maps and prediction masks for both branches, thus achieving synchronous supervised optimization of both branches. It designs a joint total loss function that integrates the support set loss and the query set loss, balancing the optimization weights of both branches through hyperparameters to simultaneously minimize the fitting error of the support set and the generalization error of the query set, avoiding model bias and overfitting caused by single-branch optimization in small-sample scenarios. This joint learning strategy addresses the problem of heterogeneous feature distribution in multi-center breast images caused by differences in equipment and imaging position. By backpropagating the joint loss, it simultaneously achieves cross-branch prototype semantic alignment, enabling the foreground features of the support set and query set to form a consistent feature distribution around a unified foreground prototype. This effectively solves the problem of heterogeneous feature distribution in multi-center breast images caused by differences in equipment and imaging position. The joint learning strategy simultaneously achieves a balanced optimization of model fitting ability and generalization ability, ensuring accurate learning of labeled lesion features and improving the segmentation adaptability to unknown clinical images and heterogeneous lesions. It is also more adaptable to the complex and varied breast cancer imaging data in real clinical scenarios.

[0038] 3. This invention addresses the characteristics of breast ultrasound lesions, such as blurred edges, uneven internal echoes, and easy confusion with dense glandular tissue. It customizes a foreground feature enhancement and fine-grained background coding mechanism to improve the distinction between lesions and normal tissue. Furthermore, to address the difficulty in capturing key pathological features such as calcification clusters and spiculated edges in breast X-ray images, it enhances the extraction of key local features of lesions through intra-branch self-attention, preventing core pathological features from being masked by background noise. Simultaneously, this invention pioneers a dual-branch feature de-entanglement architecture. The training set utilizes clinical pixel-level ground truth masks, and the Hadamard product precisely separates tumor foreground features from normal tissue background features, achieving semantic purification of lesion features. In an unlabeled test set scenario, this invention achieves unsupervised de-entanglement of foreground and background through feature replication initialization and cross-branch attention guidance. Combining intra-branch self-attention to enhance core tumor features and cross-branch cross-attention to achieve feature recombination effectively eliminates intra-class differences caused by tumor morphological heterogeneity. Finally, by splicing and fusing foreground and background prototypes, a fully covered semantic prototype set is formed, providing precise guidance for pixel-level segmentation.

Claims

1. A breast cancer image segmentation method based on feature deentanglement and recombination and joint learning, characterized in that: Includes the following steps: S1. Collect breast cancer imaging data. Professional doctors annotate the tumor regions of the breast cancer imaging data at the pixel level and generate corresponding GT masks. Combine the breast cancer imaging data with the GT masks to construct a dataset and divide the dataset into training set and test set. S2. Preprocess the breast cancer image data in the training set and the test set, and simultaneously retain the GT mask corresponding to the breast cancer image data in the training set. S3. Using a ResNet-101 backbone network with shared weights, image features of breast cancer image data in the training set are extracted. Imaging features of breast cancer imaging data in the test set ; S4. Extract image features from the training set. The foreground features are separated into training foreground features using the corresponding ground truth mask. Compared with training background features Image features in the test set Two sets of identically sized feature maps are generated by copying, and then initialized to form the test foreground features. With test background features Subsequently, the training foreground features were analyzed. Test foreground features Apply intra-branch self-attention separately, and then use a cross-attention mechanism to train foreground features. Test foreground features Recombination yields foreground alignment features; S5. Masked average pooling is used to process the foreground alignment features to generate a tumor foreground prototype. Training background features are processed through a fine-grained encoder. Extract potential auxiliary information and generate a training set background prototype. The prototype of the tumor prospect Training set background prototype splicing together to form a two-dimensional prototype set ; S6, Utilizing a Two-Dimensional Prototype Set It guides the segmentation prediction of the training set and the test set respectively, calculates the similarity between features and prototypes through non-parametric metric methods, and generates the segmentation probability map and prediction mask of the training set, the segmentation probability map and prediction mask of the test set, and the cross-entropy loss based on the prediction mask of the training set, the prediction mask of the test set and the GT mask, constructs a joint loss function, and simultaneously optimizes the segmentation performance of the training set and the test set to promote prototype alignment. S7. The argmax function is used to perform pixel-level classification on the segmentation probability map of the test set, and outputs a binary segmentation mask of the tumor region and normal tissue to achieve accurate localization of breast cancer lesions.

2. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 1, characterized in that: In step S4, the image features in the training set Separate foreground features for training Compared with training background features Specifically, the following steps are included: S411, Input an image feature from the training set. and the corresponding GT mask ; S412, Through Dimension Transformation Operation GT mask Zoom to image features Same size; S413. Use Hadamard product to perform element-wise feature filtering to obtain the training foreground features. Compared with training background features The formula is as follows: ； ； in, It is a training foreground feature; These are training background features; It is a Hadama pile; It is a matrix of all 1s from the k-th breast cancer image data in the training set. It is the background area mask.

3. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 2, characterized in that: In step S4, the image features in the test set Forming test foreground features With test background features Specifically, the following steps are included: S421. Image features in the input test set Image features The feature maps were copied into two sets of equal-sized maps, which were then used as initial values ​​for the foreground features of the test set. Initial values ​​of background features of the test set ; S422. Enhance tumor semantics through attention interaction, suppress background feature interference, and output test foreground features. and test set background features .

4. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 3, characterized in that: In step S4, the training foreground features Test foreground features The restructuring process includes the following steps: S431, Training foreground features Test foreground features The self-attention mechanism is applied separately, and the formula is as follows: ； ； in, , These are training self-attention enhancement features and testing self-attention enhancement features; , , , , , These are the query, key, and value matrices of the self-attention mechanism; It is a normalization function; S432. Based on the trained self-attention enhancement features and the tested self-attention enhancement features, semantic associations are established through cross-attention, and the foreground alignment features are obtained after alignment. The formula is as follows: ； in, It is a foreground alignment feature; It is the transpose matrix of the self-attention-enhanced features trained on.

5. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 4, characterized in that: In step S5, the foreground alignment features are processed using masked average pooling, specifically including the following steps: S511, Input Foreground Alignment Features and the corresponding GT mask Through truth function Filter out the GT mask The pixel locations belonging to the tumor region are identified, and the features of the tumor region are averaged and pooled. S512. The tumor region features of multiple breast cancer image data in the test set are aggregated by mean to generate a tumor foreground prototype with generalization ability. The formula is as follows: ； in, It is a prototype of tumor prospecting. It represents the number of breast cancer imaging data in the training set.

6. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 5, characterized in that: In step S5, the training background features are... Extracting potential auxiliary information specifically includes the following steps: S521, Input Training Background Features and the corresponding GT mask Linear projection operation is used to train background features Divide into n non-overlapping patch blocks, each patch block having a size of [size missing]. The formula is as follows: ； in, , ; S522, Add learnable positional codes to each patch block ,Right now: ； S523. Each patch block is feature-encoded using a Vision Transformer to mine semantic relationships between patches. Features of pure background regions are then selected based on ground truth masks, and mean aggregation is performed to generate the training set background prototype, as shown in the following formula: 。 7. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 6, characterized in that: In step S5, the tumor foreground prototype Training set background prototype The splicing process specifically includes the following steps: S531, Prototype of Tumor Prospect Training set background prototype The prototypes are spliced ​​and merged to form a two-dimensional prototype set, which is as follows: ； S532, By calculating the features of the pixels to be segmented and the two-dimensional prototype set , The similarity is used to determine the probability that the pixel features to be segmented belong to the tumor region or normal tissue.

8. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 7, characterized in that: Step S6, which guides the segmentation prediction of the training set and the test set, specifically includes the following steps: S611. Input the image features of breast cancer image data in the training set, the image features of breast cancer image data in the test set, and a two-dimensional prototype set. Use a metric method to calculate the feature of each pixel relative to the prototype set. , Cosine similarity; S612. The cosine similarity is converted into pixel-level segmentation probabilities using the softmax function, generating segmentation probability maps for the training set and the test set. The formula is as follows: ； ； in, and These are the pixel positions of the training and test sets, respectively. The probability of segmentation; It is the cosine similarity function; It is a similarity amplification factor; and These are the training set and the test set in pixels. The eigenvector at that location; S613. Extract the category corresponding to the maximum probability using the argmax function to generate the training set prediction mask and the test set prediction mask. The formula is as follows: ； ； in, , These are the prediction masks for the training set and the prediction masks for the test set, respectively.

9. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 8, characterized in that: In step S6, constructing the joint loss function specifically includes the following steps: S621. Using cross-entropy loss, calculate the pixel-level error between the predicted mask of the training set, the predicted mask of the test set, and the corresponding GT mask. S622. Calculate the training set loss and the test set loss respectively, using hyperparameters. By balancing the loss weights, we can achieve synergistic optimization of fitting ability and generalization ability, as shown in the following formula: ； ； in, and These are the training set loss and the test set loss, respectively. S623. The joint loss function is constructed, and its formula is as follows: ； in, It is the total loss of the model. It is the loss balance hyperparameter.

10. The breast cancer image segmentation method based on feature deentanglement recombination and joint learning as described in claim 9, characterized in that: Step S6, optimizing the segmentation performance of the training set and test set, specifically includes the following steps: S631, Based on the total model loss All learnable parameters of the model are optimized through stochastic gradient descent; S632. During parameter optimization, training foreground features and test foreground features establish a semantic association through a two-dimensional prototype set, so that the two revolve around the tumor foreground prototype. To form a consistent characteristic distribution; S633. Through the collaborative constraint of training set loss and test set loss, the model prioritizes optimizing the boundary pixel features between tumors and normal tissues, improving boundary segmentation accuracy and reducing the missed detection of small lesions.

Images