Breast cancer image classification domain generalization method based on suppression domain correlation features
By using adaptive channel filters and low-frequency perturbation modules in breast cancer image classification, domain-related features are suppressed, and the problem of the lack of robustness of the changes in image protocols of different devices in the prior art is solved, and higher classification accuracy and generalization performance are achieved.
Patent Information
- Application Number
- CN202510387808.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-06-27
AI Technical Summary
Existing deep learning systems lack robustness to changes in different devices’ image capturing protocols in mammogram diagnosis, resulting in a degradation in the performance of the model in the new data domain.
A breast cancer image classification domain generalization method based on inhibition domain-related features is proposed. Through adaptive channel filters and low-frequency perturbation modules, domain-related features are identified and suppressed, thereby improving the classification accuracy of the model in the unseen domain.
By inhibiting domain-related features, the risk of mispredictions in the new mammogram data domain is reduced, and the classification accuracy and generalization performance of the model are improved.
Smart Images
Figure CN120219853A_ABST
Abstract
Description
Technical Field
[0001] The present invention provides a breast cancer image classification domain generalization method based on suppression domain-related features, belonging to the field of medical image classification. Background Art
[0002] Early detection and rapid intervention of breast cancer are crucial and are the key to reducing the mortality rate of patients. Mammography has the characteristics of fast imaging speed, high resolution, and high signal-to-noise ratio, and is considered by imaging experts to be the preferred imaging method for early screening and diagnosis. However, screening mammogram images are usually evaluated by radiologists, and their diagnostic accuracy is deeply affected by their professional level. In addition, the review process is cumbersome and time-consuming, resulting in unnecessary expenditure and redundant allocation of resources. To address these challenges, many computer-aided diagnosis (CAD) systems have been designed to improve the efficiency of radiologists.
[0003] Computer-aided diagnosis based on deep learning has been proven to be able to effectively help doctors make more accurate diagnoses. However, the main obstacle to deploying current deep learning systems in medical image diagnosis is their lack of robustness to distribution changes between internal and external cohorts, which usually exists between multiple mammography devices due to protocol changes in captured images. For example, the appearance of images obtained from different mammography devices may vary significantly. This difference may deteriorate the performance of the trained model. To address this challenge, domain generalization (DG) is necessary as a more challenging but realistic approach. It aims to develop a model trained in various different but related source domains, with the goal of achieving robust performance in any unfamiliar target domain. To achieve domain generalization in mammogram image classification, a natural approach is to only capture features clinically relevant to the disease (i.e., domain-invariant features). However, current domain generalization methods mainly impose constraints on the entire network to supervise the model to learn domain-invariant features. This method acts on the prediction layer of the network, ignoring the possibility that the intermediate layers of the network may still learn too much irrelevant information. Summary of the Invention
[0004] The objective of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose a breast cancer image classification domain generalization method based on suppressing domain-related features (MC-SDS). By suppressing the erroneous influence of domain-related features on the model, the risk of misprediction in a new mammogram image data domain is reduced. The present invention proposes an adaptive channel filter based on dropout to identify channels that tend to capture domain-related features and adaptively discard these channels. The present invention introduces a low-frequency perturbation module to further suppress redundant domain-related features in low-frequency information by perturbing low-frequency components. Thereby improving the classification accuracy of the model in unseen domains. Through extensive experiments, it is proven that the classification performance of the present invention is better than the most advanced image classification domain generalization methods in the past and has better generalization performance.
[0005] A breast cancer image classification domain generalization method based on suppressing domain-related features, the overall structural diagram is as Figure 2 shown, including the following steps:
[0006] S1. The breast cancer X-ray image to be processed is initially segmented using the Otsu segmentation algorithm. The Otsu segmentation method is first used to distinguish the foreground and background. After edge detection, the breast region is segmented to remove the irrelevant background, and the size of the extracted image is adjusted to 224×224;
[0007] S2. The cropped image obtained in S1 is fed into the feature extractor, and an intermediate feature map is obtained through the first few layers of the network. The extractor uses the ResNet-50 backbone network to generate the feature map;
[0008] S3. An adaptive channel filter is inserted into multiple intermediate layers of the feature extractor. For each sample, only one randomly adaptive channel filter is activated to identify and discard the channels in the intermediate feature map obtained in S2 that tend to capture domain-related features, thereby suppressing domain-related features;
[0009] S4. After the activated adaptive channel filter, a low-frequency perturbation module is used to perform low-frequency reconstruction on the sample to further suppress redundant domain-related features in low-frequency information;
[0010] S5. The feature map obtained in S4 passes through the last few layers of the ResNet-50 network to obtain the final feature map;
[0011] S6. The final feature map obtained in S5 uses a fully connected layer to generate the final classification result.
[0012] Preferably, the background cropping process in S1 is as follows: Locate the breast through edge detection and Otsu segmentation, and remove the redundant black background to increase the proportion of effective pixels in the image. Then, select a suitable bounding box based on the segmentation line or dividing line to crop the image, thereby generating the cropped image xi .
[0013] Preferably, in the process of feature extraction by the deep convolutional neural network in S2: the CNN is used to extract features from the cropped image. With this feature extractor, the network can autonomously learn and capture the abstract and high-level features in the image. For the sample x i , first obtain the feature representation F k (x i ) ∈ R C×H×W , where C, H, and W are the number of channels, the height of the feature map, and the width of the feature map, respectively. The calculation formula is as follows:
[0014] F k (x i ) = f e (x|φ) (1)
[0015] In the formula: C, H, and W are the number of channels, the height of the feature map, and the width of the feature map, respectively.
[0016] Preferably, the motivation of the adaptive channel filter in S3 is that the features captured by the model channels can be classified into domain-related features and domain-invariant features. Therefore, identifying and discarding those channels that tend to capture domain-related features can effectively suppress domain-related features, thereby reducing the probability of the model making incorrect predictions on the new data domain. Since each domain has its unique domain-related features, these features are an important basis for the domain classification task. The steps of the adaptive channel filter include: F k (x i ) is input into an adaptive channel filter. To counteract the potential adverse effects of the adaptive channel filter on the learning of the main network, a gradient reversal layer is used in front of the adaptive channel filter. Assuming that the channels most influential for domain classification are the potential carriers of domain-related features, the correlation between each channel and domain-related information can be quantified by calculating the weighted activation value of the correct domain prediction. For the given input F k (x i ), is the weight of the fully connected layer of the adaptive channel filter for the true domain label. The scores s of all channels can be calculated. A higher score indicates that the channel makes a greater contribution to the domain label prediction. Specifically, the score of the j-th channel in F k (x i ) is:
[0017]
[0018] In the formula: represents the j-th element of, and GMP represents global max pooling.
[0019] After scoring the channels, in order to reduce the domain-related information in the feature maps, the most sensitive channels need to be selected and removed during the training process. Specifically, for all channels, in order to make the channel scores more discriminative, the scores s need to be weighted in two steps.
[0020] First, through a normalization method, each element s of the score s j is converted into a probability value p j such that the sum of the probabilities of all channels is equal to 1:
[0021]
[0022] where: C is the number of channels.
[0023] Then, p j is exponentially amplified to obtain the weighted score s'. The higher the weighted score, the greater the likelihood that the channel contains domain-related information.
[0024] Subsequently, the weighted score passes through a linear layer to obtain the threshold score s t . Then, based on the threshold score s t a binary dropout mask m is constructed to determine which channels to discard. Specifically, each element of m is generated as follows:
[0025]
[0026] where: j is the channel index.
[0027] Considering that discarding channels simultaneously in multiple layers may lead to excessive loss of features and hinder the learning process of the model, a multi-layer random activation strategy is introduced. For each sample, a different network layer is randomly selected to activate the dropout. The binary dropout mask only takes effect when the adaptive channel filter of that layer is activated, and it guides the adaptive channel filter to generate the output G(x i ). If the adaptive channel filter is in the deactivated state, it will not perform any processing on the input.
[0028] Preferably, the motivation of the low-frequency perturbation module described in S4 stems from the spectral characteristics. Through high-pass and low-pass filtering experiments on mammograms, it can be observed that high-frequency information describes the edge structure of objects; the low-frequency components retain the smooth structure and style information of objects. For breast cancer diagnosis, information such as the edges, shapes, and curvatures of lesions is crucial. The low-frequency perturbation module suppresses the domain-related features in the low-frequency information by perturbing the low-frequency components, and promotes the model to emphasize the domain-invariant features in the high-frequency components. For G(x i ) processed by the adaptive channel filter, first, G(x i ) is Fourier-transformed:
[0029]
[0030] where \(j\) represents the imaginary unit, and it is standard practice to center the low-frequency components.
[0031] Subsequently, a binary mask \(M\in\mathbb{R}\) is introduced r×r , and all values of \(M\) are zero except for the central part:
[0032]
[0033] where \(r\) represents the scale parameter that controls the size of the mask \(M\) and is used to distinguish the high-frequency and low-frequency components in the spectrum.
[0034] Using this mask, the frequency components \(H\) l (\mathbf{x} i ) of low-pass filtering and the components \(H\) h (\mathbf{x} i ) of high-pass filtering are separated. The specific steps are as follows:
[0035]
[0036] H h (\mathbf{x} i ) = \(H(\mathbf{x} i ) - H l (\mathbf{x} i )\ (8)
[0037] where: denotes element-wise multiplication.
[0038] The low-frequency perturbation module assumes that the low-frequency spectra of different samples follow a Gaussian distribution. Based on this assumption, we generate a new low-frequency spectrum by resampling to replace the initial spectrum. Specifically, for a set of input features where \(B\) is the batch size, we first calculate their Fourier transforms and then extract the low-frequency components. For simplicity of representation, we denote the low-frequency and high-frequency components as and
[0039] Subsequently, we use a Gaussian model to characterize the distribution of each element in the low-frequency spectrum. The model is centered on the value of the original element, and its variance comes from the values of the same element in different samples:
[0040]
[0041] The magnitude of the variance \(\sigma\) 2 represents the variation of the element when considering potential domain shifts. Subsequently, we resample the probability of each element in the low-frequency spectrum based on the estimated distribution:
[0042]
[0043] Finally, combine H′ l (x i ) and H h (x i ) into a new frequency representation H′(x i ), and convert H′(x i ) back to the spatial domain through two-dimensional inverse fast Fourier transform.
[0044] Preferably, the last few layers of the network of S5 receive the feature maps output by the low-frequency perturbation module and output the final feature maps.
[0045] Preferably, the fully connected layer described in S6 generates a classification result from the final feature map. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 is the technical flow chart of the present invention
[0047] Figure 2 is the overall structure diagram of MC-SDS of the present invention DETAILED DESCRIPTION OF THE INVENTION
[0048] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.
[0049] A breast cancer image classification domain generalization method based on suppression domain-related features, comprising the following steps:
[0050] S1. Initially segment the breast cancer X-ray image to be processed using the Otsu segmentation algorithm. The Otsu segmentation method is first used to distinguish the foreground and the background. After edge detection, segment the breast region to remove the irrelevant background, and adjust the size of the extracted image to 224×224;
[0051] S2. The cropped image obtained in S1 is fed into a feature extractor, and intermediate feature maps are obtained through the first few layers of the network. The extractor uses a ResNet-50 backbone network to generate feature maps;
[0052] S3. Insert an adaptive channel filter into multiple intermediate layers of the feature extractor. For each sample, only one randomly adaptive channel filter is activated to identify and discard the channels in the intermediate feature maps obtained in S2 that tend to capture domain-related features, thereby suppressing domain-related features;
[0053] S4. After the activated adaptive channel filter, use the low-frequency perturbation module to perform low-frequency reconstruction on the samples to further suppress the redundant domain-related features in the low-frequency information;
[0054] S5. Pass the feature map obtained in S4 through the last few layers of the ResNet-50 network to obtain the final feature map.
[0055] S6. Use the fully connected layer to generate the final classification result from the final feature map obtained in S5.
[0056] Now, some English meanings in the present invention are explained. Otsu: a commonly used threshold segmentation algorithm; ACF: adaptive channel filter; LFPM: low-frequency perturbation module; Conv: convolution; Dropout: a regularization technique; MC-SDS: a breast cancer image classification domain generalization method based on suppressing domain-related features.
[0057] Dataset preparation:
[0058] We use the public dataset (INbreast) from different mammography devices and two internal datasets (InH1 and InH2). Different mammography devices will cause inter-domain differences. For each dataset, we randomly divide it into a training set and a validation set at a ratio of 9:1. The test set is all the data of a single dataset.
[0059] The INbreast dataset consists of fully labeled full-field digital mammogram images, with a total of 115 different cases. Each case shows multi-view images of the left and right breasts, finally forming a dataset of 410 mammograms. The collection features various lesion types, such as masses, calcifications, asymmetries, and deformities. The InH1 and InH2 datasets are both sourced from the medical record database of the Shandong First Medical University Cancer Hospital. The datasets have been approved by the Ethics Review Committee of the Shandong First Medical University Cancer Hospital. Since the datasets involve the use of anonymous and retrospective data, the requirement for patient informed consent has been waived. It should be noted that the InH1 dataset consists of images captured by the mammography device Siemens MAMMOMAT Inspiration GESenographe Essential, with a total of 513 images. The InH2 dataset consists of images captured by the mammography device HOLOGIC Selenia Dimensions, with a total of 956 images.
[0060] Parameters and experiments:
[0061] The code in this paper is implemented under the PyTorch framework, and the experiment uses an RTX 3090 GPU. We use the pre-trained ResNet-50 on ImageNet as our backbone and obtain the pre-trained weights from the public packages provided by the torchvision library. We optimized the network for 50 epochs using Stochastic Gradient Descent (SGD) with a momentum factor of 0.9 and a weight decay of 0.0005. The training batch size is 64, and the initial learning rate is 0.001, which is decreased by 10% at the 80% mark of the total training epochs. The model performance is evaluated using accuracy (Acc). We adopt the leave-one-domain-out protocol, select the best model on the validation sets of all source domains, and report the accuracy. The reported results are the average accuracy from five independent trials.
[0062] The MC-SDS classification network of the present invention is tested and evaluated with multiple classification networks on three datasets, INbreast, InH1, and InH2. When one dataset is used as the test set, the other two datasets are used for training. The final results are shown in Table 1.
[0063] As can be seen from Table 1, the MC-SDS proposed in the present invention can obtain state-of-the-art results in the unknown domain under all settings. On the public dataset INbreast, MC-SDS achieves the highest Acc compared with other models. It is 2.69% higher than the second-ranked model, FACT. MC-SDS also achieves the highest Acc on the private datasets InH1 and InH2. For InH1, MC-SDS is 0.52% higher than the sub-optimal model, CausEB. For InH2, MC-SDS is 1.27% higher than the second-ranked MixStyle model. The average accuracy of the three datasets is also the highest, 2.07% higher than the second-ranked model, MODE.
[0064] Table 1 Classification accuracies of different classification methods on three datasets
[0065]
[0066] Table 2 Results of ablation experiments
[0067]
[0068] A series of ablation experiments were conducted in this study to verify the effectiveness of the ACF and LFPM modules in the domain generalization task of mammogram classification. First, the settings of each experiment are introduced. ResNet-50: Remove ACF and LFPM from MC-SDS; ACF: Add ACF on the basis of ResNet-50; LFPM: Add LFPM on the basis of ResNet-50.
[0069] As shown in Table 2 of the experimental results, it can be observed that introducing ACF alone or introducing LFPM alone can improve the accuracy of the model. This indicates that deleting the channels that tend to capture specific domain features and perturbing the low-frequency components can both suppress the specific domain features and enhance the generalization ability of the model. In addition, using ACF and LFPM simultaneously can further enhance the performance of the model.
[0070] The comparative experimental results on 3 datasets show that the MC-SDS network proposed by the present invention has good performance in classification accuracy, which is better than several other classification networks. The generalization of the model is improved.
[0071] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for generalizing breast cancer image classification domain based on suppression domain related features, characterized in that: The classification network includes: S1. The breast cancer X-ray image to be processed is initially segmented using the Otsu segmentation algorithm. The Otsu segmentation method is first used to distinguish the foreground and background. After edge detection, the breast area is segmented to remove irrelevant background, and the extracted image size is adjusted to 224×224; S2. The cropped image obtained in S1 is fed into the feature extractor to obtain the intermediate feature map through the first few layers of the network. The extractor uses the ResNet-50 backbone network to generate the feature map; S3. Insert adaptive channel filters in multiple network intermediate layers of the feature extractor. For each sample, only one random adaptive channel filter is activated to identify and discard channels in the intermediate feature map obtained by S2 that tend to capture domain-related features, thereby suppressing domain-related features. S4. After the activated adaptive channel filter, a low-frequency perturbation module is used to reconstruct the sample in low frequency to further suppress the redundant domain-related features in the low-frequency information; S5. Pass the feature map obtained in S4 through the last few layers of ResNet-50 to obtain the final feature map; S6. Use the fully connected layer to generate the final classification result using the final feature map obtained in S5.
2. The breast cancer image classification domain generalization method based on suppression domain related features according to claim 1 is characterized in that: The background cropping process described in S1 is as follows: the breast is located by edge detection and Otsu segmentation, and the redundant black background is removed to increase the effective pixel ratio in the image. Then, the image is cropped by selecting a suitable bounding box based on the segmentation line or separation line to generate a cropped image x c .
3. The method for generalizing breast cancer image classification domain based on suppression domain related features according to claim 1, characterized in that: The deep convolutional neural network feature extraction process described in S2: The cropped image is subjected to feature extraction through CNN. Using this feature extractor, the network can autonomously learn and capture abstract and high-level features in the image. For sample x i , first get the feature representation F from the kth intermediate layer of ResNet-50 k (x i )∈R C×H×W , C, H, and W are the number of channels, the height of the feature map, and the width of the feature map, respectively. The calculation formula is as follows: F k (x i )=f e (x|φ) Where: C, H, W are the number of channels, the height of the feature map, and the width of the feature map respectively.
4. The method for generalizing breast cancer image classification domain based on suppression domain related features according to claim 1, characterized in that: The steps of the adaptive channel filter described in S3 include: quantifying the correlation between each channel and domain-related information by calculating the weighted activation value of the correct domain prediction, calculating the scores of all channels, and selecting and deleting the most sensitive channels during the training process.
5. The method for generalizing breast cancer image classification domain based on suppression domain related features according to claim 1, characterized in that: The steps of the low-frequency perturbation module described in S4 include: firstly, using Fourier transform to perform high-low frequency separation operation, then modeling the low-frequency part as a Gaussian distribution, perturbing the low-frequency part, and then combining the perturbed low-frequency part with the original high-frequency part to form a new spectrum, and finally using two-dimensional fast Fourier inverse transform to convert back to the spatial domain. In this way, the domain-invariant features in the high frequency are emphasized and the domain-related features in the low frequency are suppressed.
6. The method for generalizing breast cancer image classification domain based on suppression domain related features according to claim 1, characterized in that: The latter few layers of the ResNet-50 described in S5 receive the feature maps output by the low-frequency perturbation module and further process them to obtain the final feature maps.
7. The method for generalizing breast cancer image classification domain based on suppression domain related features according to claim 1, characterized in that: The fully connected layer described in S6 receives the final feature map output from the last few layers of the ResNet-50 network and outputs the final classification result.