Breast cancer pathological image classification method and system based on multi-layer attention

Abstract

The application relates to the technical field of deep learning, and particularly discloses a breast cancer pathological image classification method and system based on multi-layer attention, which comprises the following steps: inputting a preprocessed image into a pre-trained residual neural network, extracting an output feature map, and introducing an attention mechanism in a residual module at each depth stage; constructing a feature pyramid structure, taking the multi-layer feature map as input, performing scale alignment and step-by-step fusion on feature maps of different scales through a top-down path and a horizontal connection, generating multi-scale fusion features; capturing long-range dependency relationships and global context information between features through self-attention operation, obtaining enhanced global features; and outputting a classification result of the breast cancer pathological image. The breast cancer pathological image can be accurately and stably automatically classified under different magnification and complex pathological structure conditions through the organic combination of the multi-layer attention mechanism, multi-scale feature fusion and cross-scale relationship modeling.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, specifically to a method and system for classifying breast cancer pathological images based on multi-layer attention. Background Technology

[0002] With the rapid development of medical imaging and digital pathology technologies, breast cancer pathology slides are gradually shifting from traditional manual microscopic observation to digital and intelligent methods. Breast tissue pathology is widely recognized as the gold standard for breast cancer diagnosis, providing crucial information for clinical diagnosis and treatment decisions through the analysis of cell morphology, tissue structure, and spatial distribution characteristics in tissue sections. However, in actual clinical practice, the sheer number of pathology slides, the complexity of tissue structures, and the subtle morphological differences between different pathological subtypes make the process heavily reliant on the professional experience and long-term accumulation of knowledge by pathologists. The diagnostic process is time-consuming, highly subjective, and easily influenced by human factors.

[0003] Existing computer-aided diagnostic methods are mainly based on traditional image processing techniques or deep learning methods. Early methods relied heavily on manually designed features, such as color histograms, texture features (GLCM, LBP, etc.), and morphological features, combined with classifiers such as support vector machines and random forests for discrimination. However, these methods have limited ability to represent complex tissue structures, are difficult to adapt to the high heterogeneity and multi-scale feature variations present in breast pathology images, and have poor generalization ability.

[0004] In recent years, Convolutional Neural Networks (CNNs) have been widely used in breast cancer pathological image classification tasks due to their advantages in automatic image feature learning. Typical methods usually use a single backbone network (such as VGG, ResNet, DenseNet, etc.) to extract features from the input image and achieve classification through fully connected layers. Although these methods improve classification accuracy to some extent, their feature extraction process mainly relies on fixed-scale convolution operations, making it difficult to simultaneously take into account cellular-level details and tissue-level structural information. Especially under different magnifications or different pathological subtypes, the model's adaptability to scale changes is insufficient.

[0005] To address these issues, some studies have introduced multi-scale feature fusion structures, such as Feature Pyramid Networks (FPNs), which enhance multi-scale expressive power by fusing feature maps from different levels of the backbone network. However, most existing multi-scale fusion methods employ simple linear superposition or splicing, which are insufficient in modeling the correlations between features at different scales and fail to fully explore cross-scale semantic relationships. Furthermore, convolutional neural networks are limited by their local receptive fields, resulting in limited ability to model long-distance dependencies and global contextual information, and still exhibiting insufficient discriminative ability in complex pathological tissue structures.

[0006] Furthermore, to enhance the model's ability to focus on key regions, some methods introduce attention mechanisms into the network structure, such as channel attention or spatial attention modules, to strengthen feature responses related to lesions. However, existing attention mechanisms typically operate only at a single scale or level, lacking unified modeling of multi-layer features and cross-scale information, making it difficult to achieve accurate localization and representation of key pathological regions globally. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for classifying breast cancer pathological images based on multi-layer attention, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A breast cancer pathology image classification method based on multi-layer attention, the method comprising:

[0010] Acquire and preprocess breast cancer pathology images;

[0011] The preprocessed image is input into a pre-trained residual neural network, and feature maps output from multiple different depth stages in the residual neural network are extracted. An attention mechanism is introduced into the residual module of each depth stage to obtain multi-layer feature maps.

[0012] A feature pyramid structure is constructed, and the multi-layer feature maps are used as input. Through top-down paths and lateral connections, feature maps of different scales are scale-aligned and fused step by step to generate multi-scale fused features.

[0013] The multi-scale fused features are input into the Transformer module based on the self-attention mechanism. The long-range dependencies and global context information between features are captured through self-attention operations to obtain the enhanced global features.

[0014] The global features are classified to output the classification results of the breast cancer pathology image.

[0015] As a further aspect of the present invention, the step of introducing an attention mechanism into the residual module of each depth stage specifically includes:

[0016] A convolutional block attention module is embedded in at least one residual block at each depth stage. The convolutional block attention module consists of sequentially connected channel attention sub-modules and spatial attention sub-modules, which are used to adaptively weight the feature map in the channel dimension and the spatial dimension.

[0017] As a further embodiment of the present invention, the feature pyramid structure is specifically used for: upsampling the feature map from the deep stage to align its spatial resolution with that of the feature map from the shallow stage; and adding the aligned deep feature map and the shallow feature map element-wise or concatenating them through horizontal connection to achieve the fusion of high-level semantic information and low-level detailed information.

[0018] As a further embodiment of the present invention, the Transformer module of the self-attention mechanism adopts a window self-attention mechanism, specifically: the input feature map is divided into multiple non-overlapping local windows, and self-attention is calculated independently within each window; and information interaction between different windows is achieved by moving the window division between different Transformer layers.

[0019] As a further embodiment of the present invention, the Transformer module includes, in each processing unit, a layer normalization layer, a window multi-head self-attention layer or a moving window multi-head self-attention layer, and a multilayer perceptron connected in sequence, and residual connections are set around each sub-module.

[0020] As a further aspect of the present invention, in the classification output step, a focus loss function is used as the optimization objective during the training process.

[0021] The present invention also provides a breast cancer pathological image classification system based on multi-layer attention, the system comprising:

[0022] The data processing module is used to acquire and preprocess breast cancer pathology images;

[0023] The feature extraction module is used to input the preprocessed image into a pre-trained residual neural network, extract feature maps output from multiple different depth stages in the residual neural network, and introduce an attention mechanism into the residual module of each depth stage to obtain multi-layer feature maps.

[0024] The fusion module is used to construct a feature pyramid structure. It takes the multi-layer feature maps as input and performs scale alignment and step-by-step fusion on feature maps of different scales through top-down paths and lateral connections to generate multi-scale fused features.

[0025] The enhancement module is used to input the multi-scale fused features into the Transformer module based on the self-attention mechanism, and capture the long-range dependencies and global context information between features through self-attention operation to obtain the enhanced global features;

[0026] The output module is used to classify the global features and output the classification results of the breast cancer pathology image.

[0027] Compared with the prior art, the beneficial effects of the present invention are: to construct a multi-layer feature extraction structure based on residual backbone network and to introduce convolutional block attention mechanism in multiple stages of backbone network, which can simultaneously retain detailed information such as texture and cell morphology in shallow pathological images as well as high-level semantic representations contained in deep features.

[0028] Secondly, in the multi-scale feature fusion and modeling stage, a cross-scale fusion network structure based on the combination of feature pyramid and window self-attention mechanism is proposed. The output features of the four stages of the residual backbone network are used as input. The alignment and fusion of features at different scales are realized through the feature pyramid module. Furthermore, a self-attention mechanism based on local windows is introduced to model the cross-scale relationship of the fused multi-scale features, thereby effectively capturing the correlation between tissue structures at different scales in breast pathology images and enhancing the network's overall modeling ability for multi-scale pathological features.

[0029] It achieves an organic combination of multi-layer attention mechanism, multi-scale feature fusion and cross-scale relationship modeling, and can accurately and stably classify breast cancer pathological images under different magnification and complex pathological structure conditions. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0031] Figure 1 This is an overall block diagram of the residual pyramid structure provided in an embodiment of the present invention.

[0032] Figure 2 This is a diagram of a multi-layer feature network structure.

[0033] Figure 3 This is a diagram of the cross-scale fusion network structure.

[0034] Figure 4 The confusion matrix is ​​40x.

[0035] Figure 5 The confusion matrix is ​​100x.

[0036] Figure 6 The confusion matrix is ​​200x.

[0037] Figure 7 The confusion matrix is ​​400x.

[0038] Figure 8 ROC curves comparing different classification methods at a 40x ratio.

[0039] Figure 9ROC curves comparing different classification methods at a 100x ratio.

[0040] Figure 10 ROC curves comparing different classification methods at a 200x ratio.

[0041] Figure 11 ROC curves comparing different classification methods at a 400x ratio. Detailed Implementation

[0042] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0043] In this embodiment of the invention, a breast cancer pathological image classification method based on multi-layer attention is provided, the method comprising:

[0044] Acquire and preprocess breast cancer pathology images;

[0045] The preprocessed image is input into a pre-trained residual neural network, and feature maps output from multiple different depth stages in the residual neural network are extracted. An attention mechanism is introduced into the residual module of each depth stage to obtain multi-layer feature maps.

[0046] A feature pyramid structure is constructed, and the multi-layer feature maps are used as input. Through top-down paths and lateral connections, feature maps of different scales are scale-aligned and fused step by step to generate multi-scale fused features.

[0047] The multi-scale fused features are input into the Transformer module based on the self-attention mechanism. The long-range dependencies and global context information between features are captured through self-attention operations to obtain the enhanced global features.

[0048] The global features are classified to output the classification results of the breast cancer pathology image.

[0049] like Figure 1 As shown, in this embodiment, training the neural network requires constructing a large-scale dataset containing a training set and a test set.

[0050] This invention directly uses the publicly available dataset BreakHis as the ground truth, but this breast cancer pathology slide image dataset has a series of drawbacks: the data volume of specific subtypes of benign and malignant is not evenly distributed; the image quality and size of the data are not perfectly uniform; in order to avoid the above problems, the original pathology images are preprocessed, including image size uniformity, color normalization, pixel standardization and data augmentation.

[0051] Data augmentation operations included random rotation, flipping, scaling, and cropping to improve the model's robustness to different tissue morphologies and imaging conditions. The original dataset contained 6,909 images, and the preprocessed dataset was expanded to include 26,307 images.

[0052] In a preferred embodiment of the present invention, the step of introducing an attention mechanism in the residual module of each depth stage specifically includes:

[0053] A convolutional block attention module is embedded in at least one residual block at each depth stage. The convolutional block attention module consists of sequentially connected channel attention sub-modules and spatial attention sub-modules, which are used to adaptively weight the feature map in the channel dimension and the spatial dimension.

[0054] like Figure 2 As shown, in this embodiment, the multi-layer feature network structure is constructed with a residual network as the backbone framework, preferably using ResNet50 or its equivalent improved structure as the base network. To enhance feature representation capabilities, a Convolutional Block Attention Module (CBAM) is introduced into each residual module of the backbone network. By adaptively modeling the importance of channel and spatial dimensions, it achieves enhanced expression of key pathological features. The backbone network outputs multiple sets of feature maps at different depth stages to form multi-layer feature representations containing shallow detail information and deep semantic information, thereby enhancing the network's adaptive modeling capability for key features. The multi-layer feature network outputs feature maps of different scales and semantic levels at multiple stages of the backbone network, specifically including stages 1, 2, 3, and 4. Each stage achieves gradual expansion of the receptive field and layer-by-layer abstraction of semantic information through progressive downsampling and residual stacking structures, thereby completing multi-scale feature extraction from shallow texture information to deep high semantic information.

[0055] In this process, Stage 1 mainly preserves the local texture, edge and detail structure information of the input image, which is suitable for representing low-level visual features. Stages 2 and 3, while maintaining a certain spatial resolution, gradually integrate a wider range of contextual information to represent mid-level spatial structure and mid-level semantic features. Stage 4, through deeper residual stacking, highly abstracts the target region to form a discriminative high-level semantic feature representation.

[0056] To further enhance the expressive power of features at each layer during the encoding stage, this invention introduces a Convolutional Block Attention Module (CBAM) in each residual stage. The CBAM module is composed of a channel attention submodule and a spatial attention submodule connected in series, capable of modeling the importance of feature maps in both the channel and spatial dimensions. During channel attention modeling, the weight allocation of different channels is adaptively adjusted through global feature statistics, thereby highlighting key feature channels relevant to the task. During spatial attention modeling, spatial location information is weighted to strengthen the response of the target region and suppress interference from invalid background regions.

[0057] The CBAM module is embedded into each bottleneck residual block in a residual manner, so that the attention enhancement process does not destroy the original feature structure, which is conducive to stable network training. By introducing the CBAM module in the multi-layer feature encoding stage, the network can adaptively focus on key information at different scales and different semantic levels, thereby significantly improving the discriminative ability of multi-layer features.

[0058] After processing by the above-mentioned coding network, multiple sets of feature maps can be obtained in stages 1, 2, 3 and 4 respectively. These multiple sets of features differ in spatial resolution, receptive field size and semantic level, thus forming a multi-scale, multi-semantic level feature representation system. This multi-layer feature output mechanism can simultaneously retain shallow detail information and deep semantic information, providing richer and more complementary feature inputs for subsequent feature fusion, target recognition or classification decisions.

[0059] Through the above structural design, the multi-layer feature network proposed in this invention, while maintaining the efficient feature extraction capability of the residual network, introduces an attention mechanism to adaptively enhance multi-scale features, effectively solving the problems of insufficient traditional single-scale feature expression and strong background interference. Without significantly increasing computational complexity, this structure improves the model's response capability to key regions and effective features, thereby enhancing the overall feature expression performance and generalization ability of the network.

[0060] In a preferred embodiment of the present invention, the feature pyramid structure is specifically used to: upsample the feature map from the deep stage to align its spatial resolution with that of the feature map from the shallow stage; and to perform element-level addition or channel splicing of the aligned deep feature map and shallow feature map through horizontal connection to achieve the fusion of high-level semantic information and low-level detailed information.

[0061] like Figure 3 As shown in this embodiment, a cross-scale fusion network structure is proposed, which consists of a multi-scale feature pyramid module and a feature modeling module based on a self-attention mechanism. Figure 3It is a cross-scale fusion network structure used to fuse and model features at different spatial scales and semantic levels, thereby achieving effective interaction of cross-scale information. The input features of the feature pyramid module come from the outputs of the four stages of the backbone residual network, which are used to construct multi-scale feature representations. Specifically, the feature pyramid module receives feature maps from the outputs of multiple stages of the encoding network, and gradually transmits high-level semantic information to low-level feature maps through progressive upsampling and lateral connections, thereby constructing a multi-layer feature set with consistent scale representation. In this way, features at different scales can achieve the fusion expression of multi-scale information while maintaining their respective semantic advantages.

[0062] By performing scale alignment, lateral connection, and hierarchical fusion on feature maps from different stages of the backbone network, hierarchical aggregation of multi-scale features is achieved, enabling high-level semantic features to complement low-level spatial detail information, thereby enhancing the ability to represent differences in tissue structure and lesion heterogeneity in breast cancer pathological images.

[0063] As a preferred embodiment of the present invention, the Transformer module of the self-attention mechanism adopts a window self-attention mechanism, specifically: the input feature map is divided into multiple non-overlapping local windows, and self-attention is calculated independently in each window; and information interaction between different windows is achieved by moving the window division between different Transformer layers.

[0064] In this embodiment, based on the output of the feature pyramid module, a feature modeling module based on a window self-attention mechanism is further introduced to model the global and local relationships of the fused multi-scale features. The feature modeling module adopts a hierarchical structure, processing the input features through multiple self-attention units. Each self-attention unit includes a normalization operation, multi-head self-attention calculation based on a local window, and feedforward network processing. Residual connections are set between the sub-modules to ensure the stability of feature propagation.

[0065] The self-attention computation employs a local window mechanism to reduce computational complexity and utilizes a window movement strategy to achieve information interaction between different windows, thereby enhancing the feature modeling ability within a larger receptive field while maintaining computational efficiency. Through this approach, the network can simultaneously capture local detailed features and cross-scale global correlation information. By leveraging the synergistic effect of the feature pyramid module and the self-attention feature modeling module, the cross-scale fusion network of this invention effectively integrates feature information from different scales and semantic levels, significantly improving the consistency and discriminative ability of feature representation, and providing more comprehensive feature support for subsequent target recognition, classification, or prediction tasks.

[0066] The fused multi-scale features are input into the self-attention feature modeling module. Through local window partitioning and window moving strategies, information interaction between features of different spatial regions and different scales is achieved, thereby obtaining feature representations containing global context information.

[0067] In the classification and prediction stage, the features after self-attention modeling are aggregated and input into the classification decision layer to output the classification result of the breast cancer pathology image. During model training, a loss function suitable for class imbalance problems is used to optimize the network parameters, thereby improving the model's ability to identify minority classes and difficult samples.

[0068] In a preferred embodiment of the present invention, the Transformer module includes, in each processing unit, a layer normalization layer, a window multi-head self-attention layer or a moving window multi-head self-attention layer, and a multilayer perceptron connected in sequence, and residual connections are set around each sub-module.

[0069] In this embodiment, RFPN-Net is divided into two main components: a Multi-Layer Feature Network (MLFN) and a Cross-Scale Fusion Network (CSEN). The MLFN consists of a ResNet50 backbone network incorporating CBAM, extracting semantic features from shallow to deep layers at Conv2x, Conv3x, Conv4x, and Conv5x. The CSEN is built on a Feature Pyramid Network (FPN), integrating feature maps from different levels into a multi-scale feature representation that combines semantics and details through a top-down multi-scale fusion mechanism. Subsequently, these multi-scale features output by the FPN are input into the SwinTransformer module, which consists of Layer Normalization (LN), Window Multi-Head Self-Attention (W-MSA), and an MLP structure. This module further enhances the network's global modeling ability and long-range dependency representation, thereby forming a unified deep feature across scales. This enables the network to capture more discriminative key lesion regions and improve the final classification performance. Finally, the processed semantic features are fed into a classifier to classify breast cancer pathology images into subtypes. At the same time, a focus loss function is introduced during training to alleviate class imbalance and improve model robustness.

[0070] In a preferred embodiment of the present invention, the focus loss function is used as the optimization objective during the training process in the classification output step.

[0071] In this embodiment, the present invention employs the Focal Loss (FL) function. The Focal Loss function addresses the problem of extreme imbalance in the number of samples detected. Traditional cross-entropy loss functions assume that the loss for each sample is equally important. However, when facing imbalanced classes, samples from a few classes often constitute only a small portion of the total sample, making it difficult for the model to learn their effective representations. This leads to the model favoring the majority of classes while exhibiting poor prediction performance for only a few classes. FL, based on the cross-entropy loss function, adjusts the weights of each data category, thereby making the model focus more on smaller or harder-to-classify samples, increasing the weight of misclassified samples to improve accuracy.

[0072] Cross-entropy loss function: .

[0073] Where p is the predicted class probability given by the model, and FL is the cross-entropy function with an added factor. ( >0) makes the model pay more attention to difficult, misclassified samples, that is: .

[0074] Finally, FL introduces a balancing factor based on the previous step. Used to balance the class imbalance between positive and negative samples, the balance factor can be adjusted according to the class weight of the samples, so that samples of a minority class have higher weights. The adjustment parameter is used to adjust the difficulty of samples that are easy to misclassify.

[0075] Finally, the formula for the focus loss function is given: .

[0076] By introducing a balancing factor and adjusting parameters, Cross-Entropy (FL) can effectively address the class imbalance problem, making the model focus more on difficult samples while improving classification accuracy for the minority classes. Compared to the traditional cross-entropy loss function, FL significantly improves the model under class imbalance conditions.

[0077] In the feature extraction stage, this invention constructs a multi-layer feature extraction structure based on a residual backbone network and introduces convolutional block attention mechanisms at multiple stages of the backbone network. This allows for the simultaneous preservation of detailed information such as texture and cell morphology in shallow pathological images, as well as high-level semantic representations contained in deep features. Secondly, in the multi-scale feature fusion and modeling stage, a cross-scale fusion network structure based on a feature pyramid and a window self-attention mechanism is proposed. This structure uses the output features of the four stages of the residual backbone network as input, aligns and fuses features at different scales through a feature pyramid module, and further introduces a self-attention mechanism based on local windows to model cross-scale relationships of the fused multi-scale features. This effectively captures the correlation between tissue structures at different scales in breast pathological images, enhancing the network's overall modeling ability for multi-scale pathological features. The breast cancer pathological image classification method can fully utilize the complementary information between multi-level and multi-scale features, effectively improving the ability to distinguish different types of breast lesions, reducing classification errors caused by scale changes and tissue morphology complexity, and exhibiting high stability and generalization ability. It is suitable for intelligent assisted diagnosis scenarios of breast cancer pathological images.

[0078] In this embodiment of the invention, a five-fold cross-validation method is used to objectively evaluate the performance of the proposed model. This validation method can reliably test the model's generalization ability with a limited number of samples and can be used to detect overfitting that may occur during training. Specifically, all data samples are divided into training, validation, and test sets in an 80:10:10 ratio. In each of the five cross-validations, the above division is recombined, so that all samples are used in the training and validation phases in different phases, thereby ensuring that the model learns and covers the entire dataset. Through this method, the complete performance of the model under class imbalance conditions can be obtained. The results are shown in Table 1.

[0079] Table 1 Performance results of five-fold cross-validation

[0080]

[0081] Experimental results show that the model achieves high classification metrics at all four magnification levels. For the 40× dataset, the metric at five folds is above 98% with a small standard deviation, indicating that the model has stable classification performance across different folds at this magnification level. For the 100× dataset, the model also achieves high average classification performance, with all metrics remaining within the 97-99% range. For the 200× dataset, the model's average performance at five folds is approximately 98%, demonstrating good adaptability at this magnification level. For the 400× dataset, the metrics fluctuate little across five folds, with overall performance remaining within the 98%-99% range. In summary, the experimental results at all four magnification levels indicate that the model's performance on multi-magnification breast pathology images is relatively stable, with the mean and standard deviation of each fold within a reasonable range. This demonstrates that the model described in this invention is applicable to pathology image classification tasks at different magnification levels and maintains stable output performance under different scale conditions.

[0082] To further visualize and analyze the model's classification performance, confusion matrices were constructed based on the results of five-fold cross-validation, and ROC curves for each category were plotted to comprehensively demonstrate the model's discriminative ability in multi-class tasks. The confusion matrix can intuitively reflect the model's prediction distribution across different categories. By statistically analyzing the number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) obtained from each fold of training and calculating their average value across the five folds, the final confusion matrix result was obtained. Simultaneously, ROC curves plotted based on the prediction probabilities of each category were used to evaluate the model's recognition ability at different discrimination thresholds, and the area under the curve (AUC) was further calculated. The AUC value can be used to quantify the model's overall discriminative ability; the closer the curve is to the upper left corner and the larger the AUC value, the higher the model's recognition performance.

[0083] Depend on Figures 4 to 7 Experimental results show that the model described in this invention presents a well-structured confusion matrix on datasets with multiple scalar ratios, and the misclassification rate between categories is low.

[0084] To verify the classification performance of the network, several mainstream deep learning models were selected as comparison algorithms. To ensure the reliability of the experimental results, the classification accuracy of these models at different magnification levels was compared, which allows for a more intuitive evaluation of the model's performance advantages. The results are shown in Table 2.

[0085] Table 2 Performance Indicators of Different Classification Methods

[0086]

[0087] pass Figures 8 to 11ROC visualization analysis showed that the model described in this invention exhibited high discrimination ability at four magnifications: 40×, 100×, 200×, and 400×. The ROC curves for each magnification were significantly closer to the upper left region of the coordinate system, with AUC values ​​ranging from 0.990 to 0.996, all higher than the comparison models. In contrast, the AUC values ​​of other models ranged from approximately 0.938 to 0.989, generally lower than the model of this invention. These results indicate that the model of this invention can maintain stable recognition performance on pathological images at different magnifications and achieve a high true positive rate under conditions of low false positive rate, demonstrating better class discrimination ability and applicability. In conclusion, RFPN-Net has the best experimental results.

[0088] To further analyze the effectiveness of RFPN-Net, several ablation experiments were conducted to investigate each contribution of this paper individually. Systematic ablation experiments were performed at different magnifications, and five-fold cross-validation was used to ensure the statistical reliability of the results.

[0089] The specific implementation method is as follows:

[0090] Baseline model: ResNet50 (2) Single module augmentation model: MLFN (ResNet50+CBAM), ResNet50+CSEN;

[0091] The complete model of this invention: RFPN-Net.

[0092] Table 3 Evaluation indicators for ablation experiments

[0093]

[0094] The experimental results are shown in Table 3. At all four magnification levels, the average accuracy of the complete model of this invention is significantly and consistently higher than all other models. Its comprehensive average accuracy across scales reaches 98.48%, an improvement of 7.38 percentage points compared to the original ResNet50 baseline model (91.10%). This significant improvement demonstrates the effectiveness and synergy of the architecture design of this invention. This proves that the introduction of the attention mechanism, multi-scale fusion, and global relationship modeling module in this invention plays a positive role, producing a synergistic gain of "1+1+1>3".

[0095] The visualization method used in this invention is t-SNE (t-Distributed Stochastic Neighbor Embedding), a nonlinear method for dimensionality reduction and visualization of high-dimensional data. It can effectively preserve the local similarity between data points and map high-dimensional features to two-dimensional or three-dimensional space, thereby intuitively showing the distribution of samples under different categories or conditions. In this invention, t-SNE is applied to the visualization of deep features extracted by the model.

[0096] The final visualization results show that the feature distribution using the method of this invention is clearer, the samples of the same category are more tightly clustered, and the boundaries between different categories are more distinct. This indicates that the model proposed in this invention has better discriminativeness and robustness in feature extraction. In contrast, the feature visualization results of other existing methods show that the sample distribution is more scattered and the categories overlap significantly, indicating that they still have certain limitations in feature expression and classification. Therefore, the technical solution of this invention significantly improves the separability of features and the discriminative ability of the model in pathological image classification tasks.

[0097] To further verify the effectiveness of the technical solution of this invention, a publicly available breast cancer pathological image dataset was used to conduct experimental verification of the proposed method under different magnification conditions. The experimental results show that the method of this invention achieves stable and excellent classification performance under various magnification conditions. Under the 40× magnification condition, the average classification accuracy of the method of this invention reaches over 98%. In the five-fold cross-validation, the results of each fold fluctuate little, with a standard deviation of less than 0.2, indicating that the model has good stability. Under the 100×, 200×, and 400× magnification conditions, the average classification accuracy of the model also remains at around 98%, indicating that the proposed method can adapt to the feature changes of pathological images at different scales. Further comparative experimental results show that compared with traditional lightweight convolutional neural network models, the classification accuracy of the method of this invention is improved by about 8%-13%; compared with single convolutional backbone or single-scale feature fusion methods, the accuracy is improved by about 2%-4%. Furthermore, ablation experiments revealed that when only the basic residual network structure was used, the average classification accuracy of the model was approximately 91%. After introducing a multi-layer attention mechanism, the accuracy improved to approximately 96%-97%. Further combining cross-scale fusion and self-attention modeling structures further improved the model accuracy to over 98%. These results demonstrate that the multi-layer attention mechanism and cross-scale feature modeling have a significant synergistic effect in improving model performance. In summary, the experimental data further proves that the technical solution proposed in this invention can stably and accurately complete the breast cancer pathological image classification task under different magnifications and complex pathological structures, demonstrating good practicality and application value.

[0098] This invention also provides a breast cancer pathology image classification system based on multi-layer attention, the system comprising:

[0099] The data processing module is used to acquire and preprocess breast cancer pathology images;

[0100] The feature extraction module is used to input the preprocessed image into a pre-trained residual neural network, extract feature maps output from multiple different depth stages in the residual neural network, and introduce an attention mechanism into the residual module of each depth stage to obtain multi-layer feature maps.

[0101] The fusion module is used to construct a feature pyramid structure. It takes the multi-layer feature maps as input and performs scale alignment and step-by-step fusion on feature maps of different scales through top-down paths and lateral connections to generate multi-scale fused features.

[0102] The enhancement module is used to input the multi-scale fused features into the Transformer module based on the self-attention mechanism, and capture the long-range dependencies and global context information between features through self-attention operation to obtain the enhanced global features;

[0103] The output module is used to classify the global features and output the classification results of the breast cancer pathology image.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A breast cancer pathological image classification method based on multi-layer attention, characterized in that, The method includes: Acquire and preprocess breast cancer pathology images; The preprocessed image is input into a pre-trained residual neural network, and feature maps output from multiple different depth stages in the residual neural network are extracted. An attention mechanism is introduced into the residual module of each depth stage to obtain multi-layer feature maps. A feature pyramid structure is constructed, and the multi-layer feature maps are used as input. Through top-down paths and lateral connections, feature maps of different scales are scale-aligned and fused step by step to generate multi-scale fused features. The multi-scale fused features are input into the Transformer module based on the self-attention mechanism. The long-range dependencies and global context information between features are captured through self-attention operations to obtain the enhanced global features. The global features are classified, and the classification results of the breast cancer pathology image are output. The steps of introducing an attention mechanism into the residual module at each depth stage specifically include: A convolutional block attention module is embedded in at least one residual block at each depth stage. The convolutional block attention module consists of sequentially connected channel attention sub-modules and spatial attention sub-modules, which are used to adaptively weight the feature map in the channel dimension and the spatial dimension. The feature pyramid structure is specifically used to: upsample the feature map from the deep stage to align its spatial resolution with that of the feature map from the shallow stage; and to add the aligned deep feature map and the shallow feature map element-wise or concatenate them through horizontal connection to achieve the fusion of high-level semantic information and low-level detailed information. The Transformer module of the self-attention mechanism adopts a window self-attention mechanism, which is as follows: the input feature map is divided into multiple non-overlapping local windows, and self-attention is calculated independently in each window; and information interaction between different windows is achieved by moving the window division between different Transformer layers. The Transformer module includes, in each processing unit, a layer normalization layer, a window multi-head self-attention layer or a moving window multi-head self-attention layer, and a multilayer perceptron connected in sequence, and residual connections are set around each sub-module.

2. The breast cancer pathological image classification method based on multi-layer attention according to claim 1, characterized in that, In the classification output step, the focus loss function is used as the optimization objective during the training process.

3. A breast cancer pathological image classification system based on multi-layer attention, used to implement the breast cancer pathological image classification method based on multi-layer attention as described in claim 1 or 2, characterized in that, The system includes: The data processing module is used to acquire and preprocess breast cancer pathology images; The feature extraction module is used to input the preprocessed image into a pre-trained residual neural network, extract feature maps output from multiple different depth stages in the residual neural network, and introduce an attention mechanism into the residual module of each depth stage to obtain multi-layer feature maps. The fusion module is used to construct a feature pyramid structure. It takes the multi-layer feature maps as input and performs scale alignment and step-by-step fusion on feature maps of different scales through top-down paths and lateral connections to generate multi-scale fused features. The enhancement module is used to input the multi-scale fused features into the Transformer module based on the self-attention mechanism, and capture the long-range dependencies and global context information between features through self-attention operation to obtain the enhanced global features; The output module is used to classify the global features and output the classification results of the breast cancer pathology image.

Images