Breast cancer pathology image classification method based on SENet channel attention and transfer learning
Through the feature extraction method combined with wavelet transformation and transfer learning, combined with the SENet channel attention mechanism, the problem of feature loss and overfitting in breast cancer pathological image classification is solved, and the classification of breast cancer pathological image is improved Accuracy.
Patent Information
- Application Number
- CN202210514927.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-11
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-05-11
AI Technical Summary
Traditional convolutional neural networks are prone to loss of pathological features in breast cancer pathological image classification, resulting in overfitting of the model and requiring a large amount of labeled data. Pathological examinations rely on doctors' experience and are inefficient.
A feature extraction method combined with wavelet transformation and transfer learning is adopted, and feature fusion is combined with the SENet channel attention mechanism to train the classifier to improve the classification accuracy of breast cancer pathological images.
By retaining low-frequency detail components and extracting high-frequency detail components using transfer learning, the SENet channel attention mechanism strengthens feature weights, improves the model resolution and breast cancer recognition accuracy, and overcomes the problems of data scarcity and overfitting.
Smart Images

Figure CN114820555B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a breast cancer pathology image classification method, and more specifically, to a breast cancer pathology image classification method based on SENet channel attention and transfer learning. Background Art
[0002] Breast disease is one of the major diseases that endanger women's health. Breast tumor is the most common breast disease, which can be divided into benign tumor and malignant tumor. Malignant tumor includes carcinoma (DO), lobular carcinoma (LC), mucinous carcinoma (MC), papillary carcinoma (PC). Breast cancer is the second leading cause of cancer death among women worldwide. Common examinations for breast cancer include mammography, ultrasound, breast MRI, and pathological examination, among which pathological examination is considered the gold standard for diagnosing breast cancer. However, pathological examination is often limited by subjective factors such as the experience of pathologists, and there is a shortage of pathologists in my country. Pathological examination is a labor-intensive and time-consuming task. These factors lead to low efficiency of pathological diagnosis. Automatically classifying breast pathological images into benign and malignant with the help of computer-aided diagnosis (CAD) can not only improve diagnostic efficiency but also provide doctors with more objective and accurate diagnostic results, so it has important clinical application value.
[0003] In recent years, with the development of machine learning and deep learning, research on deep learning-based breast cancer pathology image diagnosis has been carried out. Convolutional Neural Network (CNN) has gained an important position in medical image research. Classic neural network models include AlexNet, Inception, Residual Networks (ResNet) and Xception.
[0004] One of the most important components of traditional convolutional neural networks (CNNs) is the pooling layer. There are two types of traditional pooling layers: average pooling and maximum pooling. The purpose of the pooling layer is to reduce the size and parameters of the image, improve computational efficiency, and improve the robustness of the model. However, traditional pooling is prone to losing the features of pathological images and causing overfitting of the model. In order to solve the problem of model overfitting, a large number of pathological images are required as training sets. However, the cost of obtaining a large number of labeled breast cancer images is expensive. These problems have brought challenges to the accuracy of breast cancer pathology images based on convolutional neural networks. Summary of the invention
[0005] In order to overcome the shortcomings of the above technical problems, the present invention provides a breast cancer pathology image classification method based on SENet channel attention and transfer learning.
[0006] The breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention is characterized in that the breast cancer pathology image classification method is: firstly, feature extraction of breast cancer pathology images is performed by combining wavelet transform and transfer learning, and then the features are fused by using a method based on SENet channel attention mechanism, and finally a classifier with performance meeting the requirements is trained, so as to effectively classify breast cancer pathology images by using the trained classifier.
[0007] The breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention is specifically implemented by the following steps:
[0008] a). Construct a breast cancer pathology image dataset, and use the image data enhancement method to rotate and scale the breast cancer pathology dataset published by BreakHis to expand the breast cancer pathology image dataset;
[0009] b). Wavelet transform of the image: discrete wavelet transform is performed on the image in the data set to obtain four detail components including low-frequency detail component, horizontal detail component, diagonal detail component and vertical detail component;
[0010] c) Feature extraction, the feature extraction includes a low-frequency part and a transfer learning part. The low-frequency part is: the low-frequency detail component obtained by discrete wavelet transform in step b) is directly used as a sub-feature, and the transfer learning part is: using the transfer learning method to extract sub-features of horizontal detail components, diagonal detail components and vertical detail components;
[0011] d) Feature fusion: The SENet channel attention mechanism is used to fuse the sub-features extracted from the low-frequency part and the transfer learning part in step c), and the SENet channel attention mechanism can learn the characteristics of important features to improve the accuracy of breast cancer recognition;
[0012] e). Training component: input the fused sub-features into the SVM classifier for training component. In order to reduce overfitting during the training component process, the K-fold cross validation method is used for the data set;
[0013] f). Obtain the classifier. In order to better evaluate the performance of the classifier, three criteria, namely accuracy, specificity and sensitivity, are used to evaluate the performance of the classifier, as shown in formula (1), formula (2) and formula (3) respectively:
[0014]
[0015] Among them, P is the number of malignant specimens, N is the number of benign specimens, TP is the number of correctly classified malignant specimens, TN is the number of correctly classified benign specimens, FP is the number of misclassified benign specimens, and FN is the number of misclassified malignant specimens;
[0016] Through training, when the three standards of accuracy, specificity and sensitivity of the classifier all reach the set thresholds, it is considered that a classifier model with satisfactory performance is obtained; the trained classifier model is used to classify benign tumors and malignant tumors in the pathological images of breast cancer to be diagnosed, so as to assist doctors in diagnosis.
[0017] The breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention, in the process of constructing a breast cancer pathology image dataset as described in step a): firstly, the breast cancer pathology image is pruned according to a certain ratio, and the pruned image is rotated 90° and 180° to expand the breast cancer pathology image dataset, and the constructed breast cancer pathology dataset is X=[X1,X2,X3,X4,...X n ], label data set Y = [Y1, Y2], Y equals 0 for benign, Y equals 1 for malignant; the samples in the data set correspond to the labels one by one.
[0018] In the breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention, in the wavelet transform process of the image described in step b), the image in the data set is subjected to a first-level wavelet decomposition transform, as shown in formula (4) and formula (5):
[0019]
[0020] Where: H, V, D represent the horizontal detail component, diagonal detail component and vertical detail component respectively, M, N are the feature dimensions, m, n are the wavelet positions; is the approximation of the image at scale j0, which is a low-frequency coefficient; f(x, y) is the feature map of the image at (x, y), j and j0 represent the order, j0 is an arbitrary starting scale, The horizontal, vertical and diagonal details are added, which are high-frequency coefficients; is the scaling function, is the wavelet function, which is obtained by formula (6) and formula (7) respectively:
[0021]
[0022] In the breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention, the sub-feature extraction of the horizontal detail component, the diagonal detail component and the vertical detail component using the transfer learning method described in step c) is achieved by the following method:
[0023] c-1). Get the Xception network model, keep the 3*3 convolution of the Inception module in Inception V3, concatenate the 1*1 convolutions, and remove the average pooling layer; increase the number of branches of the 3*3 convolution kernel to be equal to the number of channels output by the 1*1 convolution kernel, so that each 3*3 convolution kernel only operates on the feature map containing one channel;
[0024] c-2). Use the ImageNet dataset containing 100 categories and 1.2 million images to pre-train a deep convolutional neural network, then use the Xception network model obtained in step c-1) as a feature extractor, and use the horizontal detail component, diagonal detail component, and vertical detail component after discrete wavelet decomposition as input for feature extraction.
[0025] The breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention, the feature fusion using SENet channel attention mechanism in step d) is specifically implemented by the following steps:
[0026] d-1). First, perform global average pooling on the input feature map, get 1 vector for each channel, and output a 1x 1x C vector, where C represents the number of channels;
[0027] d-2). Then two fully connected layers FC generate weights for each feature channel, and Sigmoid is used to limit the weight value between 0 and 1. The purpose is to make the features have the same numerical magnitude and reduce the influence of features with large variance;
[0028] d-3).Finally, the output feature weights are weighted with the original features.
[0029] The beneficial effects of the present invention are as follows: the breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention proposes a breast cancer pathology image classification method based on SENet channel attention and transfer learning, and its beneficial effects are embodied in: first, for the feature extraction of the image, it is carried out by combining wavelet decomposition and transfer learning. After the image is decomposed by discrete wavelet, the low-frequency detail component is retained, and the remaining components are used as feature extractors using a model pre-trained by transfer learning, so as to obtain more feature information and overcome the problem of few specimens; second, the extracted features are fused based on SENet channel attention. The channel attention mechanism makes it easier to learn important features and strengthen important feature weights, thereby improving the resolution of the model and making machine learning better in the medical field. It can be seen that it solves the problem of low classification accuracy caused by the lack of breast cancer pathology images, the loss of pathology image features in traditional deep convolutional neural networks, and the overfitting of models. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic diagram of the breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention;
[0031] Figure 2 Schematic diagram of the principle of discrete wavelet decomposition transformation in the present invention;
[0032] Figure 3 The schematic diagram of the Xception network model obtained by using the Inception module in Inception V3 in the present invention;
[0033] Figure 4 This is a schematic diagram of the principle of feature fusion using the SENet channel attention mechanism in the present invention. DETAILED DESCRIPTION
[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0035] like Figure 1 As shown, a schematic diagram of the breast cancer pathology image classification method based on SENet channel attention and transfer learning of the present invention is given. In order to solve the problems of lack of breast cancer pathology data set and loss of image detail features, the present invention proposes a breast cancer pathology image classification method based on SENet channel attention and transfer learning, uses discrete wavelet decomposition and transfer learning methods to extract image features, and uses SENet channel attention mechanism to perform feature fusion, thereby improving the classification accuracy of breast cancer pathology images.
[0036] The invented breast cancer pathology image classification method based on SENet channel attention and transfer learning is implemented by the following steps:
[0037] a). Construct a breast cancer pathology image dataset, and use the image data enhancement method to rotate and scale the breast cancer pathology dataset published by BreakHis to expand the breast cancer pathology image dataset;
[0038] The process of constructing the breast cancer pathology image dataset in this step is as follows: first, the breast cancer pathology images are trimmed according to a certain ratio, and the trimmed images are rotated 90° and 180° to expand the breast cancer pathology image dataset. The constructed breast cancer pathology dataset is X=[X1,X2,X3,X4,……X n ], label data set Y = [Y1, Y2], Y equals 0 for benign, Y equals 1 for malignant; the samples in the data set correspond to the labels one by one.
[0039] b). Wavelet transform of the image: discrete wavelet transform is performed on the image in the data set to obtain four detail components including low-frequency detail component, horizontal detail component, diagonal detail component and vertical detail component;
[0040] like Figure 2 As shown, a schematic diagram of the principle of discrete wavelet decomposition transformation in the present invention is given. The specific method is: perform a first-level wavelet decomposition transformation on the images in the data set, as shown in formula (4) and formula (5):
[0041]
[0042] Where: H, V, D represent the horizontal detail component, diagonal detail component and vertical detail component respectively, M, N are the feature dimensions, m, n are the wavelet positions; is the approximation of the image at scale j0, which is a low-frequency coefficient; f(x, y) is the feature map of the image at (x, y), j and j0 represent the order, j0 is an arbitrary starting scale, The horizontal, vertical and diagonal details are added, which are high-frequency coefficients; is the scaling function, is the wavelet function, which is obtained by formula (6) and formula (7) respectively:
[0043]
[0044]
[0045] c) Feature extraction, the feature extraction includes a low-frequency part and a transfer learning part. The low-frequency part is: the low-frequency detail component obtained by discrete wavelet transform in step b) is directly used as a sub-feature, and the transfer learning part is: using the transfer learning method to extract sub-features of horizontal detail components, diagonal detail components and vertical detail components;
[0046] The result of the low-frequency detail component is similar to the image and is directly retained as the sub-feature result. The horizontal detail component, diagonal detail component, and vertical detail component are extracted as sub-features using transfer learning. The Xception network model pre-trained based on the ImageNet dataset is used for sub-feature extraction by transfer learning.
[0047] like Figure 3 As shown, the principle diagram of the Xception network model obtained by using the Inception module in Inception V3 in the present invention is given. In this step, the sub-feature extraction of the horizontal detail component, the diagonal detail component and the vertical detail component using the transfer learning method is achieved by the following method:
[0048] c-1). Get the Xception network model, keep the 3*3 convolution of the Inception module in Inception V3, concatenate the 1*1 convolutions, and remove the average pooling layer; increase the number of branches of the 3*3 convolution kernel to be equal to the number of channels output by the 1*1 convolution kernel, so that each 3*3 convolution kernel only operates on the feature map containing one channel;
[0049] c-2). Use the ImageNet dataset containing 100 categories and 1.2 million images to pre-train a deep convolutional neural network, then use the Xception network model obtained in step c-1) as a feature extractor, and use the horizontal detail component, diagonal detail component, and vertical detail component after discrete wavelet decomposition as input for feature extraction.
[0050] The Xception network uses depth-wise separable convolution to replace the Inception module. Compared with ordinary convolution, the parameters and computational complexity of depth-wise separable convolution are reduced.
[0051] d) Feature fusion: The SENet channel attention mechanism is used to fuse the sub-features extracted from the low-frequency part and the transfer learning part in step c), and the SENet channel attention mechanism can learn the characteristics of important features to improve the accuracy of breast cancer recognition;
[0052] Traditional convolutional neural networks are prone to losing a lot of pathological image feature information, resulting in low classification accuracy and difficulty in model convergence. One of the key points of the present invention is to improve feature extraction. Feature extraction includes two parts, a low-frequency part and a transfer learning part. SENet channel attention mechanism is a lightweight network. SENet obtains the importance of each feature map by adjusting the channel weights of the feature map, emphasizing important feature maps with large weights and weakening feature maps with small weights, so that the model can achieve better results.
[0053] In this step, the feature fusion using the SENet channel attention mechanism is specifically implemented by the following steps:
[0054] d-1). First, perform global average pooling on the input feature map, get 1 vector for each channel, and output a 1x 1x C vector, where C represents the number of channels;
[0055] d-2). Then two fully connected layers FC generate weights for each feature channel, and Sigmoid is used to limit the weight value between 0 and 1. The purpose is to make the features have the same numerical magnitude and reduce the influence of features with large variance;
[0056] d-3).Finally, the output feature weights are weighted with the original features.
[0057] e). Training component: input the fused sub-features into the SVM classifier for training component. In order to reduce overfitting during the training component process, the K-fold cross validation method is used for the data set;
[0058] f). Obtain the classifier. In order to better evaluate the performance of the classifier, three criteria, namely accuracy, specificity and sensitivity, are used to evaluate the performance of the classifier, as shown in formula (1), formula (2) and formula (3) respectively:
[0059]
[0060] Among them, P is the number of malignant specimens, N is the number of benign specimens, TP is the number of correctly classified malignant specimens, TN is the number of correctly classified benign specimens, FP is the number of misclassified benign specimens, and FN is the number of misclassified malignant specimens;
[0061] Through training, when the three standards of accuracy, specificity and sensitivity of the classifier all reach the set thresholds, it is considered that a classifier model with satisfactory performance is obtained; the trained classifier model is used to classify benign tumors and malignant tumors in the pathological images of breast cancer to be diagnosed, so as to assist doctors in diagnosis.
[0062] After feature fusion based on SENet channel attention mechanism, the feature values are input into the SVM classifier for training and classification. The SVM algorithm has advantages in nonlinear and high-dimensional pattern applications. For binary classification problems, the optimal hyperplane can be found to separate the two categories, so this classifier uses the SVM algorithm. When performing classification training, the k-fold cross-validation method is used. The data is randomly generated into 5 folds while ensuring that the percentage of each folded subset and the original data category sample is equal. 80% of the data is used as the training set, and the remaining 20% of the data is used as the test set to prevent the model from overfitting. The algorithm of k-fold cross-validation is as follows:
[0063] Import the dataset,
[0064] Divide the data set into K parts,
[0065] for i in K:
[0066] The i-th part is the validation set: validation_data = data[mun_validation_samples*fold:mun_validation_samples*(fold+1)]
[0067] The other K-1 parts are used as training sets:
[0068] training_data=np.row_stack((data[:mun_validation_samples*fold],data[mu n_validation_samples*(fold+1):]))
[0069] Training model: model.train(training_data)
[0070] Use the validation set to get the evaluation score: validation_score = model.evaluate(validation_data)
[0071] Get the average of K scores: validation_scores = np.average(validation_score)
[0072] Step 5: Classification Model Evaluation
[0073] The classification performance was evaluated using three criteria: accuracy, specificity, and sensitivity. Specificity was used to determine the true benign rate, and sensitivity represented the proportion of malignant tumors that were correctly judged.
Claims
1. A breast cancer pathology image classification method based on SENet channel attention and transfer learning, It is characterized in that The breast cancer pathology image classification method is as follows: firstly, wavelet transform and transfer learning are combined to extract the features of breast cancer pathology images, then the features are fused by SENet channel attention mechanism, and finally a classifier with satisfactory performance is trained, so as to effectively classify breast cancer pathology images by using the trained classifier. The breast cancer pathology image classification method is implemented by the following steps: a). Construct a breast cancer pathology image dataset, and use the image data enhancement method to rotate and scale the breast cancer pathology dataset published by BreakHis to expand the breast cancer pathology image dataset; b). Wavelet transform of the image: discrete wavelet transform is performed on the image in the data set to obtain four detail components including low-frequency detail component, horizontal detail component, diagonal detail component and vertical detail component; c) Feature extraction, the feature extraction includes a low-frequency part and a transfer learning part. The low-frequency part is: the low-frequency detail component obtained by discrete wavelet transform in step b) is directly used as a sub-feature, and the transfer learning part is: using the transfer learning method to extract sub-features of horizontal detail components, diagonal detail components and vertical detail components; d) Feature fusion: The SENet channel attention mechanism is used to fuse the sub-features extracted from the low-frequency part and the transfer learning part in step c), and the SENet channel attention mechanism can learn the characteristics of important features to improve the accuracy of breast cancer recognition; e). Training component: input the fused sub-features into the SVM classifier for training component. In order to reduce overfitting during the training component process, the K-fold cross validation method is used for the data set; f). Obtain the classifier. In order to better evaluate the performance of the classifier, three criteria, namely accuracy, specificity and sensitivity, are used to evaluate the performance of the classifier, as shown in formula (1), formula (2) and formula (3) respectively: Among them, P is the number of malignant specimens, N is the number of benign specimens, TP is the number of correctly classified malignant specimens, TN is the number of correctly classified benign specimens, FP is the number of misclassified benign specimens, and FN is the number of misclassified malignant specimens; Through training, when the three standards of accuracy, specificity and sensitivity of the classifier all reach the set thresholds, it is considered that a classifier model with satisfactory performance is obtained; the trained classifier model is used to classify benign tumors and malignant tumors in the pathological images of breast cancer to be diagnosed, so as to assist doctors in diagnosis.
2. The breast cancer pathology image classification method based on SENet channel attention and transfer learning according to claim 1, It is characterized in that In the process of constructing the breast cancer pathology image dataset described in step a), firstly, the breast cancer pathology image is pruned according to a certain ratio, and the pruned image is rotated 90° and 180° to expand the breast cancer pathology image dataset. The constructed breast cancer pathology dataset is X=[X 1 ,X 2 ,X 3 ,X 4 ,……X n ], label data set Y = [Y 1 ,Y 2 ], Y equals 0 for benign, and Y equals 1 for malignant; the samples in the data set correspond to the labels one by one.
3. The breast cancer pathology image classification method based on SENet channel attention and transfer learning according to claim 1, Features: In the wavelet transform process of the image described in step b), the image in the data set is subjected to a first-level wavelet decomposition transform, as shown in formula (4) and formula (5): Where: H, V, D represent the horizontal detail component, diagonal detail component and vertical detail component respectively, M, N are the feature dimensions, m, n are the wavelet positions; is the image at scale j 0 The approximate value at (x, y) is the low-frequency coefficient; f(x, y) is the feature map of the image at (x, y), j and j 0 represents the order, j 0 is an arbitrary starting scale, The horizontal, vertical and diagonal details are added, which are high-frequency coefficients; is the scaling function, is the wavelet function, which is obtained by formula (6) and formula (7) respectively:
4. The breast cancer pathology image classification method based on SENet channel attention and transfer learning according to claim 1 or 3, It is characterized in that The sub-feature extraction of the horizontal detail component, the diagonal detail component and the vertical detail component using the transfer learning method described in step c) is achieved by the following method: c-1). Get the Xception network model, keep the 3*3 convolution of the Inception module in Inception V3, concatenate the 1*1 convolutions, and remove the average pooling layer; increase the number of branches of the 3*3 convolution kernel to be equal to the number of channels output by the 1*1 convolution kernel, so that each 3*3 convolution kernel only operates on the feature map containing one channel; c-2). Use the ImageNet dataset containing 100 categories and 1.2 million images to pre-train a deep convolutional neural network, then use the Xception network model obtained in step c-1) as a feature extractor, and use the horizontal detail component, diagonal detail component, and vertical detail component after discrete wavelet decomposition as input for feature extraction.
5. The breast cancer pathology image classification method based on SENet channel attention and transfer learning according to claim 1 or 3, It is characterized in that The feature fusion using the SENet channel attention mechanism described in step d) is specifically implemented by the following steps: d-1). First, perform global average pooling on the input feature map, get 1 vector for each channel, and output a 1x1xC vector, where C represents the number of channels; d-2). Then two fully connected layers FC generate weights for each feature channel, and Sigmoid is used to limit the weight value between 0 and 1. The purpose is to make the features have the same numerical magnitude and reduce the influence of features with large variance; d-3).Finally, the output feature weights are weighted with the original features.
Citation Information
Patent Citations
Retina OCT image classification method based on wavelet transform and transfer learning
CN110472530A
Method for realizing multi-channel convolutional-recurrent neural network electroencephalogram emotion recognition model by using transfer learning
CN113627518A
