Medical image classification method based on GAShuffleNet network
Through the design of the GAShuffleNet network, the channel shuffling self-attention sub-network and feature fusion residual blocks are used to solve the shortcomings of the existing medical image classification model in terms of accuracy and robustness, and achieve efficient medical image classification.
Patent Information
- Application Number
- CN202410986132.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
The existing medical image classification models have shortcomings in terms of accuracy and robustness, especially the limited receptive field size of the ConvNext network and lack of cross-channel and cross-space interactions, and it is necessary to manually adjust the network architecture to adapt to specific tasks.
The GAShuffleNet network is adopted to enhance feature interoperability and attention mechanisms through channel shuffling of self-attention subnets and feature fusion residual blocks to build an efficient medical image classification model.
It improves the accuracy and robustness of medical image classification, reduces the computational complexity, and achieves the effects of high classification accuracy, strong adaptability and fast learning speed.
Smart Images

Figure CN118941857A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to medical image classification. Background Art
[0002] In recent years, with the rapid development of deep learning technology, neural networks, as its core component, have made significant progress in the field of medical image classification. In this technical field, classification accuracy is not only a key indicator to measure the performance of new network models, but also an important driving force for the development of medical image processing technology.
[0003] Medical image classification is an important research topic in the fields of medical image retrieval and mining. With the development of computer vision, convolutional neural networks (CNNs) have shown excellent performance in natural image classification tasks. Currently, ConvNext networks, ResNet networks, or one of their variants are still used as backbone CNNs in medical image classification. The simple modular design and the ability to effectively extract features can be quickly adapted to various medical image processing. The ConvNext network was originally used for specific image classification, and the receptive field size was limited, lacking cross-channel and cross-space interactions. The network architecture needs to be manually adjusted to modify the ConvNext network to make it more effective for specific tasks.
[0004] Convolutional neural networks (CNNs) and their variants have achieved success in image classification by introducing different attention mechanisms to enhance feature representation. Medical image classification models require a powerful network backbone, an effective feature fusion mechanism, a focused attention mechanism, and relatively low computational complexity. Summary of the invention
[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a medical image classification method based on the GAShuffleNet network with high classification accuracy, strong robustness and adaptability.
[0006] The technical solution adopted to solve the above technical problems consists of the following steps:
[0007] (1) Dataset preprocessing
[0008] From 732 H&E stained full-field digital slice images WSI, these images were scanned at 10x, 20x, and 40x magnifications and annotated by pathologists. In these images, the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels. According to the ratio of 6:1.5:2.5, they were divided into training set, validation set, and test set. The data of the training set was normalized, scaled, center-cropped, and randomly flipped. The data of the validation set and the test set were normalized, scaled, and center-cropped, so that the initial image size of the data set input to the network was 224×224.
[0009] (2) Building the GAShuffleNet network
[0010] The GAShuffleNet network is composed of channel shuffled self-attention sub-network 1, channel shuffled self-attention sub-network 2, channel shuffled self-attention sub-network 3, and channel shuffled self-attention sub-network 4 in series.
[0011] (3) Training the GAShuffleNet network
[0012] 1) Determine the objective function
[0013] The objective function consists of the loss function Loss and the evaluation function AUC.
[0014] Determine the loss function Loss according to formula (1):
[0015]
[0016] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c.
[0017] The evaluation function AUC is determined according to formula (2):
[0018]
[0019] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.
[0020] 2) Training the GAShuffleNet network
[0021] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.
[0022] (4) Save the GAShuffleNet network training model
[0023] In the process of training the GAShuffleNet network, the deep learning framework Pytorch is used to continuously update the weights and save the weight files;
[0024] (5) Verify the GAShuffleNet network
[0025] The validation set is input into the trained GAShuffleNet network for validation.
[0026] (6) Test the GAShuffleNet network
[0027] The test set is input into the trained GAShuffleNet network for testing, and the weight file with the lowest loss function during the saved training process is loaded to obtain the classification result of the image.
[0028] In the GAShuffleNet network model constructed in step (2) of the present invention, the channel shuffled self-attention subnetwork 1 is composed of a backbone module, a feature fusion residual block 1, and a feature fusion residual block 2 connected in series in sequence; the channel shuffled self-attention subnetwork 2 is composed of a downsampling module 1, a feature fusion residual block 3, and a feature fusion residual block 4 connected in series in sequence; the channel shuffled self-attention subnetwork 3 is composed of a downsampling module 2, a feature fusion residual block 5, and a feature fusion residual block 6 connected in series in sequence; the channel shuffled self-attention subnetwork 4 is composed of a downsampling module 3, a feature fusion residual block 7, and a feature fusion residual block 8 connected in series in sequence.
[0029] The feature fusion residual block 1 of the present invention is composed of a convolution module 1, a feature fusion residual sub-module 1, a feature fusion residual sub-module 2, a feature fusion residual sub-module 3, a feature fusion residual sub-module 4, a channel attention module 1, a channel attention module 2, a channel attention module 3, a channel attention module 4, a spatial attention module, a convolution layer 1, and a convolution layer 2.
[0030] The output end of the convolution module 1 is connected to the input end of the feature fusion residual submodule 1, the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 respectively; the feature fusion residual submodule 1 is connected in series with the channel attention module 1, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 2 is connected in series with the channel attention module 2, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 3 is connected in series with the channel attention module 3, the spatial attention module, and the convolution layer 2 in sequence, and the feature fusion residual submodule 4 is connected in series with the channel attention module 4, the spatial attention module, and the convolution layer 2 in sequence; the other output end of the convolution module 1 is connected to the convolution layer 2 through the convolution layer 1; the convolution kernel size of the convolution layer 1 is 3×3, the step size is 1, and the padding is 1, and the convolution kernel size of the convolution layer 2 is 1×1, the step size is 1, and the padding is 0.
[0031] The structures of feature fusion residual block 2, feature fusion residual block 3, feature fusion residual block 4, feature fusion residual block 5, feature fusion residual block 6, feature fusion residual block 7, and feature fusion residual block 8 are the same as the structure of feature fusion residual block 1.
[0032] The downsampling module 1 is composed of a pooling layer with a window size of 2×2, a step size of 2, and a padding of 0; the structures of the downsampling module 2 and the downsampling module 3 are the same as that of the downsampling module 1.
[0033] The feature fusion residual submodule 1 of the present invention is composed of a feature shuffling module 1, a feature shuffling module 2, a feature shuffling module 3, a feature shuffling module 4, a convolution module 2, a convolution module 3, a convolution module 4, a convolution module 5, and a feature fusion module; the feature shuffling module 1 is connected in series with the convolution module 2 and the feature fusion module in sequence, the feature shuffling module 2 is connected in series with the convolution module 3 and the feature fusion module in sequence, the feature shuffling module 3 is connected in series with the convolution module 4 and the feature fusion module in sequence, and the feature shuffling module 4 is connected in series with the convolution module 5 and the feature fusion module in sequence; the structures of the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 are the same as the structure of the feature fusion residual submodule 1.
[0034] The spatial attention module of the present invention is composed of a pooling module and convolutional layer 3, convolutional layer 4, convolutional layer 5, and convolutional layer 6; the output end of the pooling module is connected to the input end of convolutional layer 3, convolutional layer 4, and convolutional layer 5 respectively, and the output ends of convolutional layer 3, convolutional layer 4, and convolutional layer 5 are connected to convolutional layer 6 respectively. The convolution kernel size of convolutional layer 3 is 7×7, the step size is 1, and the padding is 3. The convolution kernel size of convolutional layer 4 is 5×5, the step size is 1, and the padding is 2. The convolution kernel size of convolutional layer 5 is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of convolutional layer 6 is 1×1, the step size is 1, and the padding is 0.
[0035] The construction methods of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3 and the feature shuffling module 4 of the present invention are as follows.
[0036] The feature shuffling module 1 is constructed as follows:
[0037]
[0038] Among them, f1 represents feature shuffling module 1, x mm Represents a feature map, the size of the feature map is m×m, and m is a finite positive integer.
[0039] The feature shuffling module 2 is constructed as follows:
[0040]
[0041] Among them, f2 represents feature shuffling module 2, T represents permutation operation, and the value of m is the same as that of feature shuffling module 1.
[0042] The feature shuffling module 3 is constructed as follows:
[0043]
[0044] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.
[0045] Construct the characteristic shuffle module 4 as follows:
[0046]
[0047] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.
[0048] In the expressions of the feature shuffling module 1, feature shuffling module 2, feature shuffling module 3 and feature shuffling module 4 of the present invention, x mm Represents a feature map, the size of the feature map is m×m, and the preferred value range of m is 38 to 64.
[0049] In the expressions of the feature shuffling module 1, feature shuffling module 2, feature shuffling module 3 and feature shuffling module 4 of the present invention, x mm Represents the feature map, the feature map size is m×m, and the optimal value of m is 56.
[0050] Since the GAShuffleNet network of the present invention adopts a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, and a channel shuffled self-attention sub-network, and the channel shuffled self-attention sub-network adopts a feature shuffle module, in the training step, the fluidity of feature intercommunication is enhanced, and the attention mechanism enhances the important feature weights of the network in training and learning by extracting effective information in the image, thereby improving the accuracy of medical image classification and reducing the complexity of calculation. Compared with the prior art, the present invention has the advantages of high classification accuracy, strong robustness, strong adaptability, fast learning speed, etc., and can be used for medical image classification and can also be used for other image classifications. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 This is a flow chart of Example 1 of the present invention.
[0052] Figure 2 This is a schematic diagram of the structure of the GAShuffleNet network.
[0053] Figure 3 yes Figure 2 Schematic diagram of the structure of feature fusion residual block 1.
[0054] Figure 4 yes Figure 3 Schematic diagram of the structure of the feature fusion residual submodule 1.
[0055] Figure 5 yes Figure 3 Schematic diagram of the structure of the spatial attention module. DETAILED DESCRIPTION
[0056] The present invention is further described below with reference to the accompanying drawings and examples, but the present invention is not limited to the following embodiments.
[0057] Example 1
[0058] Figure 1 A flow chart of this embodiment 1 is given. Figure 1 In the present embodiment, the medical image classification method based on the GAShuffleNet network consists of the following steps:
[0059] (1) Dataset preprocessing
[0060] From 732 H&E stained full-field digital slide images WSI, these images were scanned at 10x, 20x, and 40x magnifications and annotated by pathologists. In these images, the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels. They were divided into training set, validation set, and test set in a ratio of 6:1.5:2.5. The data of the training set was normalized, scaled, center-cropped, and randomly flipped. The data of the validation set and the test set were normalized, scaled, and center-cropped, so that the initial image size of the data set input to the network was 224×224.
[0061] (2) Building the GAShuffleNet network
[0062] Figure 2 A schematic diagram of the structure of the GAShuffleNet network of this embodiment is given. Figure 2 In the embodiment, the GAShuffleNet network is composed of a channel shuffled self-attention sub-network 1, a channel shuffled self-attention sub-network 2, a channel shuffled self-attention sub-network 3, and a channel shuffled self-attention sub-network 4 connected in series.
[0063] The channel shuffled self-attention subnetwork 1 of this embodiment is composed of a backbone module, a feature fusion residual block 1, and a feature fusion residual block 2 connected in series in sequence. The channel shuffled self-attention subnetwork 2 of this embodiment is composed of a downsampling module 1, a feature fusion residual block 3, and a feature fusion residual block 4 connected in series in sequence. The channel shuffled self-attention subnetwork 3 of this embodiment is composed of a downsampling module 2, a feature fusion residual block 5, and a feature fusion residual block 6 connected in series in sequence. The channel shuffled self-attention subnetwork 4 of this embodiment is composed of a downsampling module 3, a feature fusion residual block 7, and a feature fusion residual block 8 connected in series in sequence.
[0064] Figure 3 Given Figure 2 Schematic diagram of the structure of feature fusion residual block 1. Figure 3 In the figure, the feature fusion residual block 1 of this embodiment is composed of a convolution module 1, a feature fusion residual sub-module 1, a feature fusion residual sub-module 2, a feature fusion residual sub-module 3, a feature fusion residual sub-module 4, a channel attention module 1, a channel attention module 2, a channel attention module 3, a channel attention module 4, a spatial attention module, a convolution layer 1, and a convolution layer 2.
[0065] The output end of the convolution module 1 is respectively connected to the input end of the feature fusion residual submodule 1, the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4; the feature fusion residual submodule 1 is connected in series with the channel attention module 1, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 2 is connected in series with the channel attention module 2, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 3 is connected in series with the channel attention module 3, the spatial attention module, and the convolution layer 2 in sequence, and the feature fusion residual submodule 4 is connected in series with the channel attention module 4, the spatial attention module, and the convolution layer 2 in sequence; the other output end of the convolution module 1 is connected to the convolution 2 through the convolution layer 1.
[0066] The structures of feature fusion residual block 2, feature fusion residual block 3, feature fusion residual block 4, feature fusion residual block 5, feature fusion residual block 6, feature fusion residual block 7, and feature fusion residual block 8 are the same as the structure of feature fusion residual block 1.
[0067] Figure 4 Given Figure 3 Schematic diagram of the structure of the feature fusion residual submodule 1. Figure 4 In the embodiment, the feature fusion residual submodule 1 is composed of a feature shuffling module 1, a feature shuffling module 2, a feature shuffling module 3, a feature shuffling module 4, a convolution module 2, a convolution module 3, a convolution module 4, a convolution module 5, and a feature fusion module. The feature shuffling module 1 is connected in series with the convolution module 2 and the feature fusion module in sequence, the feature shuffling module 2 is connected in series with the convolution module 3 and the feature fusion module in sequence, the feature shuffling module 3 is connected in series with the convolution module 4 and the feature fusion module in sequence, and the feature shuffling module 4 is connected in series with the convolution module 5 and the feature fusion module in sequence. The structures of the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 are the same as those of the feature fusion residual submodule 1.
[0068] The construction methods of feature shuffling module 1, feature shuffling module 2, feature shuffling module 3 and feature shuffling module 4 are as follows:
[0069] The feature shuffling module 1 is constructed as follows:
[0070]
[0071] Among them, f1 represents feature shuffling module 1, x mm represents a feature map, the size of the feature map is m×m, m is a finite positive integer, and the value of m in this embodiment is 56.
[0072] The feature shuffling module 2 is constructed as follows:
[0073]
[0074] Among them, f2 represents feature shuffling module 2, T represents permutation operation, and the value of m is the same as that of feature shuffling module 1.
[0075] The feature shuffling module 3 is constructed as follows:
[0076]
[0077] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.
[0078] Construct the characteristic shuffle module 4 as follows:
[0079]
[0080] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.
[0081] Figure 5 Given Figure 3 Schematic diagram of the structure of the spatial attention module in Fig. Figure 5 In the embodiment, the spatial attention module is composed of a pooling module and convolutional layers 3, 4, 5 and 6. The output of the pooling module is connected to the input of convolutional layers 3, 4 and 5 respectively, and the output of convolutional layers 3, 4 and 5 is connected to convolutional layer 6 respectively. The convolution kernel size of convolutional layer 3 is 7×7, the step size is 1, and the padding is 3. The convolution kernel size of convolutional layer 4 is 5×5, the step size is 1, and the padding is 2. The convolution kernel size of convolutional layer 5 is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of convolutional layer 6 is 1×1, the step size is 1, and the padding is 0.
[0082] (3) Training the GAShuffleNet network
[0083] 1) Determine the objective function
[0084] The objective function consists of the loss function Loss and the evaluation function AUC.
[0085] Determine the loss function Loss according to formula (1):
[0086]
[0087]
[0088] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c.
[0089] The evaluation function AUC is determined according to formula (2):
[0090]
[0091]
[0092] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ and P i- The value is 0.5, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.
[0093] 2) Training the GAShuffleNet network
[0094] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], the value of γ in this embodiment is 10 -5 , batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.
[0095] Since the GAShuffleNet network of the present invention adopts a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, and a channel shuffled self-attention sub-network, and the channel shuffled self-attention sub-network adopts a feature shuffle module, in the training step, the fluidity of feature intercommunication is enhanced, and the attention mechanism enhances the important feature weights of the network in training and learning by extracting effective information in the image, thereby improving the accuracy of medical image classification and reducing the complexity of calculation. Compared with the prior art, the present invention has the advantages of high classification accuracy, strong robustness, strong adaptability, fast learning speed, etc., and can be used for medical image classification and can also be used for other image classifications.
[0096] (4) Save the GAShuffleNet network training model
[0097] During the training of the GAShuffleNet network, the deep learning framework Pytorch is used to continuously update the weights and save the weight files.
[0098] (5) Verify the GAShuffleNet network
[0099] The validation set is input into the trained GAShuffleNet network for validation.
[0100] (6) Test the GAShuffleNet network
[0101] The test set is input into the trained GAShuffleNet network for testing, and the weight file with the lowest loss function during the saved training process is loaded to obtain the classification result of the image.
[0102] Complete the medical image classification method based on GAShuffleNet network.
[0103] Example 2
[0104] The medical image classification method based on the GAShuffleNet network of this embodiment consists of the following steps:
[0105] (1) Dataset preprocessing
[0106] This step is the same as in Example 1.
[0107] (2) Building the GAShuffleNet network
[0108] The GAShuffleNet network structure of this embodiment is the same as that of Embodiment 1. In this embodiment, the construction methods of feature shuffling module 1, feature shuffling module 2, feature shuffling module 3, and feature shuffling module 4 are as follows:
[0109] The feature shuffling module 1 is constructed as follows:
[0110]
[0111] Among them, f1 represents feature shuffling module 1, x mm represents a feature map, the size of the feature map is m×m, m is a finite positive integer, and the value of m in this embodiment is 38.
[0112] The feature shuffling module 2 is constructed as follows:
[0113]
[0114] Among them, f2 represents feature shuffling module 2, T represents permutation operation, and the value of m is the same as that of feature shuffling module 1.
[0115] The feature shuffling module 3 is constructed as follows:
[0116]
[0117] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.
[0118] Construct the characteristic shuffle module 4 as follows:
[0119]
[0120] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.
[0121] The other steps of this step are the same as those in Example 1.
[0122] (3) Training the GAShuffleNet network
[0123] 1) Determine the objective function
[0124] The objective function consists of the loss function Loss and the evaluation function AUC.
[0125] Determine the loss function Loss according to formula (1):
[0126]
[0127]
[0128] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c;
[0129] The evaluation function AUC is determined according to formula (2).
[0130]
[0131]
[0132] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ and P i- The value is 0, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.
[0133] 2) Training the GAShuffleNet network
[0134] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], the value of γ in this embodiment is 10 -6 , batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.
[0135] The other steps are the same as those in Example 1. A medical image classification method based on the GAShuffleNet network is completed.
[0136] Example 3
[0137] The medical image classification method based on the GAShuffleNet network of this embodiment consists of the following steps:
[0138] (1) Dataset preprocessing
[0139] This step is the same as in Example 1.
[0140] (2) Building the GAShuffleNet network
[0141] The GAShuffleNet network structure of this embodiment is the same as that of Embodiment 1. In this embodiment, the construction method of feature fusion residual submodule 1, feature fusion residual submodule 2, feature fusion residual submodule 3, and feature fusion residual submodule 4 is as follows:
[0142] The feature shuffling module 1 is constructed as follows:
[0143]
[0144] Wherein, f1 represents the feature map in the feature shuffling module 1, the size of the feature map is m×m, m is a finite positive integer, and the value of m in this embodiment is 64.
[0145] The feature shuffling module 2 is constructed as follows:
[0146]
[0147] Wherein, f2 represents feature shuffle module 2, T represents permutation operation, and the value of m is the same as that of feature shuffle module 1;
[0148] The feature shuffling module 3 is constructed as follows:
[0149]
[0150] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.
[0151] Construct the characteristic shuffle module 4 as follows:
[0152]
[0153] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.
[0154] The other steps of this step are the same as those in Example 1.
[0155] (3) Training the GAShuffleNet network
[0156] 1) Determine the objective function
[0157] The objective function consists of the loss function Loss and the evaluation function AUC.
[0158] Determine the loss function Loss according to formula (1):
[0159]
[0160]
[0161] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c;
[0162] The evaluation function AUC is determined according to formula (2):
[0163]
[0164]
[0165] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ and P i- The value is 1, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers;
[0166] 2) Training the GAShuffleNet network
[0167] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], the value of γ in this embodiment is 10 -4 , batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.
[0168] The other steps are the same as those in Example 1. A medical image classification method based on the GAShuffleNet network is completed.
[0169] Example 4
[0170] In the above embodiments 1 to 3, the medical image classification method based on the GAShuffleNet network of this embodiment consists of the following steps:
[0171] (1) Dataset preprocessing
[0172] This step is the same as in Example 1.
[0173] (2) Building the GAShuffleNet network
[0174] This step is the same as in Example 1.
[0175] (3) Training the GAShuffleNet network
[0176] 1) Determine the objective function
[0177] The objective function is composed of a loss function Loss and an evaluation function AUC; the expression of the loss function Loss, the meaning of the parameters in the expression, and the value range are the same as those in the corresponding embodiment. In the expression (2) of the evaluation function AUC, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ The value is 0.6, and P i- The value is 0.4, M is the number of positive samples, N is the number of negative samples, M and N are finite positive integers;
[0178] 2) Training the GAShuffleNet network
[0179] This step is the same as that in the corresponding embodiment.
[0180] The other steps are the same as those in Example 1. A medical image classification method based on the GAShuffleNet network is completed.
[0181] In order to verify the beneficial effects of the present invention, the inventors conducted comparative simulation experiments using the medical image classification method based on the GAShuffleNet network of Example 1 of the present invention and the convnext_tiny method and the convnext_small method. The various experimental results are as follows:
[0182] The same test set is tested with each trained model, and the evaluation code is used to test the model accuracy. The evaluation function AUC is used as the indicator of the evaluation method. The larger the evaluation function AUC, the better the method. The experimental results of the evaluation function AUC are shown in Table 1.
[0183] Table 1 Evaluation function AUC values of Example 1 and comparative methods
[0184] Test Method Evaluation function AUC Example 1 0.781 convnext_tiny method 0.734 convnext_small method 0.722
[0185] As can be seen from Table 1, the evaluation function AUC value of the method in Example 1 is 0.781, the evaluation function AUC value of the convnext_tiny method is 0.734, and the evaluation function AUC value of the convnext_small method is 0.722. The evaluation function AUC value of the method in Example 1 is 4.7% higher than the evaluation function AUC value of the convnext_tiny method and 5.9% higher than the evaluation function AUC value of the convnext_small method.
Claims
1. A medical image classification method based on GAShuffleNet network, characterized in that It consists of the following steps: (1) Dataset preprocessing From 732 H&E stained full-field digital slide images WSI, these images were scanned at 10x, 20x, and 40x magnifications and annotated by pathologists. In these images, the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels. They were divided into training set, validation set, and test set in a ratio of 6:1.5:2.
5. The data of the training set was normalized, scaled, center-cropped, and randomly flipped. The data of the validation set and the test set were normalized, scaled, and center-cropped, so that the initial image size of the data set input to the network was 224×224; (2) Building the GAShuffleNet network The GAShuffleNet network is composed of channel shuffle self-attention sub-network 1, channel shuffle self-attention sub-network 2, channel shuffle self-attention sub-network 3, and channel shuffle self-attention sub-network 4 in series; (3) Training the GAShuffleNet network 1) Determine the objective function The objective function consists of the loss function Loss and the evaluation function AUC; Determine the loss function Loss according to formula (1): Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c; The evaluation function AUC is determined according to formula (2): Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], M is the number of positive samples, N is the number of negative samples, M and N are finite positive integers; 2) Training the GAShuffleNet network The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges; (4) Save the GAShuffleNet network training model In the process of training the GAShuffleNet network, the deep learning framework Pytorch is used to continuously update the weights and save the weight files; (5) Verify the GAShuffleNet network Input the validation set into the trained GAShuffleNet network for validation; (6) Test the GAShuffleNet network The test set is input into the trained GAShuffleNet network for testing, and the weight file with the lowest loss function during the saved training process is loaded to obtain the classification result of the image.
2. The medical image classification method based on the GAShuffleNet network according to claim 1, characterized in that: In the GAShuffleNet network model constructed in step (2), the channel shuffled self-attention subnetwork 1 is composed of a backbone module, a feature fusion residual block 1, and a feature fusion residual block 2 connected in series in sequence; the channel shuffled self-attention subnetwork 2 is composed of a downsampling module 1, a feature fusion residual block 3, and a feature fusion residual block 4 connected in series in sequence; the channel shuffled self-attention subnetwork 3 is composed of a downsampling module 2, a feature fusion residual block 5, and a feature fusion residual block 6 connected in series in sequence; the channel shuffled self-attention subnetwork 4 is composed of a downsampling module 3, a feature fusion residual block 7, and a feature fusion residual block 8 connected in series in sequence.
3. The image classification method based on the GAShuffleNet network according to claim 2, characterized in that: The feature fusion residual block 1 is composed of a convolution module 1, a feature fusion residual sub-module 1, a feature fusion residual sub-module 2, a feature fusion residual sub-module 3, a feature fusion residual sub-module 4, a channel attention module 1, a channel attention module 2, a channel attention module 3, a channel attention module 4, a spatial attention module, a convolution layer 1, and a convolution layer 2. The output end of the convolution module 1 is connected to the input end of the feature fusion residual submodule 1, the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 respectively; the feature fusion residual submodule 1 is connected in series with the channel attention module 1, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 2 is connected in series with the channel attention module 2, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 3 is connected in series with the channel attention module 3, the spatial attention module, and the convolution layer 2 in sequence, and the feature fusion residual submodule 4 is connected in series with the channel attention module 4, the spatial attention module, and the convolution layer 2 in sequence; the other output end of the convolution module 1 is connected to the convolution layer 2 through the convolution layer 1; the convolution kernel size of the convolution layer 1 is 3×3, the step size is 1, and the padding is 1, and the convolution kernel size of the convolution layer 2 is 1×1, the step size is 1, and the padding is 0; The structures of feature fusion residual block 2, feature fusion residual block 3, feature fusion residual block 4, feature fusion residual block 5, feature fusion residual block 6, feature fusion residual block 7, and feature fusion residual block 8 are the same as the structure of feature fusion residual block 1. The downsampling module 1 is composed of a pooling layer with a window size of 2×2, a step size of 2, and a padding of 0; the structures of the downsampling module 2 and the downsampling module 3 are the same as that of the downsampling module 1.
4. The image classification method based on the GAShuffleNet network according to claim 3, characterized in that: The feature fusion residual submodule 1 is composed of a feature shuffling module 1, a feature shuffling module 2, a feature shuffling module 3, a feature shuffling module 4, a convolution module 2, a convolution module 3, a convolution module 4, a convolution module 5, and a feature fusion module; the feature shuffling module 1 is connected in series with the convolution module 2 and the feature fusion module in sequence, the feature shuffling module 2 is connected in series with the convolution module 3 and the feature fusion module in sequence, the feature shuffling module 3 is connected in series with the convolution module 4 and the feature fusion module in sequence, and the feature shuffling module 4 is connected in series with the convolution module 5 and the feature fusion module in sequence; the structures of the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 are the same as the structure of the feature fusion residual submodule 1.
5. The image classification method based on the GAShuffleNet network according to claim 3, characterized in that: The spatial attention module is composed of a pooling module and convolutional layer 3, convolutional layer 4, convolutional layer 5, and convolutional layer 6; the output end of the pooling module is connected to the input end of convolutional layer 3, convolutional layer 4, and convolutional layer 5 respectively, and the output ends of convolutional layer 3, convolutional layer 4, and convolutional layer 5 are connected to convolutional layer 6 respectively. The convolution kernel size of convolutional layer 3 is 7×7, the step size is 1, and the padding is 3. The convolution kernel size of convolutional layer 4 is 5×5, the step size is 1, and the padding is 2. The convolution kernel size of convolutional layer 5 is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of convolutional layer 6 is 1×1, the step size is 1, and the padding is 0.
6. The image classification method based on the GAShuffleNet network according to claim 4, characterized in that The construction methods of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3, and the feature shuffling module 4 are as follows: The feature shuffling module 1 is constructed as follows: Among them, f1 represents feature shuffling module 1, x mm Represents a feature map, the size of the feature map is m×m, where m is a finite positive integer; The feature shuffling module 2 is constructed as follows: Wherein, f2 represents feature shuffle module 2, T represents permutation operation, and the value of m is the same as that of feature shuffle module 1; The feature shuffling module 3 is constructed as follows: Among them, f3 represents feature shuffle module 3, F represents flip operation, and the value of m is the same as that of feature shuffle module 1; Construct the characteristic shuffle module 4 as follows: Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.
7. The image classification method based on the GAShuffleNet network according to claim 6, characterized in that: In the expressions of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3 and the feature shuffling module 4, x mm Represents the feature map. The size of the feature map is m×m, and the value of m is 38 to 64.
8. The image classification method based on the GAShuffleNet network according to claim 6, characterized in that: In the expressions of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3 and the feature shuffling module 4, x mm Represents the feature map, the feature map size is m×m, and the value of m is 56.
Citation Information
Cited By
Image classification method based on SN-HiFuse network
CN116824245A
Image classification method based on sn-hifuse network
CN116824245B
