Medical image classification method based on GAShuffleNet network

Through the design of the GAShuffleNet network, the channel shuffling self-attention sub-network and feature fusion residual blocks are used to solve the shortcomings of the existing medical image classification model in terms of accuracy and robustness, and achieve efficient medical image classification.

CN118941857APending Publication Date: 2024-11-12SHAANXI NORMAL UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410986132.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing medical image classification models have shortcomings in terms of accuracy and robustness, especially the limited receptive field size of the ConvNext network and lack of cross-channel and cross-space interactions, and it is necessary to manually adjust the network architecture to adapt to specific tasks.

Method used

The GAShuffleNet network is adopted to enhance feature interoperability and attention mechanisms through channel shuffling of self-attention subnets and feature fusion residual blocks to build an efficient medical image classification model.

Benefits of technology

It improves the accuracy and robustness of medical image classification, reduces the computational complexity, and achieves the effects of high classification accuracy, strong adaptability and fast learning speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941857A_ABST
    Figure CN118941857A_ABST
Patent Text Reader

Abstract

The invention discloses a medical image classification method based on a GAShuffleNet network. The medical image classification method based on the GAShuffleNet network comprises the steps of preprocessing a data set, constructing the GAShuffleNet network, training the GAShuffleNet network, storing a training model of the GAShuffleNet network, verifying the GAShuffleNet network and testing the GAShuffleNet network. The GAShuffleNet adopts a channel shuffling self-attention sub-network, a channel shuffling self-attention sub-network, a channel shuffling self-attention sub-network and a channel shuffling self-attention sub-network, wherein the channel shuffling self-attention sub-network adopts a feature shuffling module. In the training step, the mobility of feature intercommunication is enhanced, and meanwhile, the attention mechanism enables the network to enhance important feature weights in training learning by paying attention to effective information in the images, so that the precision of medical image classification is improved, and the complexity of calculation is reduced. Compared with the prior art, the method has the advantages of high classification accuracy, high robustness, high adaptability, high learning speed and the like, can be used for medical image classification and can also be used for other image classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to medical image classification. Background Art

[0002] In recent years, with the rapid development of deep learning technology, neural networks, as its core component, have made significant progress in the field of medical image classification. In this technical field, classification accuracy is not only a key indicator to measure the performance of new network models, but also an important driving force for the development of medical image processing technology.

[0003] Medical image classification is an important research topic in the fields of medical image retrieval and mining. With the development of computer vision, convolutional neural networks (CNNs) have shown excellent performance in natural image classification tasks. Currently, ConvNext networks, ResNet networks, or one of their variants are still used as backbone CNNs in medical image classification. The simple modular design and the ability to effectively extract features can be quickly adapted to various medical image processing. The ConvNext network was originally used for specific image classification, and the receptive field size was limited, lacking cross-channel and cross-space interactions. The network architecture needs to be manually adjusted to modify the ConvNext network to make it more effective for specific tasks.

[0004] Convolutional neural networks (CNNs) and their variants have achieved success in image classification by introducing different attention mechanisms to enhance feature representation. Medical image classification models require a powerful network backbone, an effective feature fusion mechanism, a focused attention mechanism, and relatively low computational complexity. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a medical image classification method based on the GAShuffleNet network with high classification accuracy, strong robustness and adaptability.

[0006] The technical solution adopted to solve the above technical problems consists of the following steps:

[0007] (1) Dataset preprocessing

[0008] From 732 H&E stained full-field digital slice images WSI, these images were scanned at 10x, 20x, and 40x magnifications and annotated by pathologists. In these images, the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels. According to the ratio of 6:1.5:2.5, they were divided into training set, validation set, and test set. The data of the training set was normalized, scaled, center-cropped, and randomly flipped. The data of the validation set and the test set were normalized, scaled, and center-cropped, so that the initial image size of the data set input to the network was 224×224.

[0009] (2) Building the GAShuffleNet network

[0010] The GAShuffleNet network is composed of channel shuffled self-attention sub-network 1, channel shuffled self-attention sub-network 2, channel shuffled self-attention sub-network 3, and channel shuffled self-attention sub-network 4 in series.

[0011] (3) Training the GAShuffleNet network

[0012] 1) Determine the objective function

[0013] The objective function consists of the loss function Loss and the evaluation function AUC.

[0014] Determine the loss function Loss according to formula (1):

[0015]

[0016] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c.

[0017] The evaluation function AUC is determined according to formula (2):

[0018]

[0019] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.

[0020] 2) Training the GAShuffleNet network

[0021] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.

[0022] (4) Save the GAShuffleNet network training model

[0023] In the process of training the GAShuffleNet network, the deep learning framework Pytorch is used to continuously update the weights and save the weight files;

[0024] (5) Verify the GAShuffleNet network

[0025] The validation set is input into the trained GAShuffleNet network for validation.

[0026] (6) Test the GAShuffleNet network

[0027] The test set is input into the trained GAShuffleNet network for testing, and the weight file with the lowest loss function during the saved training process is loaded to obtain the classification result of the image.

[0028] In the GAShuffleNet network model constructed in step (2) of the present invention, the channel shuffled self-attention subnetwork 1 is composed of a backbone module, a feature fusion residual block 1, and a feature fusion residual block 2 connected in series in sequence; the channel shuffled self-attention subnetwork 2 is composed of a downsampling module 1, a feature fusion residual block 3, and a feature fusion residual block 4 connected in series in sequence; the channel shuffled self-attention subnetwork 3 is composed of a downsampling module 2, a feature fusion residual block 5, and a feature fusion residual block 6 connected in series in sequence; the channel shuffled self-attention subnetwork 4 is composed of a downsampling module 3, a feature fusion residual block 7, and a feature fusion residual block 8 connected in series in sequence.

[0029] The feature fusion residual block 1 of the present invention is composed of a convolution module 1, a feature fusion residual sub-module 1, a feature fusion residual sub-module 2, a feature fusion residual sub-module 3, a feature fusion residual sub-module 4, a channel attention module 1, a channel attention module 2, a channel attention module 3, a channel attention module 4, a spatial attention module, a convolution layer 1, and a convolution layer 2.

[0030] The output end of the convolution module 1 is connected to the input end of the feature fusion residual submodule 1, the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 respectively; the feature fusion residual submodule 1 is connected in series with the channel attention module 1, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 2 is connected in series with the channel attention module 2, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 3 is connected in series with the channel attention module 3, the spatial attention module, and the convolution layer 2 in sequence, and the feature fusion residual submodule 4 is connected in series with the channel attention module 4, the spatial attention module, and the convolution layer 2 in sequence; the other output end of the convolution module 1 is connected to the convolution layer 2 through the convolution layer 1; the convolution kernel size of the convolution layer 1 is 3×3, the step size is 1, and the padding is 1, and the convolution kernel size of the convolution layer 2 is 1×1, the step size is 1, and the padding is 0.

[0031] The structures of feature fusion residual block 2, feature fusion residual block 3, feature fusion residual block 4, feature fusion residual block 5, feature fusion residual block 6, feature fusion residual block 7, and feature fusion residual block 8 are the same as the structure of feature fusion residual block 1.

[0032] The downsampling module 1 is composed of a pooling layer with a window size of 2×2, a step size of 2, and a padding of 0; the structures of the downsampling module 2 and the downsampling module 3 are the same as that of the downsampling module 1.

[0033] The feature fusion residual submodule 1 of the present invention is composed of a feature shuffling module 1, a feature shuffling module 2, a feature shuffling module 3, a feature shuffling module 4, a convolution module 2, a convolution module 3, a convolution module 4, a convolution module 5, and a feature fusion module; the feature shuffling module 1 is connected in series with the convolution module 2 and the feature fusion module in sequence, the feature shuffling module 2 is connected in series with the convolution module 3 and the feature fusion module in sequence, the feature shuffling module 3 is connected in series with the convolution module 4 and the feature fusion module in sequence, and the feature shuffling module 4 is connected in series with the convolution module 5 and the feature fusion module in sequence; the structures of the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 are the same as the structure of the feature fusion residual submodule 1.

[0034] The spatial attention module of the present invention is composed of a pooling module and convolutional layer 3, convolutional layer 4, convolutional layer 5, and convolutional layer 6; the output end of the pooling module is connected to the input end of convolutional layer 3, convolutional layer 4, and convolutional layer 5 respectively, and the output ends of convolutional layer 3, convolutional layer 4, and convolutional layer 5 are connected to convolutional layer 6 respectively. The convolution kernel size of convolutional layer 3 is 7×7, the step size is 1, and the padding is 3. The convolution kernel size of convolutional layer 4 is 5×5, the step size is 1, and the padding is 2. The convolution kernel size of convolutional layer 5 is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of convolutional layer 6 is 1×1, the step size is 1, and the padding is 0.

[0035] The construction methods of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3 and the feature shuffling module 4 of the present invention are as follows.

[0036] The feature shuffling module 1 is constructed as follows:

[0037]

[0038] Among them, f1 represents feature shuffling module 1, x mm Represents a feature map, the size of the feature map is m×m, and m is a finite positive integer.

[0039] The feature shuffling module 2 is constructed as follows:

[0040]

[0041] Among them, f2 represents feature shuffling module 2, T represents permutation operation, and the value of m is the same as that of feature shuffling module 1.

[0042] The feature shuffling module 3 is constructed as follows:

[0043]

[0044] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.

[0045] Construct the characteristic shuffle module 4 as follows:

[0046]

[0047] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.

[0048] In the expressions of the feature shuffling module 1, feature shuffling module 2, feature shuffling module 3 and feature shuffling module 4 of the present invention, x mm Represents a feature map, the size of the feature map is m×m, and the preferred value range of m is 38 to 64.

[0049] In the expressions of the feature shuffling module 1, feature shuffling module 2, feature shuffling module 3 and feature shuffling module 4 of the present invention, x mm Represents the feature map, the feature map size is m×m, and the optimal value of m is 56.

[0050] Since the GAShuffleNet network of the present invention adopts a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, and a channel shuffled self-attention sub-network, and the channel shuffled self-attention sub-network adopts a feature shuffle module, in the training step, the fluidity of feature intercommunication is enhanced, and the attention mechanism enhances the important feature weights of the network in training and learning by extracting effective information in the image, thereby improving the accuracy of medical image classification and reducing the complexity of calculation. Compared with the prior art, the present invention has the advantages of high classification accuracy, strong robustness, strong adaptability, fast learning speed, etc., and can be used for medical image classification and can also be used for other image classifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of Example 1 of the present invention.

[0052] Figure 2 This is a schematic diagram of the structure of the GAShuffleNet network.

[0053] Figure 3 yes Figure 2 Schematic diagram of the structure of feature fusion residual block 1.

[0054] Figure 4 yes Figure 3 Schematic diagram of the structure of the feature fusion residual submodule 1.

[0055] Figure 5 yes Figure 3 Schematic diagram of the structure of the spatial attention module. DETAILED DESCRIPTION

[0056] The present invention is further described below with reference to the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0057] Example 1

[0058] Figure 1 A flow chart of this embodiment 1 is given. Figure 1 In the present embodiment, the medical image classification method based on the GAShuffleNet network consists of the following steps:

[0059] (1) Dataset preprocessing

[0060] From 732 H&E stained full-field digital slide images WSI, these images were scanned at 10x, 20x, and 40x magnifications and annotated by pathologists. In these images, the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels. They were divided into training set, validation set, and test set in a ratio of 6:1.5:2.5. The data of the training set was normalized, scaled, center-cropped, and randomly flipped. The data of the validation set and the test set were normalized, scaled, and center-cropped, so that the initial image size of the data set input to the network was 224×224.

[0061] (2) Building the GAShuffleNet network

[0062] Figure 2 A schematic diagram of the structure of the GAShuffleNet network of this embodiment is given. Figure 2 In the embodiment, the GAShuffleNet network is composed of a channel shuffled self-attention sub-network 1, a channel shuffled self-attention sub-network 2, a channel shuffled self-attention sub-network 3, and a channel shuffled self-attention sub-network 4 connected in series.

[0063] The channel shuffled self-attention subnetwork 1 of this embodiment is composed of a backbone module, a feature fusion residual block 1, and a feature fusion residual block 2 connected in series in sequence. The channel shuffled self-attention subnetwork 2 of this embodiment is composed of a downsampling module 1, a feature fusion residual block 3, and a feature fusion residual block 4 connected in series in sequence. The channel shuffled self-attention subnetwork 3 of this embodiment is composed of a downsampling module 2, a feature fusion residual block 5, and a feature fusion residual block 6 connected in series in sequence. The channel shuffled self-attention subnetwork 4 of this embodiment is composed of a downsampling module 3, a feature fusion residual block 7, and a feature fusion residual block 8 connected in series in sequence.

[0064] Figure 3 Given Figure 2 Schematic diagram of the structure of feature fusion residual block 1. Figure 3 In the figure, the feature fusion residual block 1 of this embodiment is composed of a convolution module 1, a feature fusion residual sub-module 1, a feature fusion residual sub-module 2, a feature fusion residual sub-module 3, a feature fusion residual sub-module 4, a channel attention module 1, a channel attention module 2, a channel attention module 3, a channel attention module 4, a spatial attention module, a convolution layer 1, and a convolution layer 2.

[0065] The output end of the convolution module 1 is respectively connected to the input end of the feature fusion residual submodule 1, the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4; the feature fusion residual submodule 1 is connected in series with the channel attention module 1, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 2 is connected in series with the channel attention module 2, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 3 is connected in series with the channel attention module 3, the spatial attention module, and the convolution layer 2 in sequence, and the feature fusion residual submodule 4 is connected in series with the channel attention module 4, the spatial attention module, and the convolution layer 2 in sequence; the other output end of the convolution module 1 is connected to the convolution 2 through the convolution layer 1.

[0066] The structures of feature fusion residual block 2, feature fusion residual block 3, feature fusion residual block 4, feature fusion residual block 5, feature fusion residual block 6, feature fusion residual block 7, and feature fusion residual block 8 are the same as the structure of feature fusion residual block 1.

[0067] Figure 4 Given Figure 3 Schematic diagram of the structure of the feature fusion residual submodule 1. Figure 4 In the embodiment, the feature fusion residual submodule 1 is composed of a feature shuffling module 1, a feature shuffling module 2, a feature shuffling module 3, a feature shuffling module 4, a convolution module 2, a convolution module 3, a convolution module 4, a convolution module 5, and a feature fusion module. The feature shuffling module 1 is connected in series with the convolution module 2 and the feature fusion module in sequence, the feature shuffling module 2 is connected in series with the convolution module 3 and the feature fusion module in sequence, the feature shuffling module 3 is connected in series with the convolution module 4 and the feature fusion module in sequence, and the feature shuffling module 4 is connected in series with the convolution module 5 and the feature fusion module in sequence. The structures of the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 are the same as those of the feature fusion residual submodule 1.

[0068] The construction methods of feature shuffling module 1, feature shuffling module 2, feature shuffling module 3 and feature shuffling module 4 are as follows:

[0069] The feature shuffling module 1 is constructed as follows:

[0070]

[0071] Among them, f1 represents feature shuffling module 1, x mm represents a feature map, the size of the feature map is m×m, m is a finite positive integer, and the value of m in this embodiment is 56.

[0072] The feature shuffling module 2 is constructed as follows:

[0073]

[0074] Among them, f2 represents feature shuffling module 2, T represents permutation operation, and the value of m is the same as that of feature shuffling module 1.

[0075] The feature shuffling module 3 is constructed as follows:

[0076]

[0077] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.

[0078] Construct the characteristic shuffle module 4 as follows:

[0079]

[0080] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.

[0081] Figure 5 Given Figure 3 Schematic diagram of the structure of the spatial attention module in Fig. Figure 5 In the embodiment, the spatial attention module is composed of a pooling module and convolutional layers 3, 4, 5 and 6. The output of the pooling module is connected to the input of convolutional layers 3, 4 and 5 respectively, and the output of convolutional layers 3, 4 and 5 is connected to convolutional layer 6 respectively. The convolution kernel size of convolutional layer 3 is 7×7, the step size is 1, and the padding is 3. The convolution kernel size of convolutional layer 4 is 5×5, the step size is 1, and the padding is 2. The convolution kernel size of convolutional layer 5 is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of convolutional layer 6 is 1×1, the step size is 1, and the padding is 0.

[0082] (3) Training the GAShuffleNet network

[0083] 1) Determine the objective function

[0084] The objective function consists of the loss function Loss and the evaluation function AUC.

[0085] Determine the loss function Loss according to formula (1):

[0086]

[0087]

[0088] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c.

[0089] The evaluation function AUC is determined according to formula (2):

[0090]

[0091]

[0092] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ and P i- The value is 0.5, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.

[0093] 2) Training the GAShuffleNet network

[0094] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], the value of γ in this embodiment is 10 -5 , batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.

[0095] Since the GAShuffleNet network of the present invention adopts a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, a channel shuffled self-attention sub-network, and a channel shuffled self-attention sub-network, and the channel shuffled self-attention sub-network adopts a feature shuffle module, in the training step, the fluidity of feature intercommunication is enhanced, and the attention mechanism enhances the important feature weights of the network in training and learning by extracting effective information in the image, thereby improving the accuracy of medical image classification and reducing the complexity of calculation. Compared with the prior art, the present invention has the advantages of high classification accuracy, strong robustness, strong adaptability, fast learning speed, etc., and can be used for medical image classification and can also be used for other image classifications.

[0096] (4) Save the GAShuffleNet network training model

[0097] During the training of the GAShuffleNet network, the deep learning framework Pytorch is used to continuously update the weights and save the weight files.

[0098] (5) Verify the GAShuffleNet network

[0099] The validation set is input into the trained GAShuffleNet network for validation.

[0100] (6) Test the GAShuffleNet network

[0101] The test set is input into the trained GAShuffleNet network for testing, and the weight file with the lowest loss function during the saved training process is loaded to obtain the classification result of the image.

[0102] Complete the medical image classification method based on GAShuffleNet network.

[0103] Example 2

[0104] The medical image classification method based on the GAShuffleNet network of this embodiment consists of the following steps:

[0105] (1) Dataset preprocessing

[0106] This step is the same as in Example 1.

[0107] (2) Building the GAShuffleNet network

[0108] The GAShuffleNet network structure of this embodiment is the same as that of Embodiment 1. In this embodiment, the construction methods of feature shuffling module 1, feature shuffling module 2, feature shuffling module 3, and feature shuffling module 4 are as follows:

[0109] The feature shuffling module 1 is constructed as follows:

[0110]

[0111] Among them, f1 represents feature shuffling module 1, x mm represents a feature map, the size of the feature map is m×m, m is a finite positive integer, and the value of m in this embodiment is 38.

[0112] The feature shuffling module 2 is constructed as follows:

[0113]

[0114] Among them, f2 represents feature shuffling module 2, T represents permutation operation, and the value of m is the same as that of feature shuffling module 1.

[0115] The feature shuffling module 3 is constructed as follows:

[0116]

[0117] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.

[0118] Construct the characteristic shuffle module 4 as follows:

[0119]

[0120] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.

[0121] The other steps of this step are the same as those in Example 1.

[0122] (3) Training the GAShuffleNet network

[0123] 1) Determine the objective function

[0124] The objective function consists of the loss function Loss and the evaluation function AUC.

[0125] Determine the loss function Loss according to formula (1):

[0126]

[0127]

[0128] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c;

[0129] The evaluation function AUC is determined according to formula (2).

[0130]

[0131]

[0132] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ and P i- The value is 0, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.

[0133] 2) Training the GAShuffleNet network

[0134] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], the value of γ in this embodiment is 10 -6 , batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.

[0135] The other steps are the same as those in Example 1. A medical image classification method based on the GAShuffleNet network is completed.

[0136] Example 3

[0137] The medical image classification method based on the GAShuffleNet network of this embodiment consists of the following steps:

[0138] (1) Dataset preprocessing

[0139] This step is the same as in Example 1.

[0140] (2) Building the GAShuffleNet network

[0141] The GAShuffleNet network structure of this embodiment is the same as that of Embodiment 1. In this embodiment, the construction method of feature fusion residual submodule 1, feature fusion residual submodule 2, feature fusion residual submodule 3, and feature fusion residual submodule 4 is as follows:

[0142] The feature shuffling module 1 is constructed as follows:

[0143]

[0144] Wherein, f1 represents the feature map in the feature shuffling module 1, the size of the feature map is m×m, m is a finite positive integer, and the value of m in this embodiment is 64.

[0145] The feature shuffling module 2 is constructed as follows:

[0146]

[0147] Wherein, f2 represents feature shuffle module 2, T represents permutation operation, and the value of m is the same as that of feature shuffle module 1;

[0148] The feature shuffling module 3 is constructed as follows:

[0149]

[0150] Among them, f3 represents feature shuffling module 3, F represents flipping operation, and the value of m is the same as that of feature shuffling module 1.

[0151] Construct the characteristic shuffle module 4 as follows:

[0152]

[0153] Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.

[0154] The other steps of this step are the same as those in Example 1.

[0155] (3) Training the GAShuffleNet network

[0156] 1) Determine the objective function

[0157] The objective function consists of the loss function Loss and the evaluation function AUC.

[0158] Determine the loss function Loss according to formula (1):

[0159]

[0160]

[0161] Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c;

[0162] The evaluation function AUC is determined according to formula (2):

[0163]

[0164]

[0165] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ and P i- The value is 1, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers;

[0166] 2) Training the GAShuffleNet network

[0167] The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], the value of γ in this embodiment is 10 -4 , batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges.

[0168] The other steps are the same as those in Example 1. A medical image classification method based on the GAShuffleNet network is completed.

[0169] Example 4

[0170] In the above embodiments 1 to 3, the medical image classification method based on the GAShuffleNet network of this embodiment consists of the following steps:

[0171] (1) Dataset preprocessing

[0172] This step is the same as in Example 1.

[0173] (2) Building the GAShuffleNet network

[0174] This step is the same as in Example 1.

[0175] (3) Training the GAShuffleNet network

[0176] 1) Determine the objective function

[0177] The objective function is composed of a loss function Loss and an evaluation function AUC; the expression of the loss function Loss, the meaning of the parameters in the expression, and the value range are the same as those in the corresponding embodiment. In the expression (2) of the evaluation function AUC, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], P in this embodiment i+ The value is 0.6, and P i- The value is 0.4, M is the number of positive samples, N is the number of negative samples, M and N are finite positive integers;

[0178] 2) Training the GAShuffleNet network

[0179] This step is the same as that in the corresponding embodiment.

[0180] The other steps are the same as those in Example 1. A medical image classification method based on the GAShuffleNet network is completed.

[0181] In order to verify the beneficial effects of the present invention, the inventors conducted comparative simulation experiments using the medical image classification method based on the GAShuffleNet network of Example 1 of the present invention and the convnext_tiny method and the convnext_small method. The various experimental results are as follows:

[0182] The same test set is tested with each trained model, and the evaluation code is used to test the model accuracy. The evaluation function AUC is used as the indicator of the evaluation method. The larger the evaluation function AUC, the better the method. The experimental results of the evaluation function AUC are shown in Table 1.

[0183] Table 1 Evaluation function AUC values ​​of Example 1 and comparative methods

[0184] Test Method Evaluation function AUC Example 1 0.781 convnext_tiny method 0.734 convnext_small method 0.722

[0185] As can be seen from Table 1, the evaluation function AUC value of the method in Example 1 is 0.781, the evaluation function AUC value of the convnext_tiny method is 0.734, and the evaluation function AUC value of the convnext_small method is 0.722. The evaluation function AUC value of the method in Example 1 is 4.7% higher than the evaluation function AUC value of the convnext_tiny method and 5.9% higher than the evaluation function AUC value of the convnext_small method.

Claims

1. A medical image classification method based on GAShuffleNet network, characterized in that It consists of the following steps: (1) Dataset preprocessing From 732 H&E stained full-field digital slide images WSI, these images were scanned at 10x, 20x, and 40x magnifications and annotated by pathologists. In these images, the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels. They were divided into training set, validation set, and test set in a ratio of 6:1.5:2.

5. The data of the training set was normalized, scaled, center-cropped, and randomly flipped. The data of the validation set and the test set were normalized, scaled, and center-cropped, so that the initial image size of the data set input to the network was 224×224; (2) Building the GAShuffleNet network The GAShuffleNet network is composed of channel shuffle self-attention sub-network 1, channel shuffle self-attention sub-network 2, channel shuffle self-attention sub-network 3, and channel shuffle self-attention sub-network 4 in series; (3) Training the GAShuffleNet network 1) Determine the objective function The objective function consists of the loss function Loss and the evaluation function AUC; Determine the loss function Loss according to formula (1): Where I is the number of samples, V is the number of categories, I and V are finite positive integers; y ic is the true probability that sample i belongs to category c, p ic is the predicted probability that the observed sample i belongs to category c; The evaluation function AUC is determined according to formula (2): Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted as a negative sample, i∈[1,I], I is the number of samples, which is a finite positive integer, P i+ ∈[0,1],P i- ∈[0,1], M is the number of positive samples, N is the number of negative samples, M and N are finite positive integers; 2) Training the GAShuffleNet network The preprocessed training set is input into the GAShuffleNet network for training. During the training process, the learning rate of the GAShuffleNet network is γ, γ∈[10 -6 ,10 -4 ], batch size is 32, random dropout rate is 0, and weight decay is 10 -2 , the optimizer momentum parameter is 0.99, the optimizer uses the AdamW optimizer, and iterates until the loss function converges; (4) Save the GAShuffleNet network training model In the process of training the GAShuffleNet network, the deep learning framework Pytorch is used to continuously update the weights and save the weight files; (5) Verify the GAShuffleNet network Input the validation set into the trained GAShuffleNet network for validation; (6) Test the GAShuffleNet network The test set is input into the trained GAShuffleNet network for testing, and the weight file with the lowest loss function during the saved training process is loaded to obtain the classification result of the image.

2. The medical image classification method based on the GAShuffleNet network according to claim 1, characterized in that: In the GAShuffleNet network model constructed in step (2), the channel shuffled self-attention subnetwork 1 is composed of a backbone module, a feature fusion residual block 1, and a feature fusion residual block 2 connected in series in sequence; the channel shuffled self-attention subnetwork 2 is composed of a downsampling module 1, a feature fusion residual block 3, and a feature fusion residual block 4 connected in series in sequence; the channel shuffled self-attention subnetwork 3 is composed of a downsampling module 2, a feature fusion residual block 5, and a feature fusion residual block 6 connected in series in sequence; the channel shuffled self-attention subnetwork 4 is composed of a downsampling module 3, a feature fusion residual block 7, and a feature fusion residual block 8 connected in series in sequence.

3. The image classification method based on the GAShuffleNet network according to claim 2, characterized in that: The feature fusion residual block 1 is composed of a convolution module 1, a feature fusion residual sub-module 1, a feature fusion residual sub-module 2, a feature fusion residual sub-module 3, a feature fusion residual sub-module 4, a channel attention module 1, a channel attention module 2, a channel attention module 3, a channel attention module 4, a spatial attention module, a convolution layer 1, and a convolution layer 2. The output end of the convolution module 1 is connected to the input end of the feature fusion residual submodule 1, the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 respectively; the feature fusion residual submodule 1 is connected in series with the channel attention module 1, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 2 is connected in series with the channel attention module 2, the spatial attention module, and the convolution layer 2 in sequence, the feature fusion residual submodule 3 is connected in series with the channel attention module 3, the spatial attention module, and the convolution layer 2 in sequence, and the feature fusion residual submodule 4 is connected in series with the channel attention module 4, the spatial attention module, and the convolution layer 2 in sequence; the other output end of the convolution module 1 is connected to the convolution layer 2 through the convolution layer 1; the convolution kernel size of the convolution layer 1 is 3×3, the step size is 1, and the padding is 1, and the convolution kernel size of the convolution layer 2 is 1×1, the step size is 1, and the padding is 0; The structures of feature fusion residual block 2, feature fusion residual block 3, feature fusion residual block 4, feature fusion residual block 5, feature fusion residual block 6, feature fusion residual block 7, and feature fusion residual block 8 are the same as the structure of feature fusion residual block 1. The downsampling module 1 is composed of a pooling layer with a window size of 2×2, a step size of 2, and a padding of 0; the structures of the downsampling module 2 and the downsampling module 3 are the same as that of the downsampling module 1.

4. The image classification method based on the GAShuffleNet network according to claim 3, characterized in that: The feature fusion residual submodule 1 is composed of a feature shuffling module 1, a feature shuffling module 2, a feature shuffling module 3, a feature shuffling module 4, a convolution module 2, a convolution module 3, a convolution module 4, a convolution module 5, and a feature fusion module; the feature shuffling module 1 is connected in series with the convolution module 2 and the feature fusion module in sequence, the feature shuffling module 2 is connected in series with the convolution module 3 and the feature fusion module in sequence, the feature shuffling module 3 is connected in series with the convolution module 4 and the feature fusion module in sequence, and the feature shuffling module 4 is connected in series with the convolution module 5 and the feature fusion module in sequence; the structures of the feature fusion residual submodule 2, the feature fusion residual submodule 3, and the feature fusion residual submodule 4 are the same as the structure of the feature fusion residual submodule 1.

5. The image classification method based on the GAShuffleNet network according to claim 3, characterized in that: The spatial attention module is composed of a pooling module and convolutional layer 3, convolutional layer 4, convolutional layer 5, and convolutional layer 6; the output end of the pooling module is connected to the input end of convolutional layer 3, convolutional layer 4, and convolutional layer 5 respectively, and the output ends of convolutional layer 3, convolutional layer 4, and convolutional layer 5 are connected to convolutional layer 6 respectively. The convolution kernel size of convolutional layer 3 is 7×7, the step size is 1, and the padding is 3. The convolution kernel size of convolutional layer 4 is 5×5, the step size is 1, and the padding is 2. The convolution kernel size of convolutional layer 5 is 3×3, the step size is 1, and the padding is 1. The convolution kernel size of convolutional layer 6 is 1×1, the step size is 1, and the padding is 0.

6. The image classification method based on the GAShuffleNet network according to claim 4, characterized in that The construction methods of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3, and the feature shuffling module 4 are as follows: The feature shuffling module 1 is constructed as follows: Among them, f1 represents feature shuffling module 1, x mm Represents a feature map, the size of the feature map is m×m, where m is a finite positive integer; The feature shuffling module 2 is constructed as follows: Wherein, f2 represents feature shuffle module 2, T represents permutation operation, and the value of m is the same as that of feature shuffle module 1; The feature shuffling module 3 is constructed as follows: Among them, f3 represents feature shuffle module 3, F represents flip operation, and the value of m is the same as that of feature shuffle module 1; Construct the characteristic shuffle module 4 as follows: Among them, f4 represents feature shuffling module 4, the value of m is the same as that of feature shuffling module 1, T represents the same meaning as that of feature shuffling module 2, and F represents the same meaning as that of feature shuffling module 3.

7. The image classification method based on the GAShuffleNet network according to claim 6, characterized in that: In the expressions of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3 and the feature shuffling module 4, x mm Represents the feature map. The size of the feature map is m×m, and the value of m is 38 to 64.

8. The image classification method based on the GAShuffleNet network according to claim 6, characterized in that: In the expressions of the feature shuffling module 1, the feature shuffling module 2, the feature shuffling module 3 and the feature shuffling module 4, x mm Represents the feature map, the feature map size is m×m, and the value of m is 56.

Citation Information

Cited By

  • Image classification method based on SN-HiFuse network

    CN116824245A

  • Image classification method based on sn-hifuse network

    CN116824245B