Image classification method based on cot-circonvnext network

Abstract

An image classification method based on a Cot-cirConvNeXt network, which is composed of the steps of data set preprocessing, Cot-cirConvNeXt network construction, Cot-cirConvNeXt network training, model saving, Cot-cirConvNeXt network verification and Cot-cirConvNeXt network testing. The Cot-cirConvNeXt network uses a ConvNeXt-t network as a backbone feature extraction network, combines local features with global features through context attention and position perception recurrent convolution in the attention convolution module and the perception convolution module, uses context attention to enhance the representation ability of global features and the attention degree of different regions in the image, uses position perception recurrent convolution to obtain advanced information representation, obtains a Cot-cirConvNeXt network model, and classifies the image by using effective information in the image. The present application has the advantages of high classification accuracy, strong robustness and the like, and can be used for image classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to image classification. BACKGROUND

[0002] Since the breakthrough of AlexNet network, the convolutional neural network has been the dominant model architecture in computer vision. In recent years, visual self-attention models have made great progress in various fields. In the field of computer vision, visual self-attention models have also become substitutes for convolutional neural networks. Although visual self-attention has achieved great success, visual self-attention still cannot completely replace convolutional neural networks due to their respective advantages. Visual self-attention is good at extracting global features using attention mechanisms, while convolutional neural networks are more effective in modeling local relationships due to their inductive bias. Therefore, the ConvNeXt network can effectively extract local features for pure convolutional structure networks, and achieves good results in image classification, but the effect still needs to be improved due to the lack of good use of global features. SUMMARY

[0003] The technical problem to be solved by the application is to overcome the shortcomings of the prior art and provide an image classification method based on a Cot-cirConvNeXt network with high classification accuracy and strong robustness.

[0004] The technical solution adopted to solve the above technical problem is composed of the following steps:

[0005] (1) Data set preprocessing

[0006] Take 1000 full field of view digital slices, cut out the labeled image area, cut into 224x224 pixel image blocks, and divide them into training set, validation set and test set in the ratio of 8:1:1.

[0007] (2) Constructing a Cot-cirConvNeXt network

[0008] The Cot-cirConvNeXt network is composed of a first feature construction module, a second feature construction module, a third feature construction module, a fourth feature construction module and a full connection layer connected in sequence.

[0009] The first feature construction module is composed of a convolution layer with a convolution kernel size of 4x4 and a step of 4, a normalization layer and three attention convolution modules connected in sequence; the second feature construction module is composed of a down-sampling layer and three attention convolution modules connected in sequence; the third feature construction module is composed of a down-sampling layer and nine perception convolution modules connected in sequence; and the fourth feature construction module is composed of a down-sampling layer and three perception convolution modules connected in sequence.

[0010] (3) training the Cot-cirConvNeXt network

[0011] 1) determining the objective function

[0012] The objective function includes a loss function L, an evaluation function ACC, and an evaluation function F1, and the loss function L is determined according to the following formula:

[0013] L = - [y log p + (1-y) log (1-p i )] i ]

[0014]

[0015] wherein y represents a target class or a non-target class, the target class y takes a value of 0, the non-target class y takes a value of 1, p i is a prediction probability of the model for the i-th class, z i represents a prediction score of the model for the i-th class, and z c represents a prediction score of the model for the c-th class.

[0016] The evaluation function ACC is determined according to the following formula:

[0017]

[0018] wherein TP represents the number of positive examples predicted correctly, FP represents the number of negative examples predicted incorrectly, TN represents the number of negative examples predicted correctly, FN represents the number of positive examples predicted incorrectly, the sum of TP, FP, TN, and FN is the total number of samples, the sum of TP and TN is the number of samples predicted correctly, TP, FP, TN, and FN ∈ N, N is a finite positive integer, and TP, FP, TN, and FN are not all 0 at the same time.

[0019] The evaluation function F1 is determined according to the following formula:

[0020]

[0021]

[0022]

[0023] wherein P is the precision, R is the recall, and TP, FP, TN, and FN are not all 0 at the same time.

[0024] 2) training the Cot-cirConvNeXt network

[0025] The training set is sent into the Cot-cirConvNeXt network for training, and in the training process, the learning rate of the Cot-cirConvNeXt network is γ ∈ [10 -5 , 10-3 ], the optimizer adopts the AdamW optimization method, and iteration is performed until the loss function converges.

[0026] (4) Save the model

[0027] During the training of the Cot-cirConvNeXt network, the weights are constantly updated, and the weight file is saved.

[0028] (5) Verify the Cot-cirConvNeXt network

[0029] The verification set is input into the Cot-cirConvNeXt network for verification.

[0030] (6) Test the Cot-cirConvNeXt network

[0031] The test set is input into the Cot-cirConvNeXt network for testing, the saved weight file is loaded, and the classification of the image is obtained.

[0032] In step (2) of the present application, the attention convolution module of the Cot-cirConvNeXt network is composed of a depth separable convolution layer, a normalization layer 1, a convolution kernel size of 1x1 step of 1 convolution layer 1, an activation function layer 1, a convolution kernel size of 1x1 step of 1 convolution layer 2, a normalization layer 2, a regularization layer 1, and a context attention module 1 connected in sequence; the depth separable convolution layer is composed of a channel-wise convolution layer and a point-wise convolution layer with a convolution kernel size of 1x1 and a step of 1, and the channel-wise convolution layer has a convolution kernel size of 7x7, a step of 1, a padding number of 3, and a group number of the current channel number; the attention convolution module of the second feature construction module has the same structure as the attention convolution module of the first feature construction module.

[0033] The convolution kernel size of the context attention module 1 of the present application is 3, and the step is 1.

[0034] In step (2) of the present application, the perception convolution module of the third feature construction module of the Cot-cirConvNeXt network model is composed of a position perception recurrent convolution layer with a convolution kernel size of 16 and a step of 1, a normalization layer 3, a convolution layer 3 with a convolution kernel size of 1x1, an activation function layer 2, a convolution layer 4 with a convolution kernel size of 1x1, a normalization layer 4, a regularization layer 2, and a context attention module 2 connected in sequence; the perception convolution module of the fourth feature construction module is composed of a position perception recurrent convolution layer with a convolution kernel size of 8 and a step of 1, a normalization layer 3, a convolution layer 3 with a convolution kernel size of 1x1, an activation function layer 2, a convolution layer 4 with a convolution kernel size of 1x1, a normalization layer 4, a regularization layer 2, and a context attention module 2 connected in sequence.

[0035] In the step (2) of the application, the down-sampling layer is composed of a normalization layer and a convolution layer with a convolution kernel size of 2x2 and a step of 2.

[0036] Since the attention convolution module is used in the application, the representation ability of the network aggregated feature map is enhanced, the high-level information representation of the network is obtained by using the position-aware recurrent convolution layer, and the attention convolution module and the perception convolution module combine the local features and the global features to classify the image by using the effective information in the image, so that higher classification accuracy is obtained. Compared with the prior art, the application has the advantages of combining local features and global features, high classification accuracy, strong robustness and the like, and is suitable for image classification. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flowchart of embodiment 1 of the application.

[0038] Figure 2 is a structural schematic diagram of the Cot-cirConvNeXt network.

[0039] Figure 3 is a structural schematic diagram of the attention convolution module.

[0040] Figure 4 is a structural schematic diagram of the perception convolution module. DETAILED DESCRIPTION

[0041] The application will be further described in detail below in combination with the drawings and embodiments, but the application is not limited to the following embodiments.

[0042] Embodiment 1

[0043] The image classification method based on the Cot-cirConvNeXt network of the present embodiment comprises the following steps (see Figure 1 ):

[0044] (1) Data set preprocessing

[0045] Take 1000 full field digital slices, cut out the labeled image region, cut into 12000 224x224 pixel image blocks, and divide the image blocks into target classes and non-target classes according to the labels, and divide them into training set, validation set and test set in the ratio of 8:1:1.

[0046] (2) Constructing Cot-cirConvNeXt network

[0047] In the step (2) of the application, the down-sampling layer is composed of a normalization layer and a convolution layer with a convolution kernel size of 2x2 and a step of 2. Figure 2In specific embodiments, the Cot-cirConvNeXt network of the present embodiment is composed of a first feature construction module, a second feature construction module, a third feature construction module, a fourth feature construction module, and a full connection layer in sequence. The first feature construction module of the present embodiment is composed of a convolution layer with a convolution kernel size of 4x4 and a step of 4, a normalization layer, three attention convolution modules in sequence. The second feature construction module is composed of a down-sampling layer and three attention convolution modules in sequence. The third feature construction module is composed of a down-sampling layer and nine perception convolution modules in sequence. The fourth feature construction module is composed of a down-sampling layer and three perception convolution modules in sequence.

[0048] In specific embodiments, the Cot-cirConvNeXt network of the present embodiment is composed of a first feature construction module, a second feature construction module, a third feature construction module, a fourth feature construction module, and a full connection layer in sequence. The first feature construction module of the present embodiment is composed of a convolution layer with a convolution kernel size of 4x4 and a step of 4, a normalization layer, three attention convolution modules in sequence. The second feature construction module is composed of a down-sampling layer and three attention convolution modules in sequence. The third feature construction module is composed of a down-sampling layer and nine perception convolution modules in sequence. The fourth feature construction module is composed of a down-sampling layer and three perception convolution modules in sequence. Figure 3 In specific embodiments, the attention convolution module of the present embodiment is composed of a depth separable convolution layer and a normalization layer 1, a convolution layer 1 with a convolution kernel size of 1x1 and a step of 1, an activation function layer 1, a convolution layer 2 with a convolution kernel size of 1x1 and a step of 1, a normalization layer 2, a regularization layer 1, and a context attention module 1 in sequence. The convolution kernel size of the context attention module 1 of the present embodiment is 3 and the step is 1. The depth separable convolution layer is composed of a channel-wise convolution layer and a point-wise convolution layer with a convolution kernel size of 1x1 and a step of 1. The convolution kernel size of the channel-wise convolution layer is 7x7, the step is 1, the padding number is 3, and the grouping number is the current channel number. The structure of the attention convolution module of the second feature construction module is the same as that of the first feature construction module.

[0049] In specific embodiments, the Cot-cirConvNeXt network of the present embodiment is composed of a first feature construction module, a second feature construction module, a third feature construction module, a fourth feature construction module, and a full connection layer in sequence. The first feature construction module of the present embodiment is composed of a convolution layer with a convolution kernel size of 4x4 and a step of 4, a normalization layer, three attention convolution modules in sequence. The second feature construction module is composed of a down-sampling layer and three attention convolution modules in sequence. The third feature construction module is composed of a down-sampling layer and nine perception convolution modules in sequence. The fourth feature construction module is composed of a down-sampling layer and three perception convolution modules in sequence. Figure 4 In specific embodiments, the perception convolution module of the third feature construction module of the present embodiment is composed of a position perception recurrent convolution layer with a convolution kernel size of 16 and a step of 1, a normalization layer 3, a convolution layer 3 with a convolution kernel size of 1x1, an activation function layer 2, a convolution layer 4 with a convolution kernel size of 1x1, a normalization layer 4, a regularization layer 2, and a context attention module 2 in sequence. The perception convolution module of the fourth feature construction module is composed of a position perception recurrent convolution layer with a convolution kernel size of 8 and a step of 1, a normalization layer 3, a convolution layer 3 with a convolution kernel size of 1x1, an activation function layer 2, a convolution layer 4 with a convolution kernel size of 1x1, a normalization layer 4, a regularization layer 2, and a context attention module 2 in sequence.

[0050] The down-sampling layer of the present embodiment is composed of a normalization layer and a convolution layer with a convolution kernel size of 2x2 and a step of 2.

[0051] (3) Training the Cot-cirConvNeXt network

[0052] 1) Determining the objective function

[0053] The objective function includes a loss function L, an evaluation function ACC, and an evaluation function F1. The loss function L is determined as follows:

[0054] L = -[y log p + (1 - y) log (1 - p)] i i ]

[0055]

[0056] wherein y represents a target class or a non-target class, the target class y takes a value of 0, the non-target class y takes a value of 1, p i is a prediction probability of the model for the i-th class, z i represents a prediction score of the model for the i-th class, and z c represents a prediction score of the model for the c-th class.

[0057] An evaluation function ACC is determined according to the following formula:

[0058]

[0059] wherein TP represents a number of positive examples predicted correctly, FP represents a number of negative examples predicted incorrectly, TN represents a number of negative examples predicted correctly, FN represents a number of positive examples predicted incorrectly, the sum of TP, FP, TN and FN is a total number of samples, the sum of TP and TN is a number of samples predicted correctly, TP, FP, TN and FN are in N, N is a number of samples, N takes a value of 12000, and TP, FP, TN and FN are not all 0 at the same time.

[0060] An evaluation function F1 is determined according to the following formula:

[0061]

[0062]

[0063]

[0064] wherein P is a precision rate, R is a recall rate, and TP, FP, TN and FN of the embodiment are not all 0 at the same time.

[0065] 2) Training of the Cot-cirConvNeXt network

[0066] The training set is sent into the Cot-cirConvNeXt network for training, in the training process, the learning rate of the Cot-cirConvNeXt network is γ ∈ [10 -5 , 10 -3 ], the value of γ of the embodiment is 10 -4 , the optimizer adopts the AdamW optimization method, and iteration is performed until the loss function converges.

[0067] (4) Saving of the model

[0068] ​In the process of training the Cot-cirConvNeXt network, the weights are constantly updated, and the weight file is saved.

[0069] (5) Verify the Cot-cirConvNeXt network

[0070] The verification set is input into the Cot-cirConvNeXt network for verification.

[0071] (6) Test the Cot-cirConvNeXt network

[0072] The test set is input into the Cot-cirConvNeXt network for testing, the saved weight file is loaded, and the classification of the image is obtained.

[0073] The image classification method based on the Cot-cirConvNeXt network is completed.

[0074] Example 2

[0075] The image classification method based on the Cot-cirConvNeXt network of the present embodiment consists of the following steps:

[0076] (1) Data set preprocessing

[0077] This step is the same as in Example 1.

[0078] (2) Constructing the Cot-cirConvNeXt network

[0079] This step is the same as in Example 1.

[0080] (3) Training the Cot-cirConvNeXt network

[0081] 1) Determine the objective function

[0082] The objective function includes the loss function L, the evaluation function ACC, and the evaluation function F1, and the loss function L is determined as follows:

[0083] L=-[y log p i +(1-y)log(1-p i )]

[0084]

[0085] Where y represents the target class or non-target class, the target class y takes the value 0, and the non-target class y takes the value 1, p i is the prediction probability of the model for the i-th class, z i represents the prediction score of the model for the i-th class, and z c represents the prediction score of the model for the c-th class.

[0086] The evaluation function ACC is determined as follows:

[0087]

[0088] Wherein, TP represents the number of positive examples predicted correctly, FP represents the number of negative examples predicted incorrectly, TN represents the number of negative examples predicted correctly, FN represents the number of positive examples predicted incorrectly, the sum of TP, FP, TN and FN is the total number of samples, the sum of TP and TN is the number of samples predicted correctly, TP, FP, TN, FN e N, N is the number of samples, N is 12000, and TP, FP, TN and FN are not zero at the same time.

[0089] The evaluation function F1 is determined as follows:

[0090]

[0091]

[0092]

[0093] Wherein, P is the precision, R is the recall, and TP, FP, TN and FN are not zero at the same time.

[0094] 2) Training the Cot-cirConvNeXt network

[0095] The training set is sent into the Cot-cirConvNeXt network for training. During the training process, the learning rate of the Cot-cirConvNeXt network is γ e [10 -5 , 10 -3 ], the value of γ in this embodiment is 10 -5 , the optimizer uses the AdamW optimization method, and iteration is performed until the loss function converges.

[0096] The other steps are the same as in Example 1. The image classification method based on the Cot-cirConvNeXt network is completed.

[0097] Example 2

[0098] The image classification method based on the Cot-cirConvNeXt network in this embodiment consists of the following steps:

[0099] (1) Data set preprocessing

[0100] This step is the same as in Example 1.

[0101] (2) Constructing the Cot-cirConvNeXt network

[0102] This step is the same as in Example 1.

[0103] (3) Training the Cot-cirConvNeXt network

[0104] 1) Determining the objective function

[0105] The objective function includes a loss function L, an evaluation function ACC, and an evaluation function F1, and the loss function L is determined according to the following formula

[0106] L = - [y log p + (1-y) log (1-p i )] i )

[0107]

[0108] wherein y represents a target class or a non-target class, the target class y takes a value of 0, the non-target class y takes a value of 1, p i is a prediction probability of the model for the i-th class, z i represents a prediction score of the model for the i-th class, and z c represents a prediction score of the model for the c-th class.

[0109] The evaluation function ACC is determined according to the following formula:

[0110]

[0111] wherein TP represents the number of positive examples predicted correctly, FP represents the number of negative examples predicted incorrectly, TN represents the number of negative examples predicted correctly, FN represents the number of positive examples predicted incorrectly, the sum of TP, FP, TN, and FN is the total number of samples, the sum of TP and TN is the number of samples predicted correctly, TP, FP, TN, and FN ∈ N, N is the number of samples, N takes a value of 12000, and TP, FP, TN, and FN are not all 0 at the same time;

[0112] The evaluation function F1 is determined according to the following formula:

[0113]

[0114]

[0115]

[0116] wherein P is the precision, R is the recall, and TP, FP, TN, and FN are not all 0 at the same time;

[0117] 2) Training the Cot-cirConvNeXt network

[0118] The training set is sent into the Cot-cirConvNeXt network for training, and in the training process, the learning rate of the Cot-cirConvNeXt network is γ ∈ [10-5 ,10 -3 ] The value of gamma in this embodiment is 10 -3 The optimizer adopts the AdamW optimization method, and iteration is performed until the loss function converges.

[0119] The other steps are the same as in Embodiment 1. The image classification method based on the Cot-cirConvNeXt network is completed.

[0120] In order to verify the beneficial effects of the present application, comparative simulation experiments were carried out between the image classification method based on the Cot-cirConvNeXt network of Embodiment 1 of the present application and the ConvNeXt method and the VGG16 method, and various experimental conditions are as follows:

[0121] The saved models were used to test the same test set, and the evaluation code was used to test the model accuracy, the evaluation function ACC and the evaluation function F1 were used as the evaluation method indicators, the larger the values of the evaluation function ACC and the evaluation function F1, the better the method, and the experimental results of the evaluation function ACC and the evaluation function F1 are shown in Table 1.

[0122] Table 1 Evaluation results of the evaluation function ACC and the evaluation function F1 of Embodiment 1 and the existing method

[0123] Experimental methods Evaluation function ACC (%) Evaluation function Fl (%) VGG16 67.62 58.72 ConvNeXt 69.20 59.32 Example 1 71.56 61.24

[0124] As can be seen from Table 1, the evaluation function ACC value of the method of Embodiment 1 is 2.36% higher than the evaluation function ACC value of the ConvNeXt method, and 3.94% higher than the evaluation function ACC value of the VGG16 method; the evaluation function F1 value of the method of Embodiment 1 is 1.92% higher than the evaluation function F1 value of the ConvNeXt method, and 2.52% higher than the evaluation function F1 value of the VGG16 method.

Claims

1. An image classification method based on Cot-cirConvNeXt network, characterized in that... It consists of the following steps: (1) Dataset preprocessing Take 1000 full-view digital slices, extract the labeled image regions, cut them into 224×224 pixel image blocks, and divide them into training set, validation set and test set in a ratio of 8:1:1; (2) Constructing the Cot-cirConvNeXt network The Cot-cirConvNeXt network consists of a first feature construction module, a second feature construction module, a third feature construction module, a fourth feature construction module, and a fully connected layer connected in series. The first feature construction module consists of a convolutional layer with a kernel size of 4×4 and a stride of 4, a normalization layer, and three attention convolutional modules connected in series; the second feature construction module consists of a downsampling layer and three attention convolutional modules connected in series; the third feature construction module consists of a downsampling layer and nine perceptual convolutional modules connected in series; the fourth feature construction module consists of a downsampling layer and three perceptual convolutional modules connected in series. The perceptual convolutional module of the third feature construction module consists of a position-aware recurrent convolutional layer with a kernel size of 16 and a stride of 1, a normalization layer 3, a convolutional layer 3 with a kernel size of 1×1, an activation function layer 2, a convolutional layer 4 with a kernel size of 1×1, a normalization layer 4, a regularization layer 2, and a context attention module 2 connected in series. The perceptual convolutional module of the fourth feature construction module consists of a position-aware recurrent convolutional layer with a kernel size of 8 and a stride of 1, a normalization layer 3, a convolutional layer 3 with a kernel size of 1×1, an activation function layer 2, a convolutional layer 4 with a kernel size of 1×1, a normalization layer 4, a regularization layer 2, and a context attention module 2 connected in series. (3) Training the Cot-cirConvNeXt network 1) Determine the objective function The objective function includes the loss function L, the evaluation function ACC, and the evaluation function F1. The loss function L is determined by the following formula: Where y represents the target class or non-target class, with y being 0 for the target class and 1 for the non-target class. For the model to the first The predicted probabilities of each category, The model represents the first Predicted scores for each category, The model represents the first Predicted scores for each category; The evaluation function ACC is determined by the following formula: in, This indicates the number of correctly predicted positive examples. This indicates the number of incorrect predictions for negative examples. This indicates the number of correctly predicted negative examples. This represents the number of incorrect positive predictions. The sum of TP, FP, TN, and FN represents the total number of samples, while the sum of TP and TN represents the number of correctly predicted samples. N is a finite positive integer, and Not both are 0; The evaluation function F1 is determined by the following formula: in, It's about accuracy. It is the recall rate, and Not both are 0; 2) Training the Cot-cirConvNeXt network The training set is fed into the Cot-cirConvNeXt network for training. During training, the learning rate of the Cot-cirConvNeXt network is... The optimizer uses the AdamW optimization method and iterates until the loss function converges; (4) Save the model During the training of the Cot-cirConvNeXt network, the weights are continuously updated and the weight file is saved. (5) Verify the Cot-cirConvNeXt network The validation set is input into the Cot-cirConvNeXt network for validation. (6) Testing the Cot-cirConvNeXt network The test set is input into the Cot-cirConvNeXt network for testing, and the saved weight file is loaded to obtain the image classification.

2. The image classification method based on the Cot-cirConvNeXt network according to claim 1, characterized in that: In step (2) of constructing the Cot-cirConvNeXt network, the attention convolution module is composed of a depthwise separable convolutional layer and a normalization layer 1, a convolutional layer 1 with a kernel size of 1×1 and a stride of 1, an activation function layer 1, a convolutional layer 2 with a kernel size of 1×1 and a stride of 1, a normalization layer 2, a regularization layer 1, and a context attention module 1 connected in series. The depthwise separable convolutional layer is composed of a channel-wise convolutional layer and a pointwise convolutional layer with a kernel size of 1x1 and a stride of 1 connected in series. The kernel size of the channel-wise convolutional layer is 7x7, the stride is 1, the number of paddings is 3, and the number of groups is the current number of channels. The attention convolution module of the second feature construction module has the same structure as the attention convolution module of the first feature construction module.

3. The image classification method based on the Cot-cirConvNeXt network according to claim 2, characterized in that: The convolution kernel size of the context attention module 1 is 3 and the stride is 1.

4. The image classification method based on the Cot-cirConvNeXt network according to claim 1, characterized in that: In step (2), the Cot-cirConvNeXt network model is constructed. The downsampling layer is composed of a normalized layer and a convolutional layer with a kernel size of 2×2 and a stride of 2 connected in series.

Images