Image classification method based on EMA-ConvNeXt network
By introducing multi-scale attention module and multi-layer convolution module in EMA-ConvNeXt network, combining local and global features, the problem of lack of global perception in image classification is solved, and classification accuracy and robustness are improved , suitable for a variety of image types.
Patent Information
- Application Number
- CN202410975696.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-15
AI Technical Summary
The lack of global perception in image classification of existing convolutional neural networks leads to insufficient classification accuracy and robustness, which cannot completely replace convolutional neural networks.
A method of image classification based on EMA-ConvNeXt network was designed. Through the multi-scale attention module, the feature representation ability of the network is enhanced by combining local features and global features, and the network is constructed using multi-layer convolution module and attention mechanism to optimize the training process. Use the Adam optimizer and appropriate loss functions.
It improves the accuracy and robustness of image classification, can understand image content more comprehensively, is suitable for various types of images such as natural images and medical images, and has high classification accuracy and wide application prospects.
Smart Images

Figure CN118968131A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to image classification. Background Art
[0002] Since the breakthrough of the AlexNet network, convolutional neural networks (CNNs) have been the dominant model architecture in the field of computer vision. However, in recent years, visual self-attention models have made significant progress in many fields and have gradually become a powerful alternative to convolutional neural networks. In the field of computer vision, visual self-attention models have also performed well, but they have not completely replaced convolutional neural networks. This is mainly because both have their own advantages: visual self-attention models are good at extracting global features using attention mechanisms, while convolutional neural networks are more effective in modeling local relationships due to their inductive bias. Therefore, the ConvNeXt network is a network with a pure convolutional structure that can effectively extract local features and achieve good results in image classification, but due to the lack of global perception, the effect needs to be improved. Summary of the invention
[0003] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide an image classification method based on EMA-ConvNeXt network with high classification accuracy and strong robustness.
[0004] The technical solution adopted to solve the above technical problems is: an image classification method based on EMA-ConvNeXt network, which consists of the following steps:
[0005] Step 1. Dataset preprocessing
[0006] Obtain a dataset of images to be classified, process the images into a uniform size, and divide them into a training set, a validation set, and a test set in proportion;
[0007] Step 2. Construct EMA-ConvNeXt network
[0008] The EMA-ConvNeXt network is composed of a first feature building module, a second feature building module, a third feature building module, a fourth feature building module, and a fully connected layer connected in series;
[0009] The first feature construction module is used to extract basic features from the input image, and is composed of a first convolutional layer, a first normalization layer, three convolutional modules, and a first multi-scale attention module connected in series;
[0010] The second feature construction module is used to further extract and refine features while increasing the depth of the network, and is composed of a first downsampling layer, three convolution modules, and a second multi-scale attention module connected in series;
[0011] The third feature construction module is used to further extract features, capture more complex features, increase the complexity of the network and the richness of features, and is composed of a second downsampling layer, 9 convolution modules, and a third multi-scale attention module connected in series in sequence;
[0012] The fourth feature construction module is used to further extract features at the high level of the network to provide support for the final classification task, and is composed of the third downsampling layer, three convolution modules, and the fourth multi-scale attention module connected in series in sequence;
[0013] Step 3. Train the EMA-ConvNeXt network
[0014] 1) Determine the objective function
[0015] The objective function includes the loss function Loss and the evaluation function AUC. The loss function Loss is determined as follows:
[0016]
[0017] Among them, m is the total number of training samples, x i is the i-th sample, y i is the label corresponding to the i-th sample, l θ (x i ) is the output of the EMA-ConvNeXt network, m and i are finite positive integers;
[0018] The evaluation function AUC is determined by the following formula:
[0019]
[0020] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted to be a negative sample, I is a finite positive integer, P∈(0,1], M is the number of positive samples, N is the number of negative samples, M and N are finite positive integers;
[0021] 2) Training EMA-ConvNeXt network
[0022] The training set is input into the EMA-ConvNeXt network for training. The learning rate γ∈[10 -5 ,10 -3 ], the optimizer uses Adam optimizer, and the training iteration is carried out until the loss function of the EMA-ConvNeXt network converges;
[0023] Step 4. Save the network model
[0024] During the training of the EMA-ConvNeXt network, the weights are continuously updated and the corresponding parameters and weight files are saved;
[0025] Step 5. Verify the EMA-ConvNeXt network
[0026] Input the validation set into the EMA-ConvNeXt network for validation;
[0027] Step 6. Test the EMA-ConvNeXt network
[0028] Input the test set into the EMA-ConvNeXt network for testing, load the saved parameter and weight files, and obtain the image classification results.
[0029] Preferably, the convolution module is composed of a depthwise separable convolution layer, a second normalization layer, a second convolution layer, an activation function layer, a third convolution layer, a third normalization layer, and a regularization layer connected in series in sequence.
[0030] Preferably, the depth-wise separable convolution layer is composed of a channel-by-channel convolution layer and a point-by-point convolution layer connected in series, the convolution kernel size of the channel-by-channel convolution layer is 7x7, the step size is 1, the padding number is 3, and the number of groups is the current number of channels, and the convolution kernel size of the point-by-point convolution layer is 1x1, and the step size is 1.
[0031] Preferably, the convolution kernel size of the second convolution layer is 1×1 and the step size is 1; the convolution kernel size of the third convolution layer is 1×1 and the step size is 1.
[0032] Preferably, in the first feature construction module of step 2, the convolution kernel size of the first convolution layer is 4×4 and the step size is 4.
[0033] Preferably, in step 2, the convolution kernel size of the first multi-scale attention module is 3 and the step size is 1, and the second multi-scale attention module to the fourth multi-scale attention module have the same structure as the first multi-scale attention module.
[0034] Preferably, in step 2, the first downsampling layer is composed of a fourth normalization layer and a fourth convolutional layer connected in series, and the second downsampling layer and the third downsampling layer have the same structure as the first downsampling layer.
[0035] Preferably, in step 2, the convolution kernel size of the fourth convolutional layer is 2×2 and the step size is 2.
[0036] The beneficial effects of the present invention are as follows:
[0037] The EMA-ConvNeXt network of the present invention can effectively combine local features and global features through a multi-scale attention module. This combination enhances the representation ability of the network aggregate feature map, allowing the network to understand the image content more comprehensively, thereby improving the accuracy and precision of classification. Compared with the prior art, the present invention has the advantages of combining the effectiveness of local features and global features, high classification accuracy and strong robustness, and can adapt to various types of images, including natural images, medical images, etc., and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 This is a flow chart of Example 1 of the present invention.
[0039] Figure 2 It is a structural diagram of the EMA-ConvNeXt network.
[0040] Figure 3 It is a structural diagram of the convolution module. DETAILED DESCRIPTION
[0041] The present invention is further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the following embodiments.
[0042] Example 1
[0043] exist Figure 1 In this implementation, the image classification method based on the EMA-ConvNeXt network consists of the following steps:
[0044] Step 1. Dataset preprocessing
[0045] 1001 full-field digital slices were taken, and the annotated image areas were cut out and cut into image blocks of 2000×2000 pixels, which were divided into training set, validation set, and test set in a ratio of 16:4:5;
[0046] Step 2. Construct EMA-ConvNeXt network
[0047] exist Figure 2In the embodiment, the EMA-ConvNeXt network is composed of a first feature construction module, a second feature construction module, a third feature construction module, a fourth feature construction module, and a fully connected layer, which are connected in series in sequence. The first feature construction module, as the initial feature extraction part of the network, is used to extract basic features from the input image. The second feature construction module is used to further extract and refine features while increasing the depth of the network. The third feature construction module is used to continue to extract features in depth, increase the complexity of the network and the richness of features. The fourth feature construction module further extracts features at the high level of the network to provide support for the final classification task. The fully connected layer is used to finally integrate and abstract the features output by each feature construction module, and output a comprehensive feature vector for classification decision.
[0048] The first feature construction module of this embodiment is composed of the first convolution layer, the first normalization layer, three convolution modules, and the first multi-scale attention module connected in series, and the convolution kernel size of the first convolution layer is 4×4 and the step size is 4. This module quickly captures the local features of the image through the first convolution layer and significantly reduces the spatial dimension of the feature map. It further refines the features through the three convolution modules, enhances the nonlinear expression ability of the network, and extracts deeper features to obtain preliminary feature information. The multi-scale features of the image are captured through the first multi-scale attention module, and the representation ability of the features is enhanced, so that the network can pay attention to local details and global context information at the same time.
[0049] The second feature construction module is composed of the first downsampling layer, three convolution modules, and the second multi-scale attention module in series. The first downsampling layer is composed of the fourth normalization layer and the fourth convolution layer in series. The convolution kernel size of the fourth convolution layer is 2×2 and the stride is 2. The first downsampling layer further reduces the spatial dimension of the feature map through a convolution operation with a stride of 2, and the three convolution modules and the second multi-scale attention module further extract and refine the features.
[0050] The third feature construction module is composed of the second downsampling layer, 9 convolution modules, and the third multi-scale attention module connected in series; the second downsampling layer has the same structure as the first downsampling layer. This module uses more convolution modules to enable the network to capture more complex features. At the same time, the third multi-scale attention module enhances the feature representation ability.
[0051] The fourth feature construction module is composed of the third downsampling layer, three convolution modules, and the fourth multi-scale attention module connected in series. The third downsampling layer has the same structure as the first downsampling layer. This module is at the high level of the network. The spatial dimension of the feature map is further reduced, and the degree of feature abstraction is increased, which is conducive to higher-level classification tasks.
[0052] All convolution modules in the first feature building module to the fourth feature building module of this embodiment have the same structure. Figure 3 The convolution module is composed of a depth-wise separable convolution layer, a second normalization layer, a second convolution layer, an activation function layer, a third convolution layer, a third normalization layer, and a regularization layer in series. The depth-wise separable convolution layer is composed of a channel-by-channel convolution layer and a point-by-point convolution layer in series. The convolution kernel size of the channel-by-channel convolution layer is 7x7, the stride is 1, the number of padding is 3, and the number of groups is the current number of channels. The convolution kernel size of the point-by-point convolution layer is 1x1 and the stride is 1. The convolution kernel size of the second convolution layer is 1×1 and the stride is 1. The convolution kernel size of the third convolution layer is 1×1 and the stride is 1.
[0053] The channel-by-channel convolution layer uses a larger convolution kernel (such as 7x7) to extract local features, and the point-by-point convolution layer further refines these features. After the depth-wise separable convolution layer, the features are normalized by the second normalization layer, which helps stabilize the training process. The second convolution layer further refines the features, and the activation function layer introduces nonlinearity to enhance the expressiveness of the model. The third convolution layer integrates the features, and the third normalization layer normalizes the features again, which helps the subsequent multi-scale attention module. The regularization layer is used to prevent overfitting and improve the generalization ability of the model.
[0054] In this embodiment, the convolution kernel size of the first multi-scale attention module is 3 and the step size is 1. The second to fourth multi-scale attention modules have the same structure as the first multi-scale attention module. The four multi-scale attention modules capture local and global features through convolution kernels of different scales to enhance the representation ability of features.
[0055] Step 3. Train the EMA-ConvNeXt network
[0056] 1) Determine the objective function
[0057] The objective function includes the loss function Loss and the evaluation function AUC. The loss function Loss is determined as follows:
[0058]
[0059] Among them, m is the total number of training samples, x i is the i-th sample, y i is the label corresponding to the i-th sample, l θ (x i ) is the output of the EMA-ConvNeXt network, and m and i are finite positive integers.
[0060] The evaluation function AUC is determined by the following formula:
[0061]
[0062] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, Pi- is the probability that the i-th sample is predicted to be a negative sample, I is a finite positive integer, P∈(0,1], the value of P in this embodiment is 0.5, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.
[0063] 2) Training EMA-ConvNeXt network
[0064] The training set is input into the EMA-ConvNeXt network for training. The learning rate γ∈[10 -5 ,10 -3 ], the learning rate γ in this embodiment is 10 -4 , the optimizer uses Adam optimizer, and the training iteration is performed until the loss function of the EMA-ConvNeXt network converges.
[0065] Step 4. Save the model
[0066] During the training of the EMA-ConvNeXt network, the weights are continuously updated and the corresponding parameters and weight files are saved.
[0067] Step 5. Verify the EMA-ConvNeXt network
[0068] The validation set is input into the EMA-ConvNeXt network for validation.
[0069] Step 6. Test the EMA-ConvNeXt network
[0070] Input the test set into the EMA-ConvNeXt network for testing, load the saved parameter and weight files, and obtain the image classification results.
[0071] Complete the image classification method based on EMA-ConvNeXt network.
[0072] Example 2
[0073] The image classification method based on the EMA-ConvNeXt network of this embodiment consists of the following steps:
[0074] Step 1. Dataset preprocessing
[0075] This step is the same as in Example 1.
[0076] Step 2. Construct EMA-ConvNeXt network
[0077] This step is the same as in Example 1.
[0078] Step 3. Train the EMA-ConvNeXt network
[0079] 1) Determine the objective function
[0080] The objective function includes a loss function Loss and an evaluation function AUC. The loss function Loss is the same as that in Example 1. The evaluation function AUC is determined as follows:
[0081]
[0082] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted to be a negative sample, I is a finite positive integer, P∈(0,1], in this embodiment, the value of P is 0.1, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.
[0083] 2) Training EMA-ConvNeXt network
[0084] The training set is input into the EMA-ConvNeXt network for training. The learning rate γ∈[10 -5 ,10 -3 ], the learning rate γ in this embodiment is 10 -5 , the optimizer uses Adam optimizer, and the training iteration is performed until the loss function of the EMA-ConvNeXt network converges.
[0085] The other steps are the same as those in Example 1. The image classification method based on the EMA-ConvNeXt network is completed.
[0086] Example 3
[0087] The image classification method based on the EMA-ConvNeXt network of this embodiment consists of the following steps:
[0088] Step 1. Dataset preprocessing
[0089] This step is the same as in Example 1.
[0090] Step 2. Construct EMA-ConvNeXt network
[0091] This step is the same as in Example 1.
[0092] Step 3. Train the EMA-ConvNeXt network
[0093] 1) Determine the objective function
[0094] The objective function includes a loss function Loss and an evaluation function AUC, and the loss function Loss is the same as that in Example 1.
[0095] The evaluation function AUC is determined by the following formula:
[0096]
[0097] Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted to be a negative sample, I is a finite positive integer, P∈(0,1], in this embodiment, the value of P is 1, M is the number of positive samples, N is the number of negative samples, and M and N are finite positive integers.
[0098] 2) Training EMA-ConvNeXt network
[0099] The training set is input into the EMA-ConvNeXt network for training. The learning rate γ∈[10 -5 ,10 -3 ], the learning rate γ in this embodiment is 10 -3 , the optimizer uses Adam optimizer, and the training iteration is performed until the loss function of the EMA-ConvNeXt network converges.
[0100] The other steps are the same as those in Example 1. The image classification method based on the EMA-ConvNeXt network is completed.
[0101] In order to verify the beneficial effects of the present invention, a comparative simulation experiment was conducted using the image classification method based on the EMA-ConvNeXt network of Example 1 of the present invention and the ConvNeXt method and the ShuffleNet method. The various experimental results are as follows:
[0102] The same test set is tested with each trained model, and the evaluation code is used to test the model accuracy. The evaluation function AUC is used to evaluate the quality of the evaluation method. The larger the AUC value of the evaluation function, the better the method. The experimental results of the evaluation function AUC are shown in Table 1.
[0103] Table 1 Evaluation function AUC values of Example 1 and comparative experimental methods
[0104] Test Method Evaluation function AUC Example 1 0.770 ConvNeXt Method 0.747 ShuffleNet method 0.736
[0105] As can be seen from Table 1, the evaluation function AUC value of the method in Example 1 is 0.770, the evaluation function AUC value of the ConvNeXt method is 0.747, and the evaluation function AUC value of the ShuffleNet method is 0.736. The evaluation function AUC value of the method in Example 1 is 2.3% higher than the evaluation function AUC value of the ConvNeXt method and 3.4% higher than the evaluation function AUC value of the ShuffleNet method.
Claims
1. An image classification method based on EMA-ConvNeXt network, characterized in that It consists of the following steps: Step 1. Dataset preprocessing Obtain a dataset of images to be classified, process the images into a uniform size, and divide them into a training set, a validation set, and a test set in proportion; Step 2. Construct EMA-ConvNeXt network The EMA-ConvNeXt network is composed of a first feature building module, a second feature building module, a third feature building module, a fourth feature building module, and a fully connected layer connected in series; The first feature construction module is used to extract basic features from the input image, and is composed of a first convolutional layer, a first normalization layer, three convolutional modules, and a first multi-scale attention module connected in series; The second feature construction module is used to further extract and refine features while increasing the depth of the network, and is composed of a first downsampling layer, three convolution modules, and a second multi-scale attention module connected in series; The third feature construction module is used to further extract features, capture more complex features, increase the complexity of the network and the richness of features, and is composed of a second downsampling layer, 9 convolution modules, and a third multi-scale attention module connected in series in sequence; The fourth feature construction module is used to further extract features at the high level of the network to provide support for the final classification task, and is composed of the third downsampling layer, three convolution modules, and the fourth multi-scale attention module connected in series in sequence; Step 3. Train the EMA-ConvNeXt network 1) Determine the objective function The objective function includes the loss function Loss and the evaluation function AUC. The loss function Loss is determined as follows: Among them, m is the total number of training samples, x i is the i-th sample, y i is the label corresponding to the i-th sample, l θ (x i ) is the output of the EMA-ConvNeXt network, m and i are finite positive integers; The evaluation function AUC is determined by the following formula: Among them, P i+ is the probability that the i-th sample is predicted to be a positive sample, P i- is the probability that the i-th sample is predicted to be a negative sample, I is a finite positive integer, P∈(0,1], M is the number of positive samples, N is the number of negative samples, M and N are finite positive integers; 2) Training EMA-ConvNeXt network The training set is input into the EMA-ConvNeXt network for training. The learning rate γ∈[10 -5 ,10 -3 ], the optimizer uses Adam optimizer, and the training iteration is carried out until the loss function of the EMA-ConvNeXt network converges; Step 4. Save the network model During the training of the EMA-ConvNeXt network, the weights are continuously updated and the corresponding parameters and weight files are saved; Step 5. Verify the EMA-ConvNeXt network Input the validation set into the EMA-ConvNeXt network for validation; Step 6. Test the EMA-ConvNeXt network Input the test set into the EMA-ConvNeXt network for testing, load the saved parameter and weight files, and obtain the image classification results.
2. The image classification method based on the EMA-ConvNeXt network according to claim 1, characterized in that: In step 2, the convolution module is composed of a depthwise separable convolution layer, a second normalization layer, a second convolution layer, an activation function layer, a third convolution layer, a third normalization layer, and a regularization layer connected in series in sequence.
3. The image classification method based on the EMA-ConvNeXt network according to claim 2, characterized in that: The depthwise separable convolution layer is composed of a channel-by-channel convolution layer and a point-by-point convolution layer connected in series. The convolution kernel size of the channel-by-channel convolution layer is 7x7, the step size is 1, the number of padding is 3, and the number of groups is the current number of channels. The convolution kernel size of the point-by-point convolution layer is 1x1 and the step size is 1.
4. The image classification method based on the EMA-ConvNeXt network according to claim 2, characterized in that: The convolution kernel size of the second convolution layer is 1×1 and the step size is 1; the convolution kernel size of the third convolution layer is 1×1 and the step size is 1.
5. The image classification method based on the EMA-ConvNeXt network according to claim 1, characterized in that: In the first feature construction module of step 2, the convolution kernel size of the first convolution layer is 4×4 and the step size is 4.
6. The image classification method based on the EMA-ConvNeXt network according to claim 1, characterized in that: In step 2, the convolution kernel size of the first multi-scale attention module is 3 and the step size is 1, and the second to fourth multi-scale attention modules have the same structure as the first multi-scale attention module.
7. The image classification method based on EMA-ConvNeXt network according to claim 1, characterized in that: In step 2, the first downsampling layer is composed of a fourth normalization layer and a fourth convolutional layer connected in series, and the second downsampling layer and the third downsampling layer have the same structure as the first downsampling layer.
8. The image classification method based on the EMA-ConvNeXt network according to claim 7, characterized in that: In step 2, the convolution kernel size of the fourth convolutional layer is 2×2 and the step size is 2.
Citation Information
Patent Citations
Dynamic gesture recognition model building method and dynamic gesture recognition method
CN115578683A
Image classification method based on Cot-circConvNeXt network
CN116664917A
Casting defect identification method based on ECA-ConvNext
CN117975087A
Cited By
Image classification method based on SN-HiFuse network
CN116824245A
Image classification method based on sn-hifuse network
CN116824245B
Process perception type cable joint visual identification method based on deep learning
CN121811020A
A deep learning-based process-aware cable joint visual recognition method
CN121811020B
