Image classification method based on improved efficientnetv2 model, electronic device and readable storage medium

Abstract

The application provides an image classification method based on an improved EfficientNetV2 model, which comprises the following steps: S1, obtaining a sample data set; S2, constructing an improved EfficientNetV2 model; S3, inputting the sample data set into the improved EfficientNetV2 model for training; S4, judging whether the improved EfficientNetV2 model is trained or not, if yes, entering step S5, if not, adjusting model parameters and returning to step S3; S5, pre-processing a to-be-predicted image and inputting the pre-processed image into the trained improved EfficientNetV2 model to obtain a classification result of the image. Through the above method, more rich context information can be captured, the distinguishability of feature representation is improved, the model focuses on the part with more information, and therefore the detection precision is improved; and the number of parameters is reduced, and the complexity of model training is reduced.

Description

Technical Field

[0001] This invention relates to the field of image classification, and more particularly to an image classification method, electronic device, and readable storage medium based on an improved EfficientNetV2 model. Background Technology

[0002] Laryngeal cancer detection primarily utilizes electronic laryngoscopy for early clinical diagnosis. However, under electronic laryngoscopy, the morphological appearance of precancerous lesions at different stages may be similar, and diagnosis relies heavily on the clinical experience of the ENT endoscopist. This diagnostic process is slow and has relatively low accuracy.

[0003] Some deep learning methods for detecting laryngeal cancer have emerged, but the models are complex to train, have low detection accuracy, and the recognition effect is not ideal.

[0004] Therefore, in order to solve the above-mentioned technical problems, it is urgent to propose a new technical approach. Summary of the Invention

[0005] In view of this, in order to improve the accuracy of detection and reduce the complexity of model training, this invention provides an image classification method, electronic device and readable storage medium based on an improved EfficientNetV2 model.

[0006] This invention provides an image classification method based on an improved EfficientNetV2 model, comprising the following steps:

[0007] S1. Obtain the image dataset and preprocess it to obtain the sample dataset;

[0008] S2. Construct an improved EfficientNetV2 model;

[0009] The improved EfficientNetV2 model includes: the EfficientNetV2 model and a self-calibrating convolutional module;

[0010] The self-calibrating convolution module is inserted into the EfficientNetV2 model. The input of the self-calibrating convolution module is connected to the output of the MBConv module and the Fused-MBConv module. The output of the self-calibrating convolution module is connected to the input of the convolutional layer in the EfficientNetV2 model.

[0011] S3. Input the sample dataset into the improved EfficientNetV2 model for training;

[0012] S4. Determine whether the improved EfficientNetV2 model has been trained. If yes, proceed to step S5; otherwise, adjust the model parameters and return to step S3.

[0013] S5. Preprocess the image to be predicted, and input the preprocessed image into the trained improved EfficientNetV2 model to obtain the image classification result.

[0014] Furthermore, the image preprocessing steps are as follows:

[0015] Set preprocessing rule I and preprocessing rule II;

[0016] The images were processed using preprocessing rule I and preprocessing rule II respectively, resulting in images T(a) and C(a).

[0017] Images T(a) and C(a) are fused to obtain the preprocessed image E(a). The fusion formula is as follows:

[0018] E(a)=γ·T(a)+(1-γ)·C(a)

[0019] Where γ represents the adjustment parameter.

[0020] Furthermore, preprocessing rule I: First, the image is enhanced with noise, then the image with enhanced noise is subjected to median filtering, and finally, the image after median filtering is subjected to adaptive histogram equalization to enhance local contrast.

[0021] Furthermore, preprocessing rule II: Conditional generative adversarial networks are used to process the images.

[0022] Furthermore, in step S3, the improved EfficientNetV2 model is trained using the loss function Loss, which is as follows:

[0023]

[0024] Where α, β, and δ represent weight hyperparameters, m represents the number of categories, M represents the number of categories, and y m P represents the sample label. m x represents the predicted probability of the sample. m The absolute values ​​of the model weights are represented by TP, FN, and FP.

[0025] Furthermore, in step S4, when the improved EfficientNetV2 model is trained to a preset level of fit, the training of the improved EfficientNetV2 model is complete.

[0026] Accordingly, the present invention also provides an electronic device, comprising:

[0027] The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the image classification method based on the improved EfficientNetV2 model described above.

[0028] Accordingly, the present invention also provides a readable storage medium storing computer instructions, which, when executed by a processor, constitute the above-described image classification method based on the improved EfficientNetV2 model.

[0029] The beneficial effects of this invention are as follows: By preprocessing images, this invention can improve image quality, thereby increasing classification accuracy; by adding a self-calibrating convolution module to improve the EfficientNetV2 model, this invention can capture richer contextual information, improve the discriminativeness of feature representations, and allow the model to focus on more informative parts, thereby improving detection accuracy; and the EfficientNetV2 model reduces the number of parameters, thus reducing the complexity of model training. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0031] Figure 1 This is a flowchart of the present invention;

[0032] Figure 2 This is a schematic diagram of the classification process of the present invention;

[0033] Figure 3 This is a schematic diagram of the MBConv module and the Fused-MBConv module of the present invention;

[0034] Figure 4 This is a schematic diagram of the self-calibrating convolution module of the present invention;

[0035] Figure 5 This is a schematic diagram comparing the loss value and accuracy of the model in a specific embodiment;

[0036] Figure 6 This is a schematic diagram of the confusion matrix of the model in a specific embodiment. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings:

[0038] This invention provides an image classification method based on an improved EfficientNetV2 model, comprising the following steps:

[0039] S1. Obtain the image dataset and preprocess it to obtain the sample dataset;

[0040] S2. Construct an improved EfficientNetV2 model;

[0041] The improved EfficientNetV2 model includes: the EfficientNetV2 model and a self-calibrating convolutional module;

[0042] The self-calibrating convolution module is inserted into the EfficientNetV2 model. The input of the self-calibrating convolution module is connected to the output of the MBConv module and the Fused-MBConv module. The output of the self-calibrating convolution module is connected to the input of the convolutional layer in the EfficientNetV2 model.

[0043] S3. Input the sample dataset into the improved EfficientNetV2 model for training;

[0044] S4. Determine whether the improved EfficientNetV2 model has been trained. If yes, proceed to step S5; otherwise, adjust the model parameters and return to step S3.

[0045] S5. Preprocess the image to be predicted, and then input the preprocessed image into the trained improved EfficientNetV2 model to obtain the image classification result. This method reduces the complexity of model training and improves the accuracy of image classification.

[0046] The improved EfficientNetV2 model of this application can be used for various types of image classification, and is preferably used for laryngeal cancer image classification.

[0047] In this embodiment, in step S1, an image dataset is obtained and preprocessed to obtain a sample dataset;

[0048] The image preprocessing steps are as follows:

[0049] Set preprocessing rule I and preprocessing rule II;

[0050] The images were processed using preprocessing rule I and preprocessing rule II respectively, resulting in images T(a) and C(a).

[0051] Images T(a) and C(a) are fused to obtain the preprocessed image E(a). The fusion formula is as follows:

[0052] E(a)=γ·T(a)+(1-γ)·C(a)

[0053] Here, γ represents the adjustment parameter, 0≤γ≤1. The value of γ is determined according to the requirements and is used to balance the influence of the two image optimization methods.

[0054] Preprocessing rule I: First, noise enhancement is performed on the image. Then, median filtering is applied to the noise-enhanced image. Finally, adaptive histogram equalization is applied to the median-filtered image to enhance local contrast. Noise enhancement, median filtering, and adaptive histogram equalization are all existing technologies and will not be elaborated here.

[0055] Preprocessing rule II: The image is optimized using a conditional generative adversarial network. The optimization aspects include, but are not limited to, target brightness and contrast. Using a conditional generative adversarial network to optimize image quality is an existing technology and will not be elaborated here.

[0056] Fusing the images processed by preprocessing rule I and preprocessing rule II, instead of processing the image processed by preprocessing rule I using preprocessing rule II, can reduce information loss between steps and is more effective when processing complex images.

[0057] In this embodiment, in step S2, an improved EfficientNetV2 model is constructed. The platform for constructing the model includes, but is not limited to, Python, Keras, and PyThrch.

[0058] The improved EfficientNetV2 model (EfficientNetV2-Self Calibrated Convolutions, Eff-SC) includes: the EfficientNetV2 model and the self-calibrated convolution module (SC);

[0059] A self-calibrating convolutional module is inserted into the EfficientNetV2 model. The input of the self-calibrating convolutional module is connected to the outputs of the MBConv and Fused-MBConv modules, and the output of the self-calibrating convolutional module is connected to the input of the convolutional layers in the EfficientNetV2 model. Figure 2 As shown;

[0060] The structures of the MBConv module and the Fused-MBConv module are as follows: Figure 3 As shown, this is existing technology, and its structure will not be described in detail here; the structure of the self-calibrating convolution module is as follows. Figure 4 The diagram shows existing technology, and its structure will not be described in detail here.

[0061] EfficientNetV2 is a classification and recognition network that uses Neural Architecture Search (NAS) and composite model extension methods. It can select optimal composite coefficients, i.e., adjust the network's depth, width, and input image resolution to find the optimal parameters required for maximum feature recognition accuracy. EfficientNetV2 reduces the number of parameters, lowering the complexity of model training. By adaptively balancing the three-dimensional space, the model's performance is greatly improved, enabling faster and more accurate identification of laryngeal cancer compared to other deep learning models. However, under electronic laryngoscopy, the morphological appearance of precancerous lesions at different stages may be similar, necessitating further improvements in identification accuracy. To address this, a self-calibrating convolutional module is added to the EfficientNetV2 model. This module allows each different spatial location to adaptively encode relevant information from distant regions, breaking the limitations of traditional small-region (e.g., 3x3) convolutions and generating more discriminative feature representations. This global information encoding capability helps the network more accurately locate and identify target objects. The self-calibrating convolutions can be effortlessly integrated into traditional convolutional layers without introducing additional parameters or complex computational overhead, nor altering hyperparameters. This versatility allows for seamless integration into existing convolutional neural network architectures, providing performance improvements without increasing model complexity. Each point of a self-calibrating convolution in space can obtain details of neighboring regions and intra-channel interactive data, while avoiding interference from irrelevant regions. This interaction of spatial information helps the network better understand and process the spatial structure of the input data. Similar to traditional group convolution, self-calibrating convolution divides the convolution kernel of a specific layer into multiple segments, but the convolution kernel of each segment is utilized in a heterogeneous manner, which increases the expressiveness of the network. In EfficientNetV2, by combining self-calibrating convolution, faster training speed and fewer parameters can be achieved. In summary, the addition of self-calibrating convolution not only improves the feature extraction capability of EfficientNetV2, but also improves its performance in different vision tasks, while maintaining the efficiency and flexibility of the model.

[0062] In this embodiment, in step S3, the sample dataset is input into the improved EfficientNetV2 model for training;

[0063] The improved EfficientNetV2 model is trained using the loss function Loss, which is as follows:

[0064]

[0065] Where α, β, and δ represent weight hyperparameters, m represents the number of categories, M represents the number of categories, and y m P represents the sample label.m x represents the predicted probability of the sample. m The hyperparameters represent the absolute values ​​of the model weights, with TP representing true positives, FN representing false negatives, and FP representing false positives. By adjusting the weight hyperparameters between different losses, the relative importance of different losses can be adjusted.

[0066] The loss function Loss takes into account both weighted cross-entropy loss and the problem of imbalanced classes, and also measures the degree of overlap between the predicted segmentation region and the real region.

[0067] In this embodiment, in step S4, it is determined whether the improved EfficientNetV2 model has been trained. If yes, proceed to step S5; otherwise, adjust the model parameters and return to step S3.

[0068] The improved EfficientNetV2 model is considered complete when it reaches the preset fit.

[0069] In this embodiment, in step S5, the image to be predicted is preprocessed, and the preprocessed image is input into the trained improved EfficientNetV2 model to obtain the image classification result.

[0070] Accordingly, the present invention also provides an electronic device, comprising:

[0071] The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the image classification method based on the improved EfficientNetV2 model described above.

[0072] Accordingly, the present invention also provides a readable storage medium storing computer instructions, which, when executed by a processor, constitute the above-described image classification method based on the improved EfficientNetV2 model.

[0073] In this embodiment, the dataset consists of 3971 normal images and 185 laryngeal cancer images collected by the Department of Otolaryngology at the Affiliated Hospital of Chongqing Medical University using the Olympus electronic laparoscopic system OTV-S190. The dataset is then input into the Eff-SC model, Shufflenetv2 model, Resnet50 model, Mobilentv2 model, Efficientnetv2 model, and MobileViT model.

[0074] Training was performed using the SGD optimizer with a momentum of 0.937 and weight decay of 5 × 10⁻⁶. -4 And set the initial learning rate to 1×10 -2Based on experience, we set the total number of epochs to 200, the batch size to 4, and adopted a cosine annealing decay strategy to adjust the learning rate. During training and testing, the image size was adjusted to 640×640.

[0075] Figure 5 The loss-accuracy ratios for six models on the dataset are given below: (a) corresponds to the ShuffleNetv2 model, (b) to the ResNet50 model, (c) to the Mobilentv2 model, (d) to the EfficientNetv2 model, (e) to the MobileViT model, and (f) to the Eff-SC model. Figure 5 The curves show that the ShuffleNetv2 loss value increases sharply to 40, causing a sudden change and a rapid drop in accuracy, indicating poor learning and generalization abilities. The accuracy of ResNet50, EfficientNetV2, and MobileViT recovers immediately after a sudden change at a node, but the loss values ​​of MobileNetv2 and Eff-SC steadily decrease, gradually approaching equilibrium. Under these conditions, the accuracy also steadily increases and stabilizes, demonstrating that the Eff-SC model has better learning and generalization abilities. However, compared to MobileNetv2, Eff-SC's accuracy is higher.

[0076] Figure 6 The confusion matrices of the six models on the dataset clearly show the classification results: (a) corresponds to the Shufflenetv2 model, (b) corresponds to the Resnet50 model, (c) corresponds to the Mobilentv2 model, (d) corresponds to the Efficientnetv2 model, (e) corresponds to the MobileViT model, and (f) corresponds to the Eff-SC model. Figure 6 In this context, Confusion Matrix (CMM) represents the confusion matrix, Actual represents the true class of the sample, Actual Negative represents the true negative, Actual Positive represents the true positive, Predicted represents the predicted class, Predicted Positive represents the predicted positive, and Predicted Negative represents the predicted negative. When the true class of an image is positive and the predicted classes are positive and negative, we obtain True Positive (TP) and False Negative (FN), respectively. When the true class of an image is negative and the predicted classes are positive and negative, we obtain False Positive (FP) and True Negative (TN), respectively. Figure 6As shown, except for model (b), the true positive and true negative values ​​of the other models are the same, but the false positive and false negative values ​​of model (f) are smaller than those of the other models. Therefore, the Eff-SC model has the highest accuracy.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An image classification method based on an improved EfficientNetV2 model, characterized in that: Includes the following steps: S1. Obtain the image dataset and preprocess it to obtain the sample dataset; S2. Construct an improved EfficientNetV2 model; The improved EfficientNetV2 model includes: the EfficientNetV2 model and a self-calibrating convolutional module; The self-calibrating convolution module is inserted into the EfficientNetV2 model. The input of the self-calibrating convolution module is connected to the output of the MBConv module and the Fused-MBConv module. The output of the self-calibrating convolution module is connected to the input of the convolutional layer in the EfficientNetV2 model. S3. Input the sample dataset into the improved EfficientNetV2 model for training; S4. Determine whether the improved EfficientNetV2 model has been trained. If yes, proceed to step S5; otherwise, adjust the model parameters and return to step S3. S5. Preprocess the image to be predicted, and input the preprocessed image into the trained improved EfficientNetV2 model to obtain the image classification result; The image preprocessing steps are as follows: Set preprocessing rule I and preprocessing rule II; where: Preprocessing rule I: First, the image is enhanced with noise, then the image with enhanced noise is processed with median filtering, and finally the image after median filtering is processed with adaptive histogram equalization to enhance local contrast. Preprocessing rule II: Use a conditional generative adversarial network to process the image; The images were processed using preprocessing rule I and preprocessing rule II respectively, resulting in images T(a) and C(a). Images T(a) and C(a) are fused to obtain the preprocessed image E(a). The fusion formula is as follows: in, This indicates the adjustment parameter.

2. The image classification method based on the improved EfficientNetV2 model according to claim 1, characterized in that: In step S3, the improved EfficientNetV2 model is trained using the loss function Loss, which is as follows: in, , and This represents the weight hyperparameter, where m represents the number of categories and M represents the number of categories. Indicates sample label, Indicates the predicted probability of the sample. The absolute values ​​of the model weights are represented by TP, FN, and FP.

3. The image classification method based on the improved EfficientNetV2 model according to claim 2, characterized in that: In step S4, the improved EfficientNetV2 model is trained to a preset fit, and the training of the improved EfficientNetV2 model is completed.

4. An electronic device, characterized in that: include: A memory and a processor, the memory being used to store a computer program that, when executed by the processor, implements the image classification method based on the improved EfficientNetV2 model as described in any one of claims 1-3.

5. A readable storage medium, characterized in that: The readable storage medium stores computer instructions that, when executed by a processor, implement the image classification method based on the improved EfficientNetV2 model as described in any one of claims 1-3.

Images