Image classification method based on improved OfficientNetV2 model, electronic equipment and readable storage medium

By adding self-calibration convolution module and preprocessing technology to the EfficientNetV2 model, the existing models have solved the problem of low training complexity and accuracy in laryngeal cancer detection, and achieved higher detection accuracy and feature representation capabilities.

CN118941840AActive Publication Date: 2024-11-12CHONGQING JIAOTONG UNIV

Patent Information

Application Number
CN202410925308.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-12
Estimated Expiration
2044-07-11

AI Technical Summary

Technical Problem

Existing deep learning models are complex in laryngeal cancer detection and have low accuracy, making it difficult to effectively identify precancerous morphological manifestations at different stages.

Method used

The improved EfficientNetV2 model is adopted to improve image quality and train the model complexity by inserting a self-calibration convolution module and combining noise enhancement, median filtering and conditions to generate preprocessing rules for adversarial networks.

Benefits of technology

It improves the accuracy and detection accuracy of image classification, reduces the complexity of model training, and enhances the distinction of feature representation and network performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941840A_ABST
    Figure CN118941840A_ABST
Patent Text Reader

Abstract

The invention provides an image classification method based on an improved OfficientNetV2 model. The method comprises the following steps: S1, obtaining a sample data set; s2, an improved EfficientNetV2 model is constructed, and an improved EfficientNetV2 model is S3, the sample data set is input into the improved OfficientNetV2 model to be trained; s4, judging whether the improved OfficientNetV2 model is trained or not, if the improved OfficientNetV2 model is trained, entering the step S5; if not, adjusting the model parameters, and returning to the step S3; and S5, pre-processing an image to be predicted, and inputting the pre-processed image into the improved OfficientNetV2 model which completes training, so as to obtain a classification result of the image. By means of the method, richer context information can be captured, the distinguishing performance of feature representation is improved, the model is made to concentrate on the part with more information, and therefore the detection precision is improved; and the number of parameters is reduced, and the complexity of model training is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image classification, and in particular to an image classification method, electronic device and readable storage medium based on an improved EfficientNetV2 model. Background Art

[0002] Laryngeal cancer detection mainly uses electronic laryngoscopes for early clinical diagnosis. However, under electronic laryngoscope examination, the morphological manifestations of precancerous lesions at different stages may be similar, and the diagnosis mainly depends on the clinical experience of otolaryngologists, which is slow and less accurate.

[0003] There are now some deep learning detection methods for laryngeal cancer, but the model training is complex, the detection accuracy is low, and the recognition effect is not ideal.

[0004] Therefore, in order to solve the above technical problems, it is urgent to propose a new technical means. Summary of the invention

[0005] In view of this, in order to improve the accuracy of detection and reduce the complexity of model training, the present invention provides an image classification method, an electronic device and a readable storage medium based on an improved EfficientNetV2 model.

[0006] The present invention provides an image classification method based on an improved EfficientNetV2 model, comprising the following steps:

[0007] S1. Obtain an image data set, and preprocess the image data set to obtain a sample data set;

[0008] S2. Build an improved EfficientNetV2 model;

[0009] The improved EfficientNetV2 model includes: EfficientNetV2 model and self-calibrated convolution module;

[0010] Insert the self-calibration convolution module into the EfficientNetV2 model, connect the input of the self-calibration convolution module to the output of the MBConv module and the Fused-MBConv module, and connect the output of the self-calibration convolution module to the input of the convolution layer in the EfficientNetV2 model;

[0011] S3. Input the sample data set into the improved EfficientNetV2 model for training;

[0012] S4. Determine whether the improved EfficientNetV2 model has been trained. If so, proceed to step S5; if not, adjust the model parameters and return to step S3;

[0013] S5. Preprocess the image to be predicted, and input the preprocessed image into the trained improved EfficientNetV2 model to obtain the classification result of the image.

[0014] Further, the image preprocessing steps are as follows:

[0015] Set preprocessing rules I and preprocessing rules II;

[0016] The images are processed using preprocessing rules I and II to obtain images T(a) and C(a) respectively;

[0017] The images T(a) and C(a) are fused to obtain the preprocessed image E(a). The fusion formula is as follows:

[0018] E(a)=γ·T(a)+(1-γ)·C(a)

[0019] Among them, γ represents the adjustment parameter.

[0020] Furthermore, preprocessing rule I: firstly, the image is subjected to noise enhancement, then the noise enhanced image is subjected to median filtering, and finally the image after median filtering is subjected to adaptive histogram equalization to enhance the local contrast.

[0021] Furthermore, preprocessing rule II: use conditional generative adversarial network to process the image.

[0022] Furthermore, in step S3, the improved EfficientNetV2 model is trained using the loss function Loss, and the loss function Loss is as follows:

[0023]

[0024] Among them, α, β, and δ represent weight hyperparameters, m represents the category, M represents the number of categories, and y m represents the sample label, P m represents the sample prediction probability, x m Represents the absolute value of the model weight, TP represents true positive, FN represents false negative, and FP represents false positive.

[0025] Further, in step S4, when the improved EfficientNetV2 model is trained to a preset fitting degree, the training of the improved EfficientNetV2 model is completed.

[0026] Accordingly, the present invention further provides an electronic device, comprising:

[0027] A memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the image classification method based on the improved EfficientNetV2 model is implemented.

[0028] Correspondingly, the present invention also provides a readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the above-mentioned image classification method based on the improved EfficientNetV2 model is implemented.

[0029] The beneficial effects of the present invention are as follows: the present invention can improve image quality by preprocessing images, thereby improving classification accuracy; the present invention improves the EfficientNetV2 model by adding a self-calibration convolution module, thereby capturing richer contextual information, improving the distinctiveness of feature representation, and allowing the model to focus on more informative parts, thereby improving detection accuracy; and the EfficientNetV2 model reduces the number of parameters and reduces the complexity of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0031] Figure 1 It is a flow chart of the present invention;

[0032] Figure 2 It is a schematic diagram of the classification process of the present invention;

[0033] Figure 3 Schematic diagram of the MBConv module and the Fused-MBConv module of the present invention;

[0034] Figure 4 Schematic diagram of the self-calibration convolution module of the present invention;

[0035] Figure 5 A schematic diagram showing the comparison of the loss value and accuracy of the model in a specific embodiment;

[0036] Figure 6 Schematic diagram of the confusion matrix of the model in a specific embodiment. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings:

[0038] The present invention provides an image classification method based on an improved EfficientNetV2 model, comprising the following steps:

[0039] S1. Obtain an image data set, and preprocess the image data set to obtain a sample data set;

[0040] S2. Build an improved EfficientNetV2 model;

[0041] The improved EfficientNetV2 model includes: EfficientNetV2 model and self-calibrated convolution module;

[0042] Insert the self-calibration convolution module into the EfficientNetV2 model, connect the input of the self-calibration convolution module to the output of the MBConv module and the Fused-MBConv module, and connect the output of the self-calibration convolution module to the input of the convolution layer in the EfficientNetV2 model;

[0043] S3. Input the sample data set into the improved EfficientNetV2 model for training;

[0044] S4. Determine whether the improved EfficientNetV2 model has been trained. If so, proceed to step S5; if not, adjust the model parameters and return to step S3;

[0045] S5. Preprocess the image to be predicted, and input the preprocessed image into the trained improved EfficientNetV2 model to obtain the classification result of the image. Through the above method, the complexity of model training can be reduced and the accuracy of image classification can be improved.

[0046] The improved EfficientNetV2 model of the present application can be used for various types of image classification, preferably for laryngeal cancer image classification.

[0047] In this embodiment, in step S1, an image data set is obtained, and the image data set is preprocessed to obtain a sample data set;

[0048] The steps for image preprocessing are as follows:

[0049] Set preprocessing rules I and preprocessing rules II;

[0050] The images are processed using preprocessing rules I and II to obtain images T(a) and C(a) respectively;

[0051] The images T(a) and C(a) are fused to obtain the preprocessed image E(a). The fusion formula is as follows:

[0052] E(a)=γ·T(a)+(1-γ)·C(a)

[0053] Wherein, γ represents an adjustment parameter, 0≤γ≤1, and the value of γ is determined according to the demand and is used to balance the influence of the two image optimization methods.

[0054] Preprocessing rule I: First, the image is noise enhanced, then the noise enhanced image is subjected to median filtering, and finally the median filtered image is subjected to adaptive histogram equalization to enhance the local contrast. Among them, noise enhancement, median filtering and adaptive histogram equalization are all existing technologies and will not be elaborated here.

[0055] Preprocessing rule II: Use conditional generative adversarial networks to optimize the image. The optimization aspects include but are not limited to target brightness and contrast. Using conditional generative adversarial networks to optimize image quality is an existing technology and will not be described in detail here.

[0056] Fusing the images processed by preprocessing rule I and preprocessing rule II instead of processing the image processed by preprocessing rule I with preprocessing rule II can reduce the information loss between steps and achieve better results when processing complex images.

[0057] In this embodiment, in step S2, an improved EfficientNetV2 model is constructed, and the platform for constructing the model includes but is not limited to Python, Keras, and PyThrch;

[0058] The improved EfficientNetV2 model (EfficientNetV2-Self Calibrated Convolutions, Eff-SC) includes: EfficientNetV2 model and self-calibrated convolution module (Self Calibrated Convolutions, SC);

[0059] Insert the self-calibration convolution module into the EfficientNetV2 model. The input of the self-calibration convolution module is connected to the output of the MBConv module and the Fused-MBConv module. The output of the self-calibration convolution module is connected to the input of the convolution layer in the EfficientNetV2 model. Figure 2 As shown;

[0060] Among them, the structures of MBConv module and Fused-MBConv module are as follows Figure 3 As shown in FIG. 1 , it is a prior art, and its structure is not described in detail here; the structure of the self-calibration convolution module is as follows Figure 4 The above is the prior art, and its structure will not be described in detail here.

[0061] EfficientNetV2 is a classification and recognition network that uses neural architecture search (NAS) and composite model expansion methods. It can select the optimal composite coefficient, that is, adjust the depth, width and resolution of the network and the input image to find the optimal parameters required for maximum recognition feature accuracy. EfficientNetV2 reduces the number of parameters and reduces the complexity of model training. By adaptively balancing the three-dimensional space, the performance of the model is greatly improved. Compared with other deep learning models, laryngeal cancer can be identified more quickly and accurately. However, under electronic laryngoscopy, the morphological manifestations of precancerous lesions at different stages may be similar. In order to further improve the accuracy of recognition, a self-calibrated convolution module is added to the EfficientNetV2 model. The self-calibrated convolution module enables each different spatial position to adaptively encode relevant information from distant areas, breaking the limitations of traditional small-area (such as 3x3) convolutions, thereby producing more distinctive feature representations. This global information encoding capability helps the network to more accurately locate and identify target objects. The self-calibrated convolution can be effortlessly integrated into traditional convolutional layers without incurring additional parameters or complex computational overhead, and without changing hyperparameters. This versatility allows it to be seamlessly integrated into existing convolutional neural network architectures, providing performance improvements without increasing model complexity. Each point of the self-calibrated convolution in space can obtain the details of the adjacent areas and the details of the interaction data within the channel, while avoiding the interference of irrelevant areas; this interaction of spatial information helps the network better understand and process the spatial structure of the input data; similar to traditional group convolution, self-calibrated convolution divides the convolution kernel of a specific layer into multiple segments, but the convolution kernel of each part is used in a heterogeneous manner, which increases the expressiveness of the network; in EfficientNetV2, by combining self-calibrated convolution, faster training speed and fewer parameters can be achieved. In summary, the addition of self-calibrated convolution not only improves the feature extraction capability of EfficientNetV2, but also improves its performance in different visual tasks, while maintaining the efficiency and flexibility of the model.

[0062] In this embodiment, in step S3, the sample data set is input into the improved EfficientNetV2 model for training;

[0063] The improved EfficientNetV2 model is trained using the loss function Loss, and the loss function Loss is as follows:

[0064]

[0065] Among them, α, β, and δ represent weight hyperparameters, m represents the category, M represents the number of categories, and y m represents the sample label, Pm represents the sample prediction probability, x m Represents the absolute value of the model weight, TP represents true positive, FN represents false negative, and FP represents false positive. By adjusting the weight hyperparameters between different losses, the relative importance of different losses can be adjusted;

[0066] The loss function Loss not only takes into account the weighted cross entropy loss, but also can handle the problem of unbalanced categories, and also measures the overlap between the predicted segmentation area and the true area.

[0067] In this embodiment, in step S4, it is determined whether the improved EfficientNetV2 model has been trained. If so, the process proceeds to step S5; if not, the model parameters are adjusted and the process returns to step S3;

[0068] When the improved EfficientNetV2 model is trained to a preset fitting degree, the improved EfficientNetV2 model training is completed.

[0069] In this embodiment, in step S5, the image to be predicted is preprocessed, and the preprocessed image is input into the trained improved EfficientNetV2 model to obtain the classification result of the image.

[0070] Accordingly, the present invention further provides an electronic device, comprising:

[0071] A memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the image classification method based on the improved EfficientNetV2 model is implemented.

[0072] Correspondingly, the present invention also provides a readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the above-mentioned image classification method based on the improved EfficientNetV2 model is implemented.

[0073] In this embodiment, the data set uses 3971 normal images and 185 laryngeal cancer images collected by the Otolaryngology Department of the Affiliated Hospital of Chongqing Medical University using the Olympus electronic laparoscope system OTV-S190, and the data set is input into the Eff-SC model, Shufflenetv2 model, Resnet50 model, Mobilentv2 model, Efficientnetv2 model and MobileViT model.

[0074] The SGD optimizer was used for training, with momentum set to 0.937 and weight decay 5×10 -4 , and set the initial learning rate to 1×10 -2We empirically set the total number of epochs to 200, the batch size to 4, and used the cosine annealing decay strategy to adjust the learning rate. The image size was adjusted to 640×640 during training and testing.

[0075] Figure 5 The loss (loss value)-acc (accuracy) of the six models on the data set, (a) corresponds to the Shufflenetv2 model, (b) corresponds to the Resnet50 model, (c) corresponds to the Mobilentv2 model, (d) corresponds to the Efficientnetv2 model, (e) corresponds to the MobileViT model, and (f) corresponds to the Eff-SC model; Figure 5 From the curves in , we can see that the loss value of Shufflenetv2 increases sharply to 40, resulting in a sudden change, and its accuracy also drops rapidly, indicating that Shufflenetv2 has poor learning and generalization capabilities; the accuracy of the three models Resnet50, EfficientNetV2, and MobileViT returns to normal immediately after a sudden change in a node, but the loss values ​​of Mobilenetv2 and Eff-SC decrease steadily and gradually tend to balance. In this state, the accuracy is also steadily increasing and tends to be stable, proving that the Eff-SC model has better learning and generalization capabilities. But in comparison, the accuracy of Eff-SC is higher than that of Mobilenetv2.

[0076] Figure 6 The confusion matrix of the six models on the data set can clearly reflect the classification results. (a) corresponds to the Shufflenetv2 model, (b) corresponds to the Resnet50 model, (c) corresponds to the Mobilentv2 model, (d) corresponds to the Efficientnetv2 model, (e) corresponds to the MobileViT model, and (f) corresponds to the Eff-SC model. Figure 6 In the above figure, Confusion MatriX represents the confusion matrix, Actual represents the true category of the sample, Actual Negative represents the true negative, Actual Positive represents the true positive, Predicted represents the predicted category, Predicted Positive represents the predicted positive, and Predicted Negative represents the predicted negative. When the true category of an image is positive and the predicted categories are positive and negative, respectively, true positive (TP) and false negative (FN) are obtained. When the true category of an image is negative and the predicted categories are positive and negative, respectively, false positive (FP) and true negative (TN) are obtained. Figure 6As shown, except for model (b), the true positive and true negative values ​​of the other models are the same, but the false positive and false negative values ​​of model (f) are smaller than those of the other models. Therefore, the Eff-SC model has the highest accuracy.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An image classification method based on the improved EfficientNetV2 model, Features: The following steps are involved: S1. Obtain an image data set, and preprocess the image data set to obtain a sample data set; S2. Build an improved EfficientNetV2 model; The improved EfficientNetV2 model includes: EfficientNetV2 model and self-calibrated convolution module; Insert the self-calibration convolution module into the EfficientNetV2 model, connect the input of the self-calibration convolution module to the output of the MBConv module and the Fused-MBConv module, and connect the output of the self-calibration convolution module to the input of the convolution layer in the EfficientNetV2 model; S3. Input the sample data set into the improved EfficientNetV2 model for training; S4. Determine whether the improved EfficientNetV2 model has been trained. If so, proceed to step S5; if not, adjust the model parameters and return to step S3; S5. Preprocess the image to be predicted, and input the preprocessed image into the trained improved EfficientNetV2 model to obtain the classification result of the image.

2. According to the image classification method based on the improved EfficientNetV2 model of claim 1, Features: The steps for image preprocessing are as follows: Set preprocessing rules I and preprocessing rules II; The images are processed using preprocessing rules I and II to obtain images T(a) and C(a) respectively; The images T(a) and C(a) are fused to obtain the preprocessed image E(a). The fusion formula is as follows: E(a)=γ·T(a)+(1-γ)·C(a) Among them, γ represents the adjustment parameter.

3. According to the image classification method based on the improved EfficientNetV2 model according to claim 2, Features: Preprocessing rule I: First, the image is noise enhanced, then the noise enhanced image is processed by median filtering, and finally the image after median filtering is processed by adaptive histogram equalization to enhance the local contrast.

4. According to the image classification method based on the improved EfficientNetV2 model of claim 2, Features: Preprocessing rule II: Use conditional generative adversarial network to process the image.

5. According to the image classification method based on the improved EfficientNetV2 model of claim 1, Features: In step S3, the improved EfficientNetV2 model is trained using the loss function Loss, and the loss function Loss is as follows: Among them, α, β, and δ represent weight hyperparameters, m represents the category, M represents the number of categories, and y m represents the sample label, P m represents the sample prediction probability, |x m | represents the absolute value of the model weight, TP represents true positive, FN represents false negative, and FP represents false positive.

6. According to the image classification method based on the improved EfficientNetV2 model of claim 5, Features: In step S4, when the improved EfficientNetV2 model is trained to a preset fitting degree, the improved EfficientNetV2 model training is completed.

7. An electronic device, Features: include: A memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the image classification method based on the improved EfficientNetV2 model described in any one of claims 1 to 6 is implemented.

8. A readable storage medium, Features: The readable storage medium stores computer instructions, and when the computer instructions are executed by the processor, the image classification method based on the improved EfficientNetV2 model described in any one of claims 1-6 is implemented.

Citation Information

Patent Citations

  • Pulmonary nodule detection method based on self-correcting convolution and channel attention mechanism

    CN112329871A

  • Image detection method based on improved efficientnet model

    CN117011599A

  • Fan blade defect detection method and system based on improved SSD model

    WO2023173598A1

Cited By

  • Ancient book paper fiber image classification method based on deep learning and electronic equipment

    CN119360145A

  • Register resource management method and device for GPU and medium

    CN119477664A

  • A register resource management method, device and medium for GPU

    CN119477664B