Image detection classification method based on deep learning
By introducing the EMA (Efficient Multi-Scale Attention Module) and GhostConv module into the YOLOv8 model, the problems of high computational cost and low efficiency of the YOLOv8 model are solved, and more efficient image detection performance is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2024-03-19
- Publication Date
- 2026-07-24
AI Technical Summary
The existing YOLOv8 model suffers from high computational cost, low efficiency, and excessive memory consumption during image detection.
A new image detection model is constructed by introducing an efficient multi-scale attention module (EMA) into the neck network of YOLOv8 and replacing some Conv modules with GhostConv modules in the backbone network. Combined with data augmentation and optimized training process, a new image detection model is constructed.
It improves the speed, accuracy, and robustness of target detection, reduces computational load, and achieves more efficient image detection performance.
Smart Images
Figure CN118351354B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically relating to an image detection and classification method based on deep learning. Background Technology
[0002] Image classification techniques are often referred to as object classification, but relying solely on image classification is insufficient to further extract textual information or object features from images. To obtain more and more accurate information from images, a two-stage process is needed: object detection followed by image classification. While image classification performs excellently when dealing with single-category images, its effectiveness is limited when faced with multi-object or complex images. Therefore, the method of detection followed by classification is more effective.
[0003] There are many object detection models, such as SSD and YOLO. Currently, YOLO has been released to YOLOv8, but this model still suffers from insufficient detection speed and needs improvement in accuracy. The YOLOv8 structure includes a Backbone, Head, and Neck. The Backbone network is the foundation of the model, responsible for extracting features from the input image. These features form the basis for subsequent network layers to perform object detection. The backbone network adopts a structure similar to CSPDarknet. The Head network is the decision-making part of the object detection model, responsible for generating the final detection results. The Neck network is located between the backbone and the head network, and its role is to perform feature fusion and enhancement. In terms of classification networks, there are models such as ResNet, MobileNet, and ViT (VisionTransformer). Among them, ViT is one of the best-performing network models currently available. It crops the input image into a series of patches as the network's raw input and then processes the data in a natural language processing (NLP) manner. Assuming the original input image data is H×W×C, we need to cut the image into patches to obtain the input patch. Assuming the patch size is P1×P2, the final number of patches is... However, since the first stage of the YOLO model is based on a convolutional neural network (CNN), it may face problems such as low detection efficiency and high consumption of computing and memory resources. Summary of the Invention
[0004] The purpose of this invention is to provide an image detection and classification method based on deep learning, which solves the problems of low efficiency and memory resource consumption caused by the large amount of computation in image detection in the prior art.
[0005] The technical solution adopted in this invention specifically includes the following operational steps:
[0006] Step 1: Collect the image to be detected and preprocess it.
[0007] Step 2: Randomly divide the preprocessed images to be detected into detection datasets and classification datasets.
[0008] Step 3: Label the detection dataset and divide the labeled detection dataset into training set, validation set and test set in a ratio of 7:1.5:1.5.
[0009] Step 4: Construct image detection and classification models based on the original YOLOv8 network, respectively.
[0010] Step 5: Input the training set and validation set into the image detection model for basic training and validation to obtain the optimal image detection model; input the classification dataset into the classification model for advanced training to obtain the optimal classification model.
[0011] Step 6: Input the test set into the optimal image detection model for detection and obtain the detection results.
[0012] Step 7: Input the detection results into the optimal classification model to obtain the classified image.
[0013] The invention is further characterized by:
[0014] In step 1, the images to be detected are collected from the Internet. The preprocessing in step 1 involves rotating, flipping, adding noise, changing brightness, and grayscale of the images to be detected in sequence to achieve data enhancement of no less than eight times.
[0015] In step 3, the labelimg tool is used for annotation, and the annotation format is YOLO.
[0016] Step 4 involves constructing an image detection model based on the original YOLOv8 network as follows:
[0017] Replace the Conv modules in layers 2, 4, 6, and 8 of the original YOLOv8 network with GhostConv modules; then add EMA modules after layers 18 and 21 of the original YOLOv8 network.
[0018] The specific steps for basic training and validation of the input image detection model in step 5 are as follows:
[0019] Step 5.1: Input the training set into the image detection model for the first training;
[0020] Step 5.2: Input the validation set into the image detection model trained for the first time in Step 5.1 for validation, obtain the validation results, and use the accuracy evaluation index to evaluate the accuracy of the validation results to determine whether the accuracy of the validation results has reached the optimal level.
[0021] Step 5.3: If the accuracy of the verification result reaches the optimal level, the optimal image detection model is obtained. If the accuracy of the verification result does not reach the optimal level, adjust the accuracy evaluation parameters and repeat steps 5.1 to 5.2.
[0022] In step 5.2, the accuracy evaluation metric uses the loss function. In step 5.3, the criterion for determining whether the accuracy of the verification result has reached the optimal level is whether the loss function curve converges. If the loss function curve converges, the optimal image detection model is obtained. The parameter for accuracy evaluation in step 5.3 is the number of training iterations of the loss function.
[0023] The steps in step 5 for inputting the classification dataset into the classification model for advanced training are as follows: input the classification dataset into the classification model for training, select the Adam optimizer, adjust the number of training iterations until the accuracy is not less than 99.8%, and obtain the optimal classification model.
[0024] In step 6, the detection results are divided into first-class image data and suspected image data.
[0025] The suspected image data is input into the optimal classification model for classification, resulting in second-class and third-class image data; the first-class and second-class image data are then stored in a MySQL database.
[0026] The beneficial effects of this invention are:
[0027] To address the challenges of complex image shapes, large numbers, and varying sizes, this paper introduces an efficient multi-scale attention module (EMA) into the YOLOv8 neck network. This module preserves information in each channel while reducing computational load, improving the speed, accuracy, and robustness of object detection. Simultaneously, replacing some Conv modules with GhostConv modules in the YOLOv8 backbone network reduces the number of network parameters and improves computational efficiency. A novel image detection model is then proposed. These modifications result in a more lightweight image detection model with higher detection performance, better meeting current image detection needs. The detection results from this model are used as input to a classification model to further improve accuracy. Attached Figure Description
[0028] Figure 1 This is a flowchart of the deep learning-based image detection and classification method of the present invention;
[0029] Figure 2 This is a network structure diagram of the image detection model in the deep learning-based image detection and classification method of this invention;
[0030] Figure 3 This is a schematic diagram of the GhostConv module structure in the deep learning-based image detection and classification method of this invention;
[0031] Figure 4 This is a schematic diagram of the CSPlayer_2Conv module structure in the deep learning-based image detection and classification method of this invention;
[0032] Figure 5 This is a schematic diagram of the classification module structure in the deep learning-based image detection and classification method of this invention. Detailed Implementation
[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0034] Image detection and classification methods based on deep learning, such as Figure 1 As shown, the specific operation steps are as follows:
[0035] Step 1: Collect the image to be detected and preprocess it.
[0036] In step 1, the images to be detected are collected from the Internet. The preprocessing in step 1 involves rotating, flipping, adding noise, changing brightness, and grayscale of the images to be detected in sequence to achieve data enhancement of no less than eight times.
[0037] Step 2: Randomly divide the preprocessed images to be detected into detection datasets and classification datasets.
[0038] Step 3: Label the detection dataset and divide the labeled detection dataset into training set, validation set and test set in a ratio of 7:1.5:1.5.
[0039] In step 3, the labelimg tool is used for annotation, and the annotation format is YOLO.
[0040] Step 4: Construct image detection and classification models based on the original YOLOv8 network, respectively.
[0041] Step 4 involves constructing an image detection model based on the original YOLOv8 network as follows:
[0042] like Figure 2 As shown, the Conv modules in layers 2, 4, 6, and 8 of the original YOLOv8 network are replaced with GhostConv modules; then, EMA modules are added after layers 18 and 21 of the original YOLOv8 network; the structure of the GhostConv module is as follows. Figure 3 As shown.
[0043] Step 5: Input the training set and validation set into the image detection model for basic training and validation to obtain the optimal image detection model; input the classification dataset into the classification model for advanced training to obtain the optimal classification model.
[0044] The specific steps for basic training and validation of the input image detection model in step 5 are as follows:
[0045] Step 5.1: Input the training set into the image detection model for the first training;
[0046] Step 5.2: Input the validation set into the image detection model trained for the first time in Step 5.1 for validation, obtain the validation results, and use the accuracy evaluation index to evaluate the accuracy of the validation results to determine whether the accuracy of the validation results has reached the optimal level.
[0047] Step 5.3: If the accuracy of the verification result reaches the optimal level, the optimal image detection model is obtained. If the accuracy of the verification result does not reach the optimal level, adjust the accuracy evaluation parameters and repeat steps 5.1 to 5.2.
[0048] In step 5.2, the accuracy evaluation metric uses the loss function. In step 5.3, the criterion for determining whether the accuracy of the verification result has reached the optimal level is whether the loss function curve converges. If the loss function curve converges, the optimal image detection model is obtained. The parameter for accuracy evaluation in step 5.3 is the number of training iterations of the loss function.
[0049] The steps in step 5 for inputting the classification dataset into the classification model for advanced training are as follows: input the classification dataset into the classification model for training, select the Adam optimizer, adjust the number of training iterations until the accuracy is not less than 99.8%, and obtain the optimal classification model.
[0050] Step 6: Input the test set into the optimal image detection model for detection and obtain the detection results.
[0051] In step 6, the detection results are divided into first-class image data and suspected image data.
[0052] Step 7: Input the detection results into the optimal classification model to obtain the classified image.
[0053] The suspected image data is input into the optimal classification model for classification, resulting in second-class and third-class image data; the first-class and second-class image data are then stored in a MySQL database.
[0054] The image rotation formula is i′=icosθ-jsinθ, j′=isinθ+jcosθ, where i represents the x-coordinate of a pixel in the original image, j represents the y-coordinate of a pixel in the original image, i′ represents the x-coordinate of the image after the rotation transformation of the corresponding pixel i, j′ represents the y-coordinate of the image after the rotation transformation of the corresponding pixel j, and θ represents the angle of image rotation. Other preprocessing methods are similar to the rotation process, all of which involve transforming the image pixels.
[0055] The image detection model consists of a backbone, a neck region, and a detection head. The backbone comprises five parts. The first layer consists of a single 3x3 convolution with a stride of 2 and padding of 1. The second, third, and fourth parts are each composed of the GhostConv module and CSPlayer_2Conv(C2f), and their module structure diagrams are shown below. Figure 3 , 4 As shown, the GhostConv module maintains the original size and channel size of the convolutional output feature map. First, it generates half the feature map using a convolution of half the original size. Then, it obtains the other half through a linear Cheap operation function. Finally, it concatenates the two feature maps into a complete feature map using the Concat function. This method reduces the learning cost of non-critical features: by combining a small number of convolutional kernels with cheaper linear transformation operations instead of conventional convolution, it effectively reduces the computational resource requirements without affecting model performance. The first part of the image information outputs 320*320*64 pixels, the second part outputs 160*160*128 pixels, the third part outputs 80*80*256 pixels, the fourth part outputs 40*40*512 pixels, and the fifth part consists of the GhostConv module, CSPlayer_2Conv (C2f), and SPPF. The neck section contains concatenation (Concat), upsampling (Upsample), C2F, Conv, and an efficient multi-scale attention module (EMA), with their connections as shown below. Figure 2 As shown. The EMA efficient multi-scale attention module is described as follows: it selects only the shared components of 1x1 convolutions from the CA module. To aggregate multi-scale spatial structure information, it places 3x3 kernels and 1x1 branches in parallel to achieve fast response. Considering feature grouping and multi-scale structure, effectively establishing short-term and long-term dependencies is beneficial to obtaining better performance. The training of the image model refers to inputting the training set and validation set into the image detection model. The training part is to organize the detection dataset into the standard YOLOv8 input, select RMSProp as the optimizer, and set the learning rate to 1e. -3 The batch size is 16, and the number of training iterations is no less than 300. As the number of training iterations increases, the loss function curve of the image detection model gradually converges. When the loss function curve converges and stabilizes, the image detection model is trained to the optimal level, and finally a detection model with an overall accuracy of no less than 97.2% is obtained.
[0056] like Figure 5 As shown, training the classification model involves rearranging the classification dataset into standard inputs for the model, selecting Adam as the optimizer, and setting a learning rate of 1e. -3 The batch size is 48, the number of training iterations is no less than 100, and the final classification model achieves an accuracy of no less than 99.8%.
[0057] Example 1
[0058] This invention relates to an image detection and classification method based on deep learning, which specifically includes the following steps:
[0059] Step 1: Collect the images to be detected from the Internet and preprocess them.
[0060] Step 2: Randomly divide the preprocessed images to be detected into detection datasets and classification datasets.
[0061] Step 3: Label the detection dataset and divide the labeled detection dataset into training set, validation set and test set in a ratio of 7:1.5:1.5.
[0062] Step 4: Construct image detection and classification models based on the original YOLOv8 network, respectively.
[0063] The steps to construct an image detection model based on the original YOLOv8 network are as follows:
[0064] Replace the Conv modules in layers 2, 4, 6, and 8 of the original YOLOv8 network with GhostConv modules; then add EMA modules after layers 18 and 21 of the original YOLOv8 network.
[0065] The steps for inputting a classification dataset into a classification model for advanced training are as follows: input the classification dataset into the classification model for training, select the Adam optimizer, adjust the number of training iterations until the accuracy is not less than 99.8%, and obtain the optimal classification model.
[0066] Step 5: Input the training set and validation set into the image detection model for basic training and validation to obtain the optimal image detection model; input the classification dataset into the classification model for advanced training to obtain the optimal classification model.
[0067] Step 6: Input the test set into the optimal image detection model for detection and obtain the detection results. The detection results are divided into first-class image data and suspected image data.
[0068] Step 7: Input the suspected image data from the detection results into the optimal classification model to obtain the second and third class image data.
[0069] Example 2
[0070] This invention relates to an image detection and classification method based on deep learning, which specifically includes the following steps:
[0071] Step 1: Collect the images to be detected from the Internet and preprocess them.
[0072] Step 2: Randomly divide the preprocessed images to be detected into detection datasets and classification datasets.
[0073] Step 3: Label the detection dataset and divide the labeled detection dataset into training set, validation set and test set in a ratio of 7:1.5:1.5.
[0074] Step 4: Construct image detection and classification models based on the original YOLOv8 network, respectively.
[0075] The steps to construct an image detection model based on the original YOLOv8 network are as follows:
[0076] Replace the Conv modules in layers 2, 4, 6, and 8 of the original YOLOv8 network with GhostConv modules; then add EMA modules after layers 18 and 21 of the original YOLOv8 network.
[0077] The steps for inputting a classification dataset into a classification model for advanced training are as follows: input the classification dataset into the classification model for training, select the Adam optimizer, adjust the number of training iterations until the accuracy is not less than 99.8%, and obtain the optimal classification model.
[0078] Step 5: Input the training set and validation set into the image detection model for basic training and validation to obtain the optimal image detection model; input the classification dataset into the classification model for advanced training to obtain the optimal classification model.
[0079] The specific steps for basic training and validation of the input image detection model using the training and validation sets are as follows:
[0080] Step 5.1: Input the training set into the image detection model for the first training;
[0081] Step 5.2: Input the validation set into the image detection model trained for the first time in Step 5.1 for validation, obtain the validation results, and use the accuracy evaluation index to evaluate the accuracy of the validation results to determine whether the accuracy of the validation results has reached the optimal level.
[0082] Step 5.3: If the accuracy of the verification result reaches the optimal level, the optimal image detection model is obtained. If the accuracy of the verification result does not reach the optimal level, adjust the accuracy evaluation parameters and repeat steps 5.1 to 5.2.
[0083] In step 5.2, the accuracy evaluation metric uses the loss function. In step 5.3, the criterion for determining whether the accuracy of the verification result has reached the optimal level is whether the loss function curve converges. If the loss function curve converges, the optimal image detection model is obtained. The parameter for accuracy evaluation in step 5.3 is the number of training iterations of the loss function. The number of training iterations is set to 300 to obtain a converged loss function curve.
[0084] Step 6: Input the test set into the optimal image detection model for detection and obtain the detection results. The detection results are divided into first-class image data and suspected image data.
[0085] Step 7: Input the suspected image data from the detection results into the optimal classification model to obtain the second and third class image data.
[0086] Example 3
[0087] This invention relates to an image detection and classification method based on deep learning, which specifically includes the following steps:
[0088] Step 1: Collect the images to be detected from the Internet and preprocess them.
[0089] Step 2: Randomly divide the preprocessed images to be detected into detection datasets and classification datasets.
[0090] Step 3: Label the detection dataset and divide the labeled detection dataset into training set, validation set and test set in a ratio of 7:1.5:1.5.
[0091] Step 4: Construct image detection and classification models based on the original YOLOv8 network, respectively.
[0092] The steps to construct an image detection model based on the original YOLOv8 network are as follows:
[0093] Replace the Conv modules in layers 2, 4, 6, and 8 of the original YOLOv8 network with GhostConv modules; then add EMA modules after layers 18 and 21 of the original YOLOv8 network.
[0094] The steps for inputting a classification dataset into a classification model for advanced training are as follows: input the classification dataset into the classification model for training, select the Adam optimizer, adjust the number of training iterations until the accuracy is not less than 99.8%, and obtain the optimal classification model.
[0095] Step 5: Input the training set and validation set into the image detection model for basic training and validation to obtain the optimal image detection model; input the classification dataset into the classification model for advanced training to obtain the optimal classification model.
[0096] The specific steps for basic training and validation of the input image detection model using the training and validation sets are as follows:
[0097] Step 5.1: Input the training set into the image detection model for the first training;
[0098] Step 5.2: Input the validation set into the image detection model trained for the first time in Step 5.1 for validation, obtain the validation results, and use the accuracy evaluation index to evaluate the accuracy of the validation results to determine whether the accuracy of the validation results has reached the optimal level.
[0099] Step 5.3: If the accuracy of the verification result reaches the optimal level, the optimal image detection model is obtained. If the accuracy of the verification result does not reach the optimal level, adjust the accuracy evaluation parameters and repeat steps 5.1 to 5.2.
[0100] In step 5.2, the accuracy evaluation metric uses the loss function. In step 5.3, the criterion for determining whether the accuracy of the verification result has reached the optimal level is whether the loss function curve converges. If the loss function curve converges, the optimal image detection model is obtained. The parameter for accuracy evaluation in step 5.3 is the number of training iterations of the loss function. The number of training iterations is set to 260 to obtain a converged loss function curve.
[0101] Step 6: Input the test set into the optimal image detection model for detection and obtain the detection results. The detection results are divided into first-class image data and suspected image data.
[0102] Step 7: Input the suspected image data from the detection results into the optimal classification model to obtain the second and third class image data.
Claims
1. An image detection and classification method based on deep learning, characterized in that, The specific steps include the following: Step 1: Collect the image to be detected and preprocess the image to be detected; Step 2: Randomly divide the preprocessed images to be detected into detection datasets and classification datasets; Step 3: Label the detection dataset and divide the labeled detection dataset into training set, validation set and test set in a ratio of 7:1.5:1.5; Step 4: Construct image detection and classification models based on the original YOLOv8 network, respectively; Step 5: Input the training set and validation set into the image detection model for basic training and validation to obtain the optimal image detection model; input the classification dataset into the classification model for advanced training to obtain the optimal classification model; Step 6: Input the test set into the optimal image detection model for detection and obtain the detection results; Step 7: Input the detection results into the optimal classification model to obtain the classified image; The steps in step 4 for constructing an image detection model based on the original YOLOv8 network are as follows: replacing the Conv modules in layers 2, 4, 6, and 8 of the original YOLOv8 network with GhostConv modules; then adding EMA modules after layers 18 and 21 of the original YOLOv8 network. The image detection model consists of a backbone, a neck, and a detection head. The backbone comprises five parts. The first layer consists of a single 3x3 convolution with a stride of 2 and padding of 1. The second, third, and fourth parts are each composed of the GhostConv module and CSPlayer_2Conv(C2f). The GhostConv module maintains the original size and channel size of the convolution output feature map. First, it generates half of the feature map using a convolution of half the original size. Then, it obtains the other half of the feature map through a linear Cheap operation function. Finally, it concatenates the two feature maps into a complete feature map using the Concat function.
2. The image detection and classification method based on deep learning according to claim 1, characterized in that, The images to be detected collected in step 1 are from the Internet; the preprocessing in step 1 involves rotating, flipping, adding noise, changing brightness and grayscale of the images to be detected in sequence to achieve data enhancement of no less than eight times.
3. The image detection and classification method based on deep learning according to claim 1, characterized in that, The annotations in step 3 are made using the labelimg tool, and the annotation format is YOLO.
4. The image detection and classification method based on deep learning according to claim 1, characterized in that, The specific steps for basic training and validation of the input image detection model using the training and validation sets in step 5 are as follows: Step 5.1: Train the image detection model using the training set for the first time; Step 5.2: Input the validation set into the image detection model trained for the first time in Step 5.1 for validation, obtain the validation results, and use the accuracy evaluation index to evaluate the accuracy of the validation results to determine whether the accuracy of the validation results has reached the optimal level. Step 5.3: If the accuracy of the verification result reaches the optimal level, the optimal image detection model is obtained. If the accuracy of the verification result does not reach the optimal level, the accuracy evaluation parameters are adjusted, and steps 5.1 to 5.2 are repeated.
5. The image detection and classification method based on deep learning according to claim 4, characterized in that, The accuracy evaluation metric mentioned in step 5.2 uses a loss function. The criterion for judging whether the accuracy of the verification result reaches the optimal level in step 5.3 is whether the loss function curve converges. If the loss function curve converges, the optimal image detection model is obtained. The parameter for accuracy evaluation in step 5.3 is the number of training iterations of the loss function.
6. The image detection and classification method based on deep learning according to claim 1, characterized in that, The steps for inputting the classification dataset into the classification model for advanced training in step 5 are as follows: input the classification dataset into the classification model for training, select the Adam optimizer, adjust the number of training iterations until the accuracy is not less than 99.8%, and obtain the optimal classification model.
7. The image detection and classification method based on deep learning according to claim 6, characterized in that, The detection results in step 6 are divided into first-class image data and suspected image data.
8. The image detection and classification method based on deep learning according to claim 7, characterized in that, The suspected image data is input into the optimal classification model for classification to obtain second-class image data and third-class image data; the first-class image data and the second-class image data are stored in a MySQL database.