A real-time detection method, device and computer storage medium for crop diseases and insect pests based on deep learning
By improving the YOLOv5s model and combining Focal Loss and DIoU-NMS algorithms, the real-time and accuracy issues in pest and disease detection were resolved, the robustness and detection capability of the model were improved, and it was adapted to pest and disease targets at different scales and in complex environments.
Patent Information
- Application Number
- CN202311333787.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-16
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-10-16
AI Technical Summary
Existing deep learning models suffer from problems such as insufficient real-time performance, low accuracy, sensitivity to environmental factors, and difficulty in distinguishing between different pests and diseases, especially for newly emerging or rare pests and diseases.
An improved YOLOv5s model is adopted, replacing the feature extraction backbone network with the CBSM module and the Shuffle_block module from ShuffleNetv2, and introducing the skip connection and attention mechanism GAM module. The model is trained by combining Focal Loss and DIoU-NMS algorithms to enhance its robustness and accuracy.
The model's adaptability to pests and diseases at different scales has been improved, false negatives have been reduced, and detection capabilities in complex environments have been enhanced, resulting in higher detection accuracy and robustness.
Smart Images

Figure CN117351353B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device and computer storage medium for real-time detection of crop diseases and insect pests based on deep learning. Background Art
[0002] In existing technologies, deep learning models usually require a lot of computing resources and time for training and reasoning, which leads to real-time problems. In pest and disease research, timely and accurate detection and identification of pests and diseases are crucial for crop protection, but some existing deep learning models may not meet real-time requirements.
[0003] Deep learning models also have certain problems in terms of accuracy. Since the symptoms and characteristics of pests and diseases may be affected by environmental factors, the model may misjudge or miss the diagnosis. In addition, there may be some confusion and misjudgment when the model distinguishes different pests and diseases. At the same time, due to the diversity and complexity of pests and diseases, there are still certain challenges in accuracy. In particular, for newly emerging pests and diseases or rare pests and diseases, the accuracy of the model may be low. Summary of the invention
[0004] In view of the defects of the above-mentioned prior art, the present invention provides a real-time detection method for crop diseases and insect pests based on deep learning, which solves the problem that the model is difficult to accurately detect the target due to the different sizes and shapes of the pests and the images captured by the camera making the objects far and near, which improves the detection robustness. The present invention provides a real-time detection device for crop diseases and insect pests based on deep learning and a computer storage medium.
[0005] The technical solution of the present invention is as follows: A real-time detection method for crop diseases and insect pests based on deep learning, comprising the following steps:
[0006] Step 1: different types of pests and diseases of different crops and normal images are collected, the collected pest and disease images are preprocessed, and a crop pest and disease database is constructed;
[0007] Step 2: Build a target detection network model;
[0008] Step 3: training the target detection network model using the crop pest and disease database obtained in step 1 to obtain a trained network model;
[0009] Step 4: Collect images of crops to be tested, and use the trained network model to detect whether they are infected with pests and diseases;
[0010] The target detection network model is improved based on YOLOv5s, and the improvement includes replacing the feature extraction backbone network of YOLOv5s by the CBSM module, the Shuffle_block module in ShuffleNetv2 and the SPPF module of YOLOv5s, and the Shuffle_block module leads to two jump connections from the backbone network to the two Concat modules in the PAN of the Neck network of YOLOv5s, and adds an attention mechanism GAM module between the Neck network and the head module of YOLOv5s.
[0011] Furthermore, the Shuffle_block module includes a first Shuffle_block module and a second Shuffle_block module, the structure of the feature extraction backbone network replacing YOLOv5s is a CBSM module, a first Shuffle_block module, a second Shuffle_block module, a second Shuffle_block module, a first Shuffle_block module and an SPPF module connected in sequence, the jump connection is led out by two of the second Shuffle_block modules, the first Shuffle_block module is composed of unit2 and unit1*3 modules, the second Shuffle_block module is composed of unit2 and unit1*3 modules, the unit1 is a block module when the step size is 1 in ShuffleNetv2, and the unit2 is a block module when the step size is 2 in ShuffleNetv2.
[0012] Furthermore, a convolution module with a convolution kernel size of 1*1, a step size of 1, and a padding of 0 is added after the two Concat modules with the skip connection.
[0013] Furthermore, when training the target detection network model in step 3, the data in the crop disease and insect pest database are sequentially subjected to Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling processing and then input into the target detection network model. During training, the focal loss function Focal Loss and the non-maximum suppression algorithm based on the DIoU-NMS distance intersection-over-union ratio are used to replace the NMS non-maximum suppression algorithm of YOLOv5s.
[0014] Furthermore, the focal loss function is defined as:
[0015]
[0016] In the formula (1-p t ) γis the adjustment factor, γ≥0 is the adjustable focusing parameter, p y is the predicted probability, with a value between 0 and 1. It reflects the closeness to category y, p t The larger the value, the closer it is to category y, that is, the more accurate the classification is; at the same time, it also reflects the difficulty of classification, p t The larger the value, the higher the confidence of classification, which means the sample is easier to classify. t Approaching 1, (1-p t ) γ It approaches 0, which means that the weight of accurately classified samples in the loss function is reduced; the weight factor α is introduced to reduce the weight of positive samples or negative samples to solve the problem of sample imbalance.
[0017] Furthermore, the non-maximum suppression algorithm based on the DIoU-NMS distance intersection-over-union ratio replaces the NMS non-maximum suppression algorithm of YOLOv5s by replacing the IoU loss function with the DIoU loss function. The DIoU loss function is defined as
[0018]
[0019] b and b gt They are the prediction box B and the target box B gt The center point, ρ 2 (b,b gt ) is the square of the Euclidean distance, C is the distance between the covered prediction box B and the target box B gt The minimum diagonal length of the bounding box.
[0020] Furthermore, the preprocessing in step 1 includes rotation, scaling, cropping, occlusion, color change, adding Gaussian noise and data labeling.
[0021] The present invention also provides a real-time detection device for crop diseases and insect pests based on deep learning, comprising:
[0022] Data collection module: collects different types of pests and diseases of different crops and normal images, pre-processes the collected pest and disease images, and builds a crop pest and disease database;
[0023] Detection network module: build target detection network model;
[0024] Network training module: using the crop pest and disease database obtained by the data acquisition module to train the target detection network model to obtain a trained network model;
[0025] And, the detection module: collects images of crops to be detected, and uses the trained network model to detect whether they are infected with pests and diseases;
[0026] The target detection network model is improved based on YOLOv5s, and the improvement includes replacing the feature extraction backbone network of YOLOv5s by the CBSM module, the Shuffle_block module in ShuffleNetv2 and the SPPF module of YOLOv5s, and the Shuffle_block module leads to two jump connections from the backbone network to the two Concat modules in the PAN of the Neck network of YOLOv5s, and adds an attention mechanism GAM module between the Neck network and the head module of YOLOv5s.
[0027] Another technical solution of the present invention is a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the real-time detection method for crop diseases and insect pests based on deep learning is implemented.
[0028] The advantages of the technical solution provided by the present invention are:
[0029] The shuffle_block module leads to two jump connections of two Concat modules in the PAN from the backbone network to the Neck network of YOLOv5s, forming the BiFPN structure. Giving weights to the Concat modules strengthens the network's use of effective information, integrates richer detail information and semantic information at a lower cost, and can adaptively adjust the scale of the feature pyramid according to the importance and contribution of the features. If the features of a certain level contribute more to the target detection task, BiFPN will increase the resolution of the feature map of that level. Conversely, if the features of a certain level contribute less to the task, BiFPN will reduce the resolution of the feature map of that level. This can better adapt to pests and diseases targets of different scales and improve the robustness of the model. At the same time, a self-attention mechanism is added to better capture the contextual information of the object during target detection to improve the accuracy of the model.
[0030] The strategy of improving the YOLOv5s model with the module of the lightweight network ShuffleNetv2 can achieve the purpose of real-time dynamic monitoring with fast response speed. At the same time, Focal Loss and DIoU-NMS are used to improve the training process, which can more effectively detect pests and diseases blocked by branches and leaves, adapt to more complex environments, reduce missed detections, and further enhance the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a schematic flow chart of the real-time detection method for crop diseases and insect pests based on deep learning of the present invention.
[0032] Figure 2 Schematic diagram of the target detection network model structure of the present invention.
[0033] Figure 3 It is a schematic diagram of the structure of the prior art YOLOv5s model.
[0034] Figure 4 This is a schematic diagram of the structure of the first Shuffle_block module.
[0035] Figure 5 This is a schematic diagram of the structure of the second Shuffle_block module.
[0036] Figure 6 This is a schematic diagram of the unit1 module structure.
[0037] Figure 7 This is a schematic diagram of the unit2 module structure.
[0038] Figure 8 This is the result diagram of the PR curve.
[0039] Fig. 9 This is the result graph of P (precision).
[0040] Fig.10 This is the result graph of F1 (the harmonic mean of precision and recall).
[0041] Fig.11 This is the result graph of R (recall rate).
[0042] Fig.12 Result graph for visualization of loss and accuracy. DETAILED DESCRIPTION
[0043] The present invention is further described below in conjunction with examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading this description, various equivalent modifications to this description by those skilled in the art fall within the scope defined by the claims attached to this application.
[0044] The real-time detection device for crop pests and diseases based on deep learning in the embodiment of the present invention comprises a data acquisition module, a detection network module, a network training module and a detection module. The data acquisition module is used to acquire different pest and disease types and normal images of different crops, pre-process the collected pest and disease images, and construct a crop pest and disease database; the detection network module is used to construct a target detection network model; the network training module is used to train the target detection network model using the crop pest and disease database obtained by the data acquisition module to obtain a trained network model; the detection module is used to acquire images of crops to be detected, and the trained network model is used to detect whether they are infected with pests and diseases.
[0045] Please combine Figure 1As shown, the real-time detection method of crop diseases and insect pests based on deep learning is as follows:
[0046] Step 1: Use a crawler to obtain images of crop diseases and pests on the Internet. Taking corn as an example, there are eight kinds of diseases and pests, namely: large leaf spot, crazy top disease, brown spot, gall smut, head smut, ear rot, small leaf spot, and rust.
[0047] Different types of pests and diseases of different crops and normal images are collected. The collected images are preprocessed, and the preprocessing operations involve rotation, scaling, cropping, occlusion, color change, adding Gaussian noise and data annotation. After preprocessing, the images are used to construct the final crop pest and disease database.
[0048] Step 2: Build a target detection network model based on the YOLOv5s model, such as Figure 2 shown.
[0049] The target detection network model constructed in this step is based on the YOLOv5s model (such as Figure 3 As shown in FIG, the feature extraction backbone network (Backbone) is replaced with the CBSM module, the first Shuffle_block module, the second Shuffle_block module, the second Shuffle_block module, the first Shuffle_block module and the SPPF module connected in sequence. The SPPF module is the original module in the feature extraction backbone network of the YOLOv5s model.
[0050] The CBSM module consists of a convolutional layer, a BN layer, a SiLu activation layer, and a maximum pooling layer. The convolutional kernel size of the convolutional layer is 3*3, the stride is 1, the padding is 0, and the number of convolutional kernels is 64. The window size of the maximum pooling layer is 2*2, and the stride is 2.
[0051] Please combine Figure 4 , Figure 5 As shown, the first Shuffle_block module and the second Shuffle_block module are both composed of block modules in the lightweight network Shufflenetv2. Specifically, the first Shuffle_block module is composed of unit2 and unit1*3 modules, and the second Shuffle_block module is composed of unit2 and unit1*5 modules. Figure 6 , Figure 7 As shown in the figure, unit1 is the block module when the step size is 1 in ShuffleNetv2, and unit2 is the block module when the step size is 2 in ShuffleNetv2.
[0052] In the feature extraction backbone network composed of the CBSM module, the first Shuffle_block module, the second Shuffle_block module, the second Shuffle_block module, the first Shuffle_block module and the SPPF module, the outputs of the two second Shuffle_block modules and the SPPF module are respectively connected to the Neck network of the YOLOv5s model, and the outputs of the two Shuffle_block modules are jump-connected to the two Concat modules of the original PAN part of the Neck network of the YOLOv5s model. A convolution module is added after each of the two Concat modules, and the convolution kernel size of the convolution module is 1*1, the step size is 1, the padding is 0, and the number of convolution kernels is half of the original number of channels. A weighted bidirectional feature pyramid network (Bi-directional Feature Pyramid Network, BiFPN) structure is constructed in the Neck network, the original feature pyramid (Feature Pyramid Network, FPN) and path aggregation network (Path Aggregation Network, PAN) are improved, and multi-scale fusion of different feature maps is performed.
[0053] Finally, the attention mechanism GAM module is connected between the Neck network and the Head network of the YOLOv5s model. Each layer in the Head network layer has a GAM module with a self-attention mechanism to enhance the extraction of global pest and disease feature information. In this way, an improved target detection network model based on the YOLOv5s model is constructed.
[0054] Step 3: Use the crop pest and disease database obtained in step 1 to train the target detection network model constructed in step 2 to obtain a trained network model.
[0055] During training, the image is subjected to image mosaic data enhancement, adaptive anchor box calculation and adaptive image scaling at the input of the trained network model to improve the inference speed of the model. The Focal Loss loss function is used. On the basis of balancing the cross entropy loss function, the Focal Loss loss function reduces the weight of easy-to-classify samples and increases the weight of difficult-to-classify samples, thereby improving the detection effect of minority categories.
[0056] The Focal Loss loss function is defined as follows:
[0057]
[0058] In the formula (1-p t ) γis the adjustment factor, γ≥0, and is an adjustable focusing parameter. For samples with accurate classification, p t Approaching 1, (1-p t ) γ It approaches 0, which means that the weight of accurately classified samples in the loss function is reduced; at the same time, the weight factor α is introduced to reduce the weight of positive samples or negative samples to solve the problem of sample imbalance.
[0059] The non-maximum suppression algorithm based on DIoU-NMS distance intersection-over-union ratio is used to replace the NMS non-maximum suppression algorithm of YOLOv5s. Specifically, the IoU loss function is replaced by the DIoU loss function. The DIoU loss function is defined as
[0060]
[0061] b and b gt They are the prediction box B and the target box B gt The center point, ρ 2 (b,b gt ) is the square of the Euclidean distance, C is the distance between the covered prediction box B and the target box B gt The minimum diagonal length of the bounding box.
[0062] Step 4: Collect images of crops to be tested, and use the trained network model to detect whether they are infected with pests and diseases.
[0063] A specific training process of this embodiment is as follows: first, the data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1, and the data set includes 8 different crop diseases and insect pests. Then, the input image is adaptively padded and scaled to a size of 640*640, and finally input into the final model of the present invention for training.
[0064] During the training process, the batch-size is set to 4, each batch has 94 images, and the initial learning rate is 0.01. The following figures are the training results after 200 epochs.
[0065] Figure 8 This is the result graph of the PR curve. The proportion of the area under the curve is mAP. The larger the proportion, the better the model. It can be seen from the figure that its mAP is 0.919.
[0066] Fig. 9 The result graph of P (precision) means the probability of actually being a positive sample among all samples predicted to be positive. The precision represents the accuracy of the prediction of the positive sample results. The larger the value, the better. It can be seen from the figure that when the confidence level is 0.5, the precision is about 0.9.
[0067] Fig.10This is the result of F1 (the harmonic mean of precision and recall). The F1 score considers both precision and recall. It can be seen from the figure that when the confidence level is 0.327, both reach the highest value and achieve a balance.
[0068] Fig.11 The result graph of R (recall rate) means the probability of being predicted as a positive sample among the actually positive samples. The larger the value, the better. It can be seen from the figure that when the confidence level is 0.5, the recall rate is about 0.8.
[0069] Fig.12 This is a visualization of the loss and accuracy results. It can be seen from the figure that the loss function shows a gradual downward trend during the training process. The loss function drops sharply in the first 80 epochs, during which the P, R, and mAP parameters rise rapidly. From 80 to 200 epochs, the downward trend of the loss function gradually slows down, and at the same time, the upward trend of P, R, and mAP slows down. At this time, the model is close to converging to the optimal state.
[0070] The losses include positioning loss, confidence loss and classification loss. The classification loss is the loss caused by incorrectly predicting the given object category. The smaller the loss function value, the more accurate the classification. After the present invention introduces Focal Loss, the classification loss function value is significantly lower than before the improvement. The precision and recall converge faster and better than before the improvement.
[0071] It should be noted that the specific methods of the above embodiments can form a computer program product. Therefore, the computer program product implemented in this application can be stored on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.).
Claims
1. A real-time detection method for crop diseases and insect pests based on deep learning, characterized in that: The following steps are involved: Step 1: different types of pests and diseases of different crops and normal images are collected, the collected pest and disease images are preprocessed, and a crop pest and disease database is constructed; Step 2: Build a target detection network model; Step 3: training the target detection network model using the crop pest and disease database obtained in step 1 to obtain a trained network model; Step 4: Collect images of crops to be tested, and use the trained network model to detect whether they are infected with pests and diseases; The target detection network model is improved based on YOLOv5s, and the improvement includes replacing the feature extraction backbone network of YOLOv5s by the CBSM module, the Shuffle_block module in ShuffleNetv2, and the SPPF module of YOLOv5s, and deriving two jump connections from the backbone network to the two Concat modules in the PAN of the Neck network of YOLOv5s by the Shuffle_block module, and adding an attention mechanism GAM module between the Neck network and the head module of YOLOv5s, and adding a convolution module with a convolution kernel size of 1*1, a step size of 1, and a padding of 0 after the two Concat modules with the jump connection; The CBSM module is composed of a convolution layer, a BN layer, a SiLu activation layer and a maximum pooling layer. The convolution kernel size of the convolution layer is 3*3, the stride is 1, the padding is 0, and the number of convolution kernels is 64. The Shuffle_block module includes a first Shuffle_block module and a second Shuffle_block module. The structure of the feature extraction backbone network replacing YOLOv5s is a CBSM module, a first Shuffle_block module, a second Shuffle_block module, a second Shuffle_block module, a first Shuffle_block module and an SPPF module connected in sequence. The jump connection is derived from two of the second Shuffle_block modules. The first Shuffle_block module is composed of unit2 and unit1*3 modules, and the second Shuffle_block module is composed of unit2 and unit1*5 modules. The unit1 is a block module when the step size is 1 in ShuffleNetv2, and the unit2 is a block module when the step size is 2 in ShuffleNetv2.
2. The real-time detection method for crop diseases and insect pests based on deep learning according to claim 1, characterized in that: When training the target detection network model in step 3, the data in the crop disease and insect pest database are sequentially subjected to Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling processing and then input into the target detection network model. During training, the focal loss function Focal Loss and the non-maximum suppression algorithm based on the DIoU-NMS distance intersection-over-union ratio are used to replace the NMS non-maximum suppression algorithm of YOLOv5s.
3. The real-time detection method for crop diseases and insect pests based on deep learning according to claim 2, characterized in that: The focal loss function is defined as Where (1-p t ) γ is the adjustment factor, γ≥0 is the adjustable focusing parameter, p t is the predicted probability, with a value between 0 and 1.
4. The method for real-time detection of crop diseases and insect pests based on deep learning according to claim 2, characterized in that: The non-maximum suppression algorithm based on DIoU-NMS distance intersection ratio replaces the NMS non-maximum suppression algorithm of YOLOv5s by replacing the IoU loss function with the DIoU loss function. The DIoU loss function is defined as b and They are the prediction box B and the target box The center point of is the square of the Euclidean distance, C is the coverage of the prediction box B and the target box The minimum diagonal length of the bounding box.
5. The method for real-time detection of crop diseases and insect pests based on deep learning according to claim 1, characterized in that: The preprocessing in step 1 includes rotation, scaling, cropping, occlusion, color change, adding Gaussian noise and data labeling.
6. A real-time detection device for crop diseases and insect pests based on deep learning, characterized in that: include: Data collection module: collects different types of pests and diseases of different crops and normal images, pre-processes the collected pest and disease images, and builds a crop pest and disease database; Detection network module: build target detection network model; Network training module: using the crop pest and disease database obtained by the data acquisition module to train the target detection network model to obtain a trained network model; And, the detection module: collects images of crops to be detected, and uses the trained network model to detect whether they are infected with pests and diseases; The target detection network model is improved based on YOLOv5s, and the improvement includes replacing the feature extraction backbone network of YOLOv5s by the CBSM module, the Shuffle_block module in ShuffleNetv2, and the SPPF module of YOLOv5s, and deriving two jump connections from the backbone network to the two Concat modules in the PAN of the Neck network of YOLOv5s by the Shuffle_block module, and adding an attention mechanism GAM module between the Neck network and the head module of YOLOv5s, and adding a convolution module with a convolution kernel size of 1*1, a step size of 1, and a padding of 0 after the two Concat modules with the jump connection; The CBSM module is composed of a convolution layer, a BN layer, a SiLu activation layer and a maximum pooling layer. The convolution kernel size of the convolution layer is 3*3, the stride is 1, the padding is 0, and the number of convolution kernels is 64. The Shuffle_block module includes a first Shuffle_block module and a second Shuffle_block module. The structure of the feature extraction backbone network replacing YOLOv5s is a CBSM module, a first Shuffle_block module, a second Shuffle_block module, a second Shuffle_block module, a first Shuffle_block module and an SPPF module connected in sequence. The jump connection is derived from two of the second Shuffle_block modules. The first Shuffle_block module is composed of unit2 and unit1*3 modules, and the second Shuffle_block module is composed of unit2 and unit1*5 modules. The unit1 is a block module when the step size is 1 in ShuffleNetv2, and the unit2 is a block module when the step size is 2 in ShuffleNetv2.
7. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the real-time detection method for crop diseases and insect pests based on deep learning as described in any one of claims 1 to 5 is implemented.
