Plant leaf disease detection method based on deep learning, computer device and storage medium

Abstract

The application discloses a plant leaf disease detection method based on deep learning, a computer device and a storage medium. The plant leaf disease detection method based on deep learning comprises the following steps: acquiring a plant leaf image to be detected; and identifying and detecting the plant leaf image by using an optimized YOLOv5s network model to obtain the category and position of the plant leaf disease. The optimized YOLOv5s network model comprises a backbone network and a neck network, the neck network comprises a plurality of optimized CSP structures and a CAM structure, the optimized CSP structure comprises a Ghost structure and an inverted residual structure, and the optimized CAM structure comprises an EPSA attention module and an inverted residual structure. The application trains image data, fuses features with inconsistent semantics and scales to enhance the extraction capability of global information of the network, solves the problems that a prediction box center point offset amount is difficult to obtain a special point and a prediction box is incorrectly scaled in a model training process, has a lower model parameter amount, a higher average precision, a stronger robustness and a better detection effect.

Description

Technical Field

[0001] This application relates to the field of deep learning image processing technology, and in particular to a deep learning-based method for detecting plant leaf diseases, a computer device, and a storage medium. Background Technology

[0002] Accurately identifying plant diseases and pinpointing their locations during the early stages of plant growth can effectively prevent their spread and ensure normal plant growth. However, traditional plant disease detection relies on manual identification, which is not only costly in terms of manpower and resources but also inefficient and inaccurate.

[0003] The development of deep learning technology has brought faster speed and higher accuracy to image recognition and detection. In order to overcome the shortcomings of traditional methods, deep learning technology can be used to intelligently complete the entire process from extracting information on plant leaf diseases to recognition and detection. Summary of the Invention

[0004] Therefore, it is necessary to address the technical challenges of detecting plant leaf diseases that are small in size, dense in number, and similar in appearance, which are difficult, prone to errors, and lack precision.

[0005] This invention provides a deep learning-based method for detecting plant leaf diseases, comprising:

[0006] Acquire images of plant leaves to be inspected for diseases;

[0007] An optimized YOLOv5s network model was used to identify and detect plant leaf images to obtain the types and locations of plant leaf diseases.

[0008] The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network includes several optimized CSP and CAM structures. The optimized CSP structure includes a Ghost structure and an inverted residual structure. The optimized CAM structure includes an EPSA attention module and an inverted residual structure.

[0009] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0010] Optionally, the optimized YOLOv5s network model is pre-trained using samples, and the training process includes:

[0011] Obtain a sample and set the position of the ground truth bounding box on the sample;

[0012] Define a prediction box, use the first formula to calculate the extracted sample image features, and obtain the position of the prediction box relative to the ground truth box;

[0013] The second formula is used to calculate the position of the predicted box relative to the ground truth box. The loss value of the predicted result and the ground truth result is calculated. The model parameters are updated by backpropagation based on the loss value, and then the YOLOv5s network model is trained to obtain the corresponding optimized model.

[0014] Optionally, the first formula includes:

[0015] x = 2.2 × σ(t) x -0.6

[0016] y = 2.2 × σ(t) y -0.6

[0017]

[0018]

[0019] in,

[0020] b x b y These represent the positions of the predicted target center point;

[0021] σ represents the Sigmoid function;

[0022] t x t y These represent the offsets of the target center x and y coordinates of the prediction grid relative to the top left corner of the prediction grid, respectively.

[0023] c x c y This is represented by the original x and y coordinates of the top left corner of the corresponding grid.

[0024] Optionally, the second formula comprises:

[0025]

[0026] in,

[0027] ρ 2 (l,l gt ) represents the square of the Euclidean distance between the top-left corner of the ground truth bounding box and the top-left corner of the predicted bounding box;

[0028] ρ 2 (,b gt ) represents the square of the Euclidean distance from the center point of the ground truth bounding box to the center point of the predicted bounding box;

[0029] w gtand w represent the widths of the ground truth bounding box and the predicted bounding box, respectively;

[0030] c represents the diagonal distance between the closure regions of the two target boxes;

[0031] IoU represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box areas.

[0032] Optionally, the optimized CAM structure constructs a multi-branch network structure using EPSA attention modules and inverted residual structures;

[0033] The optimized CSP structure constructs a multi-branch network structure through Ghost structure and inverted residual structure.

[0034] Optionally, the backbone network includes a CSPDarknet53 network and an SPP structure and a CSP structure set behind the CSPDarknet53 network.

[0035] Optionally, the neck network includes four convolutional layers, four optimized CSP structures, and two optimized CAM structures. Each convolutional layer corresponds to one batch normalization layer and one activation layer.

[0036] Optionally, the output layer of the optimized YOLOv5s network model includes three convolutional layers, which can be of three sizes: 20×20, 40×40, and 80×80.

[0037] This application also provides the following technical solutions:

[0038] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0039] Acquire images of plant leaves to be inspected for diseases;

[0040] An optimized YOLOv5s network model was used to identify and detect plant leaf images to obtain the types and locations of plant leaf diseases.

[0041] The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network includes several optimized CSP and CAM structures. The optimized CSP structure includes a Ghost structure and an inverted residual structure. The optimized CAM structure includes an EPSA attention module and an inverted residual structure.

[0042] This application also provides the following technical solutions:

[0043] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to perform the following steps: acquiring images of plant leaves of a disease to be detected;

[0044] An optimized YOLOv5s network model was used to identify and detect plant leaf images to obtain the types and locations of plant leaf diseases.

[0045] The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network includes several optimized CSP and CAM structures. The optimized CSP structure includes a Ghost structure and an inverted residual structure. The optimized CAM structure includes an EPSA attention module and an inverted residual structure.

[0046] This application provides a deep learning-based method, computer device, and storage medium for detecting plant leaf diseases. It enhances the ability to extract global information from the network, solves the problems of difficulty in obtaining special points for the center point offset of the prediction box and the incorrect scaling of the prediction box during model training, and finally performs plant leaf disease identification and detection. It solves the problem of poor detection effect for small targets, dense numbers, and similar appearance of plant leaf diseases, and the final detection effect is more accurate with a higher average precision. Attached Figure Description

[0047] Figure 1 A schematic diagram of the structure of a deep learning-based plant leaf disease detection method provided in this application;

[0048] Figure 2 This is a schematic diagram of the optimized CSP structure;

[0049] Figure 3 This is a schematic diagram of the optimized CAM structure;

[0050] Figure 4 A schematic diagram of the prediction grid used when the target center point falls at different positions on the feature map;

[0051] Figure 5 A schematic diagram of a deep learning-based plant leaf disease detection method provided in this application;

[0052] Figure 6 The detection results of each model are shown in the image.

[0053] Figure 7 This is an internal structural diagram of a computer device in one embodiment.

[0054] The annotations in the figure are explained as follows:

[0055] 10. Predicted grid; 20. Predicted grid; 30. Ground truth bounding box. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] refer to Figure 1 The deep learning-based plant leaf disease detection method of this application includes:

[0058] S100. Acquire images of plant leaves to be detected for the disease;

[0059] S200. An optimized YOLOv5s network model is used to identify and detect the plant leaf images to obtain the type and location of plant leaf diseases.

[0060] The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network comprises several optimized CSP and CAM structures. The optimized CSP structures include a Ghost structure and an inverted residual structure. The optimized CAM structure includes an EPSA attention module and an inverted residual structure. Specifically, the optimized CSP structure replaces the original convolutional modules with Ghost structures and inverted residual structures.

[0061] Understandably, compared to the complex feature extraction process in traditional algorithms, image feature extraction modules based on deep learning technology can quickly and easily extract image features, including color, texture, and shape. In image detection problems, multi-layer image feature extraction modules extract rich detail information from low-level features and rich semantic information from high-level features, resulting in better feature robustness.

[0062] In this application, the optimized CAM structure constructs a multi-branch network structure using EPSA attention modules and inverted residual structures; the optimized CSP structure constructs a multi-branch network structure using Ghost structures and inverted residual structures.

[0063] For details, please refer to Figure 2The optimized CSP structure divides the upper-layer input into two parts, each using a convolutional layer to adjust the channel dimension. The output of one branch then replaces the two ordinary convolutional layers in the original CSP structure with a Ghost module. This part of the structure reduces computational cost while maintaining model accuracy. The output of this part is further divided into two parts: one part uses an inverse residual structure, which first increases the dimensionality through a convolutional layer to increase the number of input and output channels of the depthwise convolution, thus extracting more information. The other part uses a global average pooling layer to extract global information, enhancing the ability to extract global information. These two parts can extract different feature information from the input layer, avoiding data redundancy caused by repeatedly extracting the same features. Finally, the outputs of these three parts are concatenated along the channel dimension, and then a convolutional layer is used to adjust the final output channel number. The improved CSP structure uses two lightweight structures to extract feature information: the Ghost module and the inverse residual structure. The optimized CSP structure can extract more feature information and enhance the model's detection performance.

[0064] refer to Figure 3 The optimized CAM structure employs a plug-and-play, highly efficient multi-scale channel attention mechanism, namely the EPSA module. The optimized CAM module accepts inputs from both deep and shallow networks, with each input divided into two branches. Global average pooling and global max pooling produce different effects in different layers. In shallow networks, the image's abstraction level is not very high, and the image contains more texture features. As the network deepens, the resulting feature maps become increasingly abstract, containing more semantic and background information. Therefore, the shallow network input in the optimized CAM structure passes through a global max pooling layer to remove some invalid information, making the resulting feature maps more sensitive to detail. The deep network input, on the other hand, uses a global average pooling layer to integrate global spatial information and obtain global contextual information. The optimized CAM structure also fuses feature information from the two upper-layer inputs, preserving the original input features and supplementing the feature information lost after pooling operations. The optimized CAM structure can achieve feature fusion of global and local features, and can better integrate features with inconsistent semantics and scale, thereby enhancing the network's ability to extract global information and improving the accuracy of the network model.

[0065] The optimized YOLOv5s network model in this application is pre-trained using samples. The training process includes:

[0066] Obtain a sample and set the position of the ground truth bounding box on the sample;

[0067] Define the predicted bounding box, use the first formula to calculate the extracted sample image features, and obtain the position of the predicted bounding box relative to the ground truth bounding box.

[0068] The second formula is used to calculate the position of the predicted box relative to the ground truth box. The loss values ​​of the predicted and ground truth results are calculated. The model parameters are updated by backpropagation based on the loss values. The model is then trained again using the SGD optimizer to obtain the corresponding optimized YOLOv5s network model.

[0069] This application can calculate the mean accuracy (mAP) of plant leaf diseases based on the results obtained from the optimized YOLOv5s network model, and compare the mean accuracy with existing technologies. The higher the mAP, the better the detection effect of the model.

[0070] Understandably, in the YOLOv5s network model, nine sets of anchor values ​​are preset. The center point of each preset anchor value falls on the upper left corner of the prediction grid. Then, the prediction box is gradually shifted towards the target center point through the first formula. The closer the prediction box is to the target center point, the better the effect.

[0071] The first formula includes:

[0072] x = 2.2 × σ(t) x -0.6

[0073] y = 2.2 × σ(t) y -0.6

[0074]

[0075]

[0076] in,

[0077] b x b y These represent the positions of the predicted target center point;

[0078] σ represents the Sigmoid function;

[0079] t x t y These represent the offsets of the target center x and y coordinates of the prediction grid relative to the top left corner of the prediction grid, respectively.

[0080] c x c y This is represented by the original x and y coordinates of the top left corner of the corresponding grid.

[0081] like Figure 4As shown, the prediction grids used when the target center point falls at different positions on the feature map are illustrated. The green box represents the ground truth bounding box (30), the blue box represents the prediction grid (10), and the red box represents an additional prediction grid (20) added to increase the number of positive samples during training. The black dot represents the center point of the ground truth bounding box. Understandably, the formula first expands the value range to include all required values, and then restricts the value range to ensure it does not exceed the maximum value required for the offset of the prediction grid center point. According to the original formula, when the required offset of the predicted bounding box center point is 1.5 or -0.5, according to the prediction grid offset formula, t is required. y It tends toward positive infinity or t x The original formula tends towards negative infinity, while the modified formula does not need to tend towards infinity to obtain the desired value.

[0082] The second formula is used to calculate the position of the predicted bounding box relative to the ground truth bounding box, and the loss value between the predicted and ground truth results is calculated. The loss value between the predicted and ground truth bounding boxes is then calculated using the loss function formula, and backpropagation is performed based on this loss value to update the model parameters, resulting in more accurate model localization and higher recognition accuracy. Based on the original loss function formula:

[0083]

[0084] in,

[0085] ρ 2 (,b gt ) is the square of the Euclidean distance from the center point of the ground truth bounding box to the center point of the predicted bounding box.

[0086] c is the diagonal distance between the closure regions of the two target boxes.

[0087] When the distance between two bounding boxes remains constant, the longer the distance c, the smaller the DIoU loss function value. This suggests that the DIoU loss function may reduce its loss by enlarging the predicted bounding box. Based on this, the second formula includes:

[0088]

[0089] in,

[0090] ρ 2 (l,l gt ) represents the square of the Euclidean distance between the top-left corner of the ground truth bounding box and the top-left corner of the predicted bounding box;

[0091] ρ 2 (,b gt ) represents the square of the Euclidean distance from the center point of the ground truth bounding box to the center point of the predicted bounding box;

[0092] w gtand w represent the widths of the ground truth bounding box and the predicted bounding box, respectively;

[0093] c represents the diagonal distance between the closure regions of the two target boxes;

[0094] IoU represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box areas.

[0095] As can be seen from the formula, adding the square of the Euclidean distance between the top-left corners of the ground truth bounding box and the predicted bounding box to the denominator of the original formula reduces the change in the DIoU value when the side length of the predicted bounding box changes, thereby accelerating the model's convergence speed. Simultaneously, the formula was derived based on the parabolic concept. This formula incorporates the ratio of the width of the predicted bounding box to the width of the ground truth bounding box. When the side lengths of the predicted and ground truth bounding boxes are the same... The ratio is 1, but when the side lengths of the predicted box and the ground truth box are not the same, The ratio will be greater than 1 or less than 1, so the loss value will only become smaller when the side lengths of the predicted box and the ground truth box become closer and closer. This formula can solve the problem of the side length of the predicted box being incorrectly scaled.

[0096] In this embodiment, reference Figure 5 The optimized YOLOv5s network model consists of a backbone network and a neck network. The backbone uses a CSPDarknet53 network to extract feature information from the input image from shallow to deep layers, allowing different layers to obtain varying degrees of detail and semantic information. Following this is an SPP structure and a CSP structure, which extract spatial feature information at different sizes, enhancing the model's feature extraction capabilities. The neck network comprises four convolutional layers, four optimized CSP structures, and two optimized CAM structures, further extracting image feature information and fusing feature information at different scales to enrich feature representation and accurately characterize leaf disease information. The output layer of the optimized YOLOv5s network model consists of three convolutional layers, providing three sizes: 20×20, 40×40, and 80×80, corresponding to predictions of large, medium, and small targets. Each convolutional layer is followed by a batch normalization layer and an activation layer to prevent the destruction of convolutional network weights and accelerate network convergence.

[0097] In other embodiments, to verify the accuracy of the plant leaf disease detection method based on deep learning in this application, eight diseases of five plants—grape, peach, potato, apple, and corn—were mixed for training and testing, and compared with other detection network models.

[0098] refer to Figure 6Table 1 shows the experimental results of testing the optimized YOLOv5s network model on a mixed dataset of multiple plant diseases, comparing it with six other detection models (Faster R-CNN, SSD, YoLOv3, YOLOv4, YOLOv4-tiny, and YOLOv5s). The results include model mAP, number of parameters, computational cost, F1 score, recall, and precision. Our model has 7.62M parameters, 10.17G computation, a detection speed of 40.01 FPS, and mAP and F1 scores of 95.92% and 0.91, respectively. As can be seen from Table 1, the optimized YOLOv5s network model has fewer parameters and lower computational cost, meeting the requirements for real-time detection, and also achieves a higher mAP.

[0099] Table 1

[0100]

[0101] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements an image classification method for the severity of plant diseases. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0102] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0104] When a processor executes a computer program, it performs the following steps:

[0105] Acquire images of plant leaves to be inspected for diseases;

[0106] An optimized YOLOv5s network model was used to identify and detect plant leaf images, and the types and locations of plant leaf diseases were obtained.

[0107] The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network includes several optimized CSP and CAM structures. The optimized CSP structures include Ghost structures and inverted residual structures, and the optimized CAM structures include EPSA attention modules and inverted residual structures.

[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0109] When a processor executes a computer program, it performs the following steps:

[0110] Acquire images of plant leaves to be inspected for diseases;

[0111] An optimized YOLOv5s network model was used to identify and detect plant leaf images, and the types and locations of plant leaf diseases were obtained.

[0112] The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network includes several optimized CSP and CAM structures. The optimized CSP structures include Ghost structures and inverted residual structures, and the optimized CAM structures include EPSA attention modules and inverted residual structures.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0114] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered to be within the scope of this specification. When technical features of different embodiments are embodied in the same drawing, it can be regarded as the drawing also disclosing examples of combinations of the various embodiments involved.

[0115] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are quite specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

Claims

1. A deep learning-based method for detecting plant leaf diseases, characterized in that, include: Acquire images of plant leaves to be inspected for diseases; An optimized YOLOv5s network model was used to identify and detect plant leaf images to obtain the types and locations of plant leaf diseases. The optimized YOLOv5s network model includes a backbone network and a neck network. The neck network includes several optimized CSP structures and several optimized CAM structures. The optimized CSP structures include a Ghost structure and an inverted residual structure. The optimized CAM structures include an EPSA attention module and an inverted residual structure. The optimized CSP structure divides the input features into two parts and adjusts the channel dimension of each part through a convolutional layer. Then, one part is divided into two branches after passing through the Ghost structure. One branch uses the inverted residual structure, and the other branch uses the global average pooling layer to extract global features. Finally, the outputs of the three parts are concatenated in the channel dimension, and the number of channels of the final output is adjusted through a convolutional layer. The optimized CAM structure accepts inputs from both deep and shallow networks. The inputs of both deep and shallow networks are divided into two branches. One branch of the deep network extracts global features of the network through a global average pooling layer, while the other branch extracts features through an inverted residual structure and an EPSA module. One branch of the shallow network extracts local features of the network through a global max pooling layer, while the other branch extracts features through an inverted residual structure and an EPSA module to enhance the ability to extract multi-scale feature maps.

2. The plant leaf disease detection method based on deep learning according to claim 1, characterized in that, The optimized YOLOv5s network model is pre-trained using samples, and the training process includes: Obtain a sample and set the position of the ground truth bounding box on the sample; Define a prediction box, use the first formula to calculate the extracted sample image features, and obtain the position of the prediction box relative to the ground truth box; The second formula is used to calculate the position of the predicted box relative to the ground truth box. The loss value of the predicted result and the ground truth result is calculated. The model parameters are updated by backpropagation based on the loss value, and then the YOLOv5s network model is trained to obtain the corresponding optimized model.

3. The plant leaf disease detection method based on deep learning according to claim 2, characterized in that, The first formula includes: in, , These represent the positions of the predicted target center point; Represents the Sigmoid function; , These represent the offsets of the target center x and y coordinates of the prediction grid relative to the top left corner of the prediction grid, respectively. , This is represented by the original x and y coordinates of the top left corner of the corresponding grid.

4. The plant leaf disease detection method based on deep learning according to claim 2, characterized in that, The second formula includes: in, This represents the square of the Euclidean distance between the top-left corners of the ground truth bounding box and the predicted bounding box; This represents the square of the Euclidean distance from the center point of the ground truth bounding box to the center point of the predicted bounding box; and These represent the widths of the ground truth bounding box and the predicted bounding box, respectively. This represents the diagonal distance between the closure regions of the two bounding boxes; This represents the intersection-union ratio (IUGR) of the predicted bounding box and the actual bounding box areas.

5. The plant leaf disease detection method based on deep learning according to claim 1, characterized in that, The backbone network includes the CSPDarknet53 network and the SPP and CSP structures set behind the CSPDarknet53 network.

6. The plant leaf disease detection method based on deep learning according to claim 1, characterized in that, The neck network includes four convolutional layers, four optimized CSP structures, and two optimized CAM structures. Each convolutional layer corresponds to one batch normalization layer and one activation layer.

7. The plant leaf disease detection method based on deep learning according to claim 1, characterized in that, The output layer of the optimized YOLOv5s network model includes three convolutional layers, which can be configured in three sizes: 20×20, 40×40, and 80×80.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images