Potato leaf disease detection and grading method based on deep learning

Abstract

The invention relates to the technical field of machine learning, in particular to a potato leaf disease detection and grading method based on deep learning, which comprises the following steps: collecting pictures containing potato disease leaves in a real natural growth environment; inputting the collected pictures into an AMW-YOLOv8 network model for detection so as to locate the position of a diseased leaf in each picture and obtain a diseased leaf picture; and inputting the disease leaf picture into a DCSegFormer semantic segmentation model to obtain a potato disease grading image. According to the method, disease detection can be performed on multiple leaves in a complex background at the same time, so that the actual requirements of a planting scene are better met.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a potato leaf disease detection and grading method based on deep learning. Background Art

[0002] Potatoes are one of the most important food crops in the world and are widely grown around the world. Potatoes are low in fat and high in energy, rich in nutrients, easy to store and process, and can provide long-term nutritional support for people. At the same time, they are also an important part of diets around the world and one of the basic foods for hundreds of millions of people around the world. Therefore, potatoes play an indispensable role in agricultural production, economic development and improving human well-being. Although my country currently relies on modern science and technology to produce enough potatoes to meet social needs, crop diseases still threaten agricultural production safety. Because potatoes are susceptible to bacteria, fungi and viruses, when potatoes are planted on a large scale, the diseases of individual plants will spread rapidly throughout the entire farmland due to the dense planting between plants, and the annual yield loss caused by diseases is very serious. When the disease intensifies, growers usually choose to spray a large amount of pesticides to deal with it. However, this practice is not only ineffective, but also easy to cause pesticide residues, thereby polluting the environment. Therefore, it is crucial for potato production to accurately and quickly identify the type of potato disease, assess its severity level, and apply pesticides in a timely and accurate manner to control the spread of the disease.

[0003] At present, in most areas, during the potato planting process, the type and severity of potato diseases are mainly determined by farmers observing the appearance characteristics of potato plants with their naked eyes, such as shape and color. Experienced farmers may be able to identify diseases, but most young workers who want to devote themselves to agriculture lack field operation experience and professional knowledge of agricultural disease control, and usually need the help of plant protection experts. However, the number of plant protection experts is limited, and they are restricted by various factors such as working hours, locations, and economic costs. It is impossible to guarantee all-round, all-weather technical support and agricultural production services in the field, which leads to farmers being unable to communicate with plant protection experts in a timely manner. This will result in the inability to effectively diagnose crop diseases in a timely manner, and it is easy to miss the best time for disease prevention and control, so that diseases cannot be prevented and controlled in time, which will also cause crop yields to be damaged, and ultimately lead to economic losses for farmers. With the development of science and technology, agriculture is also moving towards modernization. Traditional machine learning has performed well in various disease identification due to its fast identification and high accuracy. However, when the identification target is in a complex natural environment, the diseased area may be similar to the background color, and the shape and texture information is also complex. In this case, machine learning methods often have difficulty in ensuring accurate regional segmentation, resulting in low recognition accuracy. Therefore, it is of great significance to quickly, accurately and conveniently identify the type and degree of potato diseases to ensure potato yield.

[0004] Deep learning technology can automatically extract image features and classify crop diseases, eliminating a lot of work such as feature extraction and classifier design in traditional image recognition technology. At present, deep learning technology has made great progress in the research of crop disease recognition. The target detection method based on deep learning can not only classify diseases, but also accurately locate the affected area in the image. For example: the SSD (Single Shot Multibox Detector) model is experimentally compared with various other models on the PlantVillage dataset to determine the most suitable model for plant leaf disease classification and location. The experimental results show that the mAP (mean Average Precision) of the model using the Adam (Adaptive Moment Estimation) optimizer reaches 73.07%, which is better than the performance of other models. For example: the original YOLOv5 model is optimized to achieve accurate detection of plant diseases even in harsh outdoor environments, with a mean average precision (mAP) of 70%, which is 5.4% higher than the original YOLOv5 model.

[0005] Deep learning-based methods can not only classify diseases, but also classify the severity of plant diseases, which can provide more detailed information about the situation of crops being attacked by diseases, and are more helpful to farmers and experts. For example, the severity of citrus diseases is divided into high, medium, low and healthy, and the severity of diseased fruits is classified using the optimized VGGNet16 (a deep convolutional neural network (CNN) architecture), with an accuracy of 97%. The core of this type of method is to classify the severity of crop diseases as a classification task, and use a suitable classification model to establish the relationship between disease severity and samples. Another example: a new two-stage DeepLabv3+ architecture with adaptive loss (an advanced model architecture in the field of semantic segmentation) is used to effectively segment apple leaf disease images with complex backgrounds. The IoU (Intersection over Union) scores of leaf segmentation and spot extraction reached 98.7% and 86.56%, respectively. Another example: a new two-stage method is used to classify the severity of early blight and late blight of potato plants. Among them, Coformer (Collaborative Transformers for Grounded Situation Recognition) achieved a classification accuracy of up to 97.86%, while SegCoformer (a model used in the field of image segmentation) achieved an mIoU (mean Intersection over Union) value of 88.50%, and the combined accuracy of the method reached 84%. This type of method segments the lesion area through semantic segmentation, and then calculates the proportion of the lesion area to the total area to estimate the severity of the disease.

[0006] Although deep learning has achieved good results in the field of crop disease identification, there is still room for improvement in the recognition of data sets in natural growth environments. Most existing crop disease recognition algorithms can only identify or classify a single leaf under a single background and are not suitable for actual scenarios.

[0007] Therefore, how to simultaneously detect diseases on multiple leaves in a complex background to better meet the actual needs of planting scenarios is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0008] The present application provides a potato leaf disease detection and grading method based on deep learning, so as to simultaneously detect diseases on multiple leaves in a complex background, thereby better meeting the actual needs of planting scenarios.

[0009] In order to solve the above technical problems, this application provides the following technical solutions:

[0010] A potato leaf disease detection and grading method based on deep learning comprises the following steps: step S110, collecting pictures of potato diseased leaves in a real natural growth environment; step S120, inputting the collected pictures into an AMW-YOLOv8 network model for detection, thereby locating the position of the diseased leaves in each picture and obtaining diseased leaf pictures; step S130, inputting the diseased leaf pictures into a DCSegFormer semantic segmentation model to obtain potato disease graded images.

[0011] As described above, the potato leaf disease detection and grading method based on deep learning, wherein, preferably, the AMW-YOLOv8 network model includes: an input end, a backbone network module, a neck network module and a head network module; the input end is connected to the backbone network module, the backbone network module is connected to the neck network, and the neck network is connected to the head module; the input end processes the image input to the input end through Mosaic data enhancement, and inputs the image features obtained by the input end processing into the backbone network module; the backbone network module includes: a Conv module, a C2f module and an SPPF module; the Conv module is connected to the input end, and the Conv module is connected to the C2f module, the C2f module is connected to the SPPF module, the C2f module and the SPPF module are connected to the neck network module, and the image features are processed by the backbone network module. The Conv module, C2f module and SPPF module of the trunk network module are processed and input into the neck network module; the neck network module includes: FPN network module and PAN network module; the C2f module and SPPF module of the trunk network module are connected to the FPN network module, the SPPF module is also connected to the PAN network module, the FPN network module and the PAN network module are both connected to the head network module, and the image features are processed by the FPN network module and the PAN network module of the neck network module and input into the head network; the head network module adopts a decoupled head structure to process the image features processed by the FPN network module and the PAN network module of the neck network module, locate the position of the diseased leaves in each picture, and use the diseased leaf picture as the output of the AMW-YOLOv8 network model.

[0012] As described above, the potato leaf disease detection and grading method based on deep learning, wherein, preferably, the backbone network module includes: five Conv modules, four C2f modules and one SPPF module; the first Conv module is connected to the second Conv module, the second Conv module is connected to the first C2f module, the first C2f module is connected to the third Conv module, the third Conv module is connected to the second C2f module, the second C2f module is connected to the fourth Conv module, the fourth Conv module is connected to the third C2f module, the third C2f module is connected to the fifth Conv module, the fifth Conv module is connected to the fourth C2f module, the fourth C2f module is connected to The first Conv module is connected to the input end, and the second C2f module, the third C2f module and the SPPF module are connected to the neck network module; the FPN network module includes: two C2f-MS modules, two Concat modules and two UpSampling modules; the first UpSampling module is connected to the first Concat module, the first Concat module is connected to the first C2f-MS module, the first C2f-MS module is connected to the second UpSampling module, the second UpSampling module is connected to the second Concat module, and the second Concat module is connected to the second C2f-MS module and the first UpSampling module is connected to the SPPF module of the backbone network module, the first Concat module is connected to the third C2f module of the backbone network module, and the second Concat module is connected to the second C2f module of the backbone network module; the PAN network module includes: two CBS modules, two Concat modules and two C2f-MS modules; the first CBS module is connected to the first Concat module, the first Concat module is connected to the first C2f-MS module, the first C2f-MS module is connected to the second CBS module, the second CBS module is connected to the second Concat module, and the second Concat module is connected to the second C2 f-MS module; and the first CBS module is connected to the second C2f-MS module of the FPN network module, the first Concat module is connected to the first C2f-MS module of the FPN network module, and the second Concat module is connected to the SPPF module of the backbone network module; the head network module includes: three Detect-ASFF modules; the first Detect-ASFF module is connected to the second C2f-MS module of the FPN network module, the second Detect-ASFF module is connected to the first C2f-MS module of the PAN network module, and the third Detect-ASFF module is connected to the second C2f-MS module of the PAN network module.

[0013] In the potato leaf disease detection and grading method based on deep learning as described above, preferably, the MSBlock module replaces the Bottleneck module in the C2f module to obtain the C2f-MS module; and the ASFF module is added before the Detect module to obtain the Detect-ASFF module.

[0014] As described above, the potato leaf disease detection and grading method based on deep learning, wherein, preferably, the DCSegFormer semantic segmentation model includes: an Overlap Patch Embeddings module, multiple Our TransformerBlock modules, a CMSFFM module, an MLP Layer module and an MLP module; all Our Transformer Block modules are connected in series, the Overlap Patch Embeddings module is connected to the series-connected Our Transformer Block modules, the series-connected Our Transformer Block modules are connected to the CMSFFM modules, the CMSFFM modules are connected to the MLP Layer modules, and the MLPLayer modules are connected to the MLP modules; the Overlap Patch Embeddings module serves as the input end of the DCSegFormer semantic segmentation model, the diseased leaf image is input into the Overlap Patch Embeddings module for spot segmentation, and the MLP module serves as the output end of the DCSegFormer semantic segmentation model to output a potato disease grading image.

[0015] As described above, in the deep learning-based potato leaf disease detection and grading method, preferably, the Mix-FFN module in the Transformer Block module is replaced by a Mix-D module, and a Pos Norm module is set after the Attention module and the Mix-D module of the Transformer Block module to obtain the Our Transformer Block module.

[0016] As described above, in the potato leaf disease detection and grading method based on deep learning, preferably, the Our Transformer Block module includes: an Attention module, two Pos Norm modules, a Mix-D module and an Overlap PatchMerging module; wherein the Attention module serves as the input end of the Our Transformer Block module, the Attention module is connected to the first Pos Norm module, the first Pos Norm module is connected to the Mix-D module, the Mix-D module is connected to the second Pos Norm module, the second Pos Norm module is connected to the Overlap Patch Merging module, and the Overlap Patch Merging module serves as the output end of the Our Transformer Block module.

[0017] As described above, in the potato leaf disease detection and grading method based on deep learning, preferably, the Mix-D module includes: two linear layers, a 3x3 convolution Conv module, two 1x1 convolution Conv modules, a 3x3 depth-separable convolution module and a sigmoid activation function module; wherein the first linear layer serves as the input end of the Mix-D module, the first linear layer is connected to the 3x3 convolution Conv module and the first 1x1 convolution Conv module, the first 1x1 convolution Conv module is connected to the 3x3 depth-separable convolution module, the 3x3 depth-separable convolution module is connected to the second 1x1 convolution Conv module, and the second 1x1 convolution Conv module communicates with the sigmoid activation function module; the 3x3 convolution Conv module and the sigmoid activation function module are connected to the second linear layer, and the second linear layer serves as the output end of the Mix-D module.

[0018] As described above, the potato leaf disease detection and grading method based on deep learning, wherein, preferably, the CMSFFM module includes: four HDF Block modules, and different outputs of the Our Transformer Block modules connected in series are used as different image feature layers; the first image feature layer and the second image feature layer are used as inputs of the first HDF Block module, the second image feature layer, the third image feature layer and the output of the first HDF Block module are used as inputs of the second HDF Block module, the third image feature layer, the fourth image feature layer and the output of the second HDF Block module are used as inputs of the third HDF Block module, and the output of the fourth image feature layer and the third HDF Block module are used as inputs of the fourth HDF Block module; the output of each HDF Block module is a set of multi-scale image features, and each set of multi-scale image features is input into the MLP Layer module.

[0019] As described above, the potato leaf disease detection and grading method based on deep learning, wherein, preferably, the HDFBlock module includes: two Contact modules, two combination modules formed by a common convolution module with a convolution kernel size of 1×1, a BN module and a Gelu activation function, and a series of parallel convolutions of different scales of hole factor sizes; the first Contact module serves as the input end of the HDF Block module, the first Contact module is connected to the first combination module, the first combination module is connected to the convolution, the convolution is connected to the second Contact module, the second Contact module is connected to the second combination module, and the second combination module serves as the output end of the HDF Block module.

[0020] Compared with the above background technology, the deep learning-based potato leaf disease detection and grading method in this application combines the AMW-YOLOv8 network model and the DCSegformer semantic segmentation two-stage recognition and extraction method to determine the disease severity of potato plants. It can realize disease detection of multiple leaves in a complex background at the same time, which is more in line with the actual needs of planting scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0022] Figure 1It is a flow chart of the potato leaf disease detection and classification method based on deep learning;

[0023] Figure 2 It is a schematic diagram of the AMW-YOLOv8 network model;

[0024] Figure 3 This is a schematic diagram of the DCSegFormer semantic segmentation model;

[0025] Figure 4 It is a schematic diagram of Mix-D module;

[0026] Figure 5a This is a schematic diagram of the Transformer Block module;

[0027] Figure 5b This is a schematic diagram of the Our Transformer Block module;

[0028] Figure 6 It is a schematic diagram of the CMSFFM module;

[0029] Figure 7 This is a schematic diagram of the HDF Block module;

[0030] Figure 8 It is the PR curve of the YOLOv8n network model and the AMW-YOLOv8 network model;

[0031] Fig. 9 This is a comparison chart of the training process before and after the improvement of the Segformer network model;

[0032] Fig.10 This is a comparison chart of feature extraction effects. DETAILED DESCRIPTION

[0033] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0034] like Figure 1 As shown, the present application provides a potato leaf disease detection and classification method based on deep learning, comprising the following steps:

[0035] Step S110, collecting pictures of diseased potato leaves in a real natural growth environment;

[0036] Collect pictures of diseased potato leaves in real natural growth environments. The sources of pictures include: field photos, open data from the journal Data in Brief (which focuses on publishing descriptions of data sets in various scientific research), the PlantDoc-Dataset dataset (a dataset focused on plant disease identification) published on the scientific research database Zenodo (an open access knowledge base developed and maintained by the European Organization for Nuclear Research (CERN)), Baidu, Google, etc., to collect pictures of diseased potato leaves, and ensure that each picture contains at least one diseased leaf.

[0037] Step S120, input the collected pictures into the AMW-YOLOv8 network model for detection, so as to locate the position of the diseased leaves in each picture and obtain the diseased leaf pictures;

[0038] The AMW-YOLOv8 network model is an improved network model for the YOLOv8 network model. Figure 2 As shown, the AMW-YOLOv8 network model includes: an input end, a backbone network module, a neck network module and a head network module; the input end is connected to the backbone network module, the backbone network module is connected to the neck network, and the neck network is connected to the head module.

[0039] The input end processes the image input to the input end through Mosaic data enhancement, specifically performing random selection, splicing, and rotation operations, and inputs the image features obtained by the input end processing into the backbone network module for processing. Mosaic data enhancement is a data enhancement technology widely used in deep learning, especially in computer vision tasks (such as object detection and image classification). Its core idea is to splice four different images into a new image. During the splicing process, the content of the image will be combined according to certain rules so that the new image contains parts of the four original images.

[0040] The backbone network module includes: a Conv module, a C2f module and an SPPF module; the Conv module is connected to the input end, and the Conv module is connected to the C2f module, the C2f module is connected to the SPPF module, the C2f module and the SPPF module are connected to the neck network module, and the image features are processed by the Conv module, the C2f module and the SPPF module of the backbone network module and then input into the neck network module for processing. Optionally, the backbone network module includes: five Conv modules, four C2f modules and one SPPF module; the first Conv module is connected to the second Conv module, the second Conv module is connected to the first C2f module, the first C2f module is connected to the third Conv module, the third Conv module is connected to the second C2f module, the second C2f module is connected to the fourth Conv module, the fourth Conv module is connected to the third C2f module, the third C2f module is connected to the fifth Conv module, the fifth Conv module is connected to the fourth C2f module, and the fourth C2f module is connected to the SPPF module; and the first Conv module is connected to the input end, and the second C2f module, the third C2f module and the SPPF module are connected to the neck network module.

[0041] Among them, "Conv module" usually refers to the convolution module, which is the core component of the Convolutional Neural Network (CNN). The Conv module can change the resolution and number of channels of the image. The C2f module better understands the image by combining global semantic information and local target information; the SPPF (Spatial Pyramid Pooling-Fast) module is an efficient feature extraction module used in the field of target detection and computer vision. The SPPF module can pool features at different scales to allow the network to perform target detection on input images of different sizes.

[0042] The neck network module includes: FPN network module and PAN network module; the C2f module and SPPF module of the backbone network module are connected to the FPN network module, the SPPF module is also connected to the PAN network module, and the FPN network module and the PAN network module are both connected to the head network module. The image features are processed by the FPN network module and the PAN network module of the neck network module and then input to the head network for processing. The image features from different levels or scales are fused through the neck network module to improve the performance of the model in multi-scale scenarios, thereby enhancing the robustness and generalization ability of the model.

[0043] Among them, the FPN (Feature Pyramid Network) network module is a feature pyramid network, a neural network structure widely used in computer vision tasks such as target detection. It is mainly used to solve the problem of multi-scale target detection in target detection. The PAN (Path Aggregation Network) network module is a path aggregation network, a neural network structure for target detection. Its main purpose is to further improve the effect of feature fusion, especially when dealing with multi-scale problems in target detection, to enhance the model's detection ability for targets of different scales through more effective feature aggregation.

[0044] Optionally, the FPN network module includes: two C2f-MS modules, two Concat modules and two UpSampling modules; the first UpSampling module is connected to the first Concat module, the first Concat module is connected to the first C2f-MS module, the first C2f-MS module is connected to the second UpSampling module, the second UpSampling module is connected to the second Concat module, and the second Concat module is connected to the second C2f-MS module; and the first UpSampling module is connected to the SPPF module of the backbone network module, the first Concat module is connected to the third C2f module of the backbone network module, and the second Concat module is connected to the second C2f module of the backbone network module.

[0045] The PAN network module includes: two CBS modules, two Concat modules and two C2f-MS modules; the first CBS module is connected to the first Concat module, the first Concat module is connected to the first C2f-MS module, the first C2f-MS module is connected to the second CBS module, the second CBS module is connected to the second Concat module, and the second Concat module is connected to the second C2f-MS module; and the first CBS module is connected to the second C2f-MS module of the FPN network module, the first Concat module is connected to the first C2f-MS module of the FPN network module, and the second Concat module is connected to the SPPF module of the backbone network module.

[0046] Among them, the UpSampling module is a technique used in deep learning to increase the dimension of data (usually spatial dimension). In computer vision tasks, especially in scenarios such as image segmentation and target detection that require restoring image details or fusing features at different levels, the UpSampling module plays an important role. Its main purpose is to convert low-resolution feature maps or images into high-resolution forms, so that the network can better utilize information at different scales. The Concat module is a commonly used operation module in deep learning, which is mainly used to splice multiple tensors (usually feature maps) in a specific dimension. In the neural network architecture, especially when processing multimodal data or fusing features at different levels, the Concat module plays a key role. It can integrate information from different sources to form a richer feature representation. In deep learning models such as YOLOv8, the CBS module is a combination module that closely combines convolution, batch normalization, and SiLU activation function. This module plays a basic and key role in the network, which can effectively extract and process features and help the model better learn patterns in the data.

[0047] In this application, the MSBlock module replaces the Bottleneck module in the C2f module. The Bottleneck module is a structure widely used in deep learning architectures, especially in convolutional neural networks (CNNs). The MSBlock module is introduced into the C2f module to obtain the C2f-MS module. The C2f module is improved by using a lightweight MSBlock module, so that the C2f-MS module is more lightweight than the C2f module, thereby reducing the overall size of the model. The MSBlock (Multi-Scale Block) module is a neural network module for processing multi-scale information. The core idea of ​​the MSBlock module is to improve the ability of the network to extract features of different scales by using different convolution kernel sizes at different stages and a hierarchical feature fusion structure, thereby improving the accuracy and speed of detecting potato leaf diseases in complex environments and reducing the impact of background interference on the detection results.

[0048] The head network module adopts a decoupled head structure to process the image features processed by the FPN network module and the PAN network module of the neck network module, so as to locate the position of the diseased leaves in each image, and use the diseased leaf images as the output of the AMW-YOLOv8 network model. The decoupled head structure is a network structure design used in computer vision tasks such as target detection. Optionally, the head network module includes: three Detect-ASFF modules; the first Detect-ASFF module is connected to the second C2f-MS module of the FPN network module, the second Detect-ASFF module is connected to the first C2f-MS module of the PAN network module, and the Detect-ASFF module is connected to the second C2f-MS module of the PAN network module.

[0049] In the deep learning architecture of target detection (such as the YOLO series), "Detect" is a key module for final target detection and positioning. It receives image features processed by the previous network (such as the backbone network and the neck network), and predicts the category, location and other information of the target object based on these image features. The ASFF module is Adaptively Spatial Feature Fusion, which is mainly used to solve the consistency problem between different feature scales in target detection. The core idea of ​​the ASFF module is to adaptively adjust the weights assigned to feature layers of different scales. The adaptive spatial feature fusion method can spatially filter conflicting image features, prevent other information from interfering, retain important useful information, and enable the network to fully extract and combine useful information. Specifically, for image features of a certain level, the first step is to adjust the image features of other levels to the same size, and then find the optimal fusion weight coefficients of image features of each scale through training. Filter out interference information and adaptively fuse useful information in spatial position.

[0050] In this application, the Detect-ASFF module is obtained by adding the ASFF module before the Detect module, so that the core idea of ​​the ASFF module can be used to filter out interference information and adaptively fuse useful information, improve the problem of spatial information conflict when the original YOLOv8n network model performs multi-scale image feature fusion, and strengthen the full fusion of multi-scale image features in potato diseased leaves, thereby improving the model detection effect.

[0051] In addition, the loss function used by the AMW-YOLOv8 network model in this application is the WIoU loss function. The WIoU (Weighted-IoU) loss function is a loss function used in target detection tasks. It mainly improves the generalization performance of the model by reducing the penalty of low-quality samples due to geometric measurement factors such as aspect ratio and distance, and improves the problem that low-quality samples in the image data set affect the performance of the algorithm, thereby improving the model's ability to detect potato leaf diseases.

[0052] Step S130, input the diseased leaf image into the DCSegFormer semantic segmentation model to obtain a potato disease classification image;

[0053] In order to obtain accurate information about the severity of potato diseases from diseased leaf images, there are many diseased potato leaves that need to be segmented in the diseased leaf image dataset. The pixels of many diseased spots account for a small proportion and are not easy to be segmented. This application improves the SegFormer semantic segmentation model to improve the accuracy of segmenting diseased spots and leaves. The improved SegFormer semantic segmentation model is named DCSegFormer semantic segmentation model.

[0054] The structure of the DCSegFormer semantic segmentation model is as follows Figure 3 As shown in the figure, the DCSegFormer semantic segmentation model includes: Overlap Patch Embeddings module, multiple Our Transformer Block modules, CMSFFM module, MLPLayer module and MLP module; all Our Transformer Block modules are connected in series, the Overlap Patch Embeddings module is connected to the series-connected Our Transformer Block modules, the series-connected Our Transformer Block modules are connected to the CMSFFM module, the CMSFFM module is connected to the MLP Layer module, and the MLP Layer module is connected to the MLP module; the Overlp Patch Embeddings module serves as the input end of the DCSegFormer semantic segmentation model, and the diseased leaf image is input into the Overlap Patch Embeddings module to perform lesion segmentation. The MLP module serves as the output end of the DCSegFormer semantic segmentation model and outputs a potato disease grading image.

[0055] Among them, the Overlap Patch Embeddings module is a technology used in deep learning, especially the Visual Transformer (ViT) architecture and its variants. It is mainly used for preprocessing of image data, dividing the image into small patches with overlapping parts (patches), and converting these patches into embedding vectors (embeddings) suitable for model processing. MLP (Multilayer Perceptron) Layer is a multi-layer perceptron layer, which is a feed-forward neural network structure. MLP is the abbreviation of Multilayer Perceptron. It is a typical artificial neural network architecture that processes data through multiple neuron layers.

[0056] Our Transformer Block module is an improved module based on the original Transformer Block module. The Transformer Block module is the core component of the Transformer architecture. The Transformer architecture has achieved great success in the field of natural language processing (NLP), such as excellent performance in tasks such as machine translation and text generation, and has also been gradually applied to other fields such as computer vision. The Transformer Block module is mainly used to extract and transform the input sequence. It processes the input information through the self-attention mechanism (Self-Attention) and the feed-forward neural network (Feed-Forward Neural Network).

[0057] The original Mix-FFN module of the Transformer Block module only contains a depth-separable convolution of size 3×3 to learn the hidden layer. It is too simple and cannot extract image features of more scales. Many spatial information will inevitably be lost. In order to solve this problem, this application obtains rich multi-scale spatial information by adding a context branch of a hole convolution to the Mix-FFN module to more accurately distinguish pixel-level details, thereby helping the network to better understand the semantic information of the input image features. Among them, the Mix-FFN (Mix Feed-Forward Network) module is a feedforward network structure that appears in Transformer architecture-related technologies. It is an improvement on the traditional feedforward neural network (FFN) and aims to enhance the expressiveness and feature extraction capabilities of the model, especially when processing sequence data (such as sentences in natural language processing, image block sequences in computer vision, etc.).

[0058] The Mix-FFN module in the Transformer Block module is replaced by the Mix-D module to obtain the OurTransformer Block module. The structure of the Mix-D module is as follows: Figure 4 As shown, the Mix-D module includes: two linear layers (MLP modules), a 3x3 convolution Conv module, two 1x1 convolution Conv modules, a 3x3 depth-separable convolution module and a sigmoid activation function module; wherein, the first linear layer serves as the input end of the Mix-D module, the first linear layer is connected to the 3x3 convolution Conv module and the first 1x1 convolution Conv module, the first 1x1 convolution Conv module is connected to the 3x3 depth-separable convolution module, the 3x3 depth-separable convolution module is connected to the second 1x1 convolution Conv module, and the second 1x1 convolution Conv module communicates with the sigmoid activation function module; the 3x3 convolution Conv module and the sigmoid activation function module are connected to the second linear layer, and the second linear layer serves as the output end of the Mix-D module.

[0059] Among them, the linear layer is usually called a fully connected layer or a dense layer in a neural network. It is one of the most basic layers in a neural network. Its function is to perform a linear transformation on the input data, mainly including matrix multiplication and adding a bias. The sigmoid activation function is a common activation function. Depthwise Separable Convolution is a convolution operation used in a convolutional neural network (CNN) to reduce the amount of computation while maintaining performance.

[0060] For image features of size B, N, C. First, the image feature X in After a linear layer, the hidden layer channels are expanded to three times the original number, and the image feature F with the number of channels expanded to three times the original number is obtained (Formula (3-1)). Then, the image feature F will pass through a trunk branch and a context branch respectively.

[0061] In the context branch, the image feature F first passes through a 1x1 convolution Conv, which can appropriately compress the 3C channel to avoid excessive parameters. Then, it passes through a depthwise separable convolution DW (Depthwise) with a void coefficient of 2 and a convolution kernel of 3x3 to learn information under different large-scale receptive fields. Subsequently, the number of channels is mapped back to 3C through a 1x1 convolution Conv. This operation can eliminate redundant features and extract valid image features. Finally, it passes through a sigmoid activation function to generate the spatial information guidance weight W for each channel (Formula (3-2)).

[0062] In the main branch, the image feature F passes through a 3x3 convolution Conv to obtain the image feature F1 (Formula (3-3)). At this time, the information-guided weight W obtained through the context branch is multiplied by the image feature F1 and then multiplied by the image feature X. in The image feature F' is obtained by adding and fusion (Formula (3-4)), and then the image feature F' is activated by the Gelu activation function, and finally output through the linear layer to obtain the output feature F out (Formula (3-5)), where Act is GeLU activation. GeLU (Gaussian Error Linear Units) is an activation function used to introduce nonlinear factors in neural networks.

[0063] The above process is shown in formulas (3-1) to (3-5):

[0064] F = MLP(X in ) (3-1)

[0065] W = Sigmoid(Conv 1×1 (DW(Conv 1×1 (F))))(3-2)

[0066] F1=Conv 3×3 (F) (3-3)

[0067] F′=F1×W+X in (3-4)

[0068] F out =MLP(Act(F')) (3-5)

[0069] In addition, the present application also improves the position of the Pos Norm module in the original Transformer Block module. The Transformer Block model originally adopted the method of performing Pos Norm before the Attention module and the Mix-FFN module. The structure of the original Transformer Block module is as follows: Figure 5aAs shown. In the Transformer architecture, the position-dependent feedforward network (Pos Norm, Position-Wise Feed-Forward Network) is an important component. Attention is a mechanism in deep learning. Its essence is to enable the model to focus on certain specific parts of the input data. However, in this way, the image features will be normalized when they are input, and the context branches are not standardized before being merged, resulting in the activation values ​​on the final context branches being accumulated on the trunk branches, which will affect the significant increase in activation values ​​observed in the deep layers, making the amplitude of the trunk branches larger and larger as the network deepens, and the amplitude differences at different levels are large, resulting in unstable training. This application adopts the method of performing normalization operations after the Attention module and the Mix-D module, and then adding residuals. The structure diagram of the improved Our Transformer Block module is shown in the figure. Figure 5b As shown in Figure 3, the complete formula flow of the Mix-D module and the Pos Norm module for processing the feature image FFN is shown in formula (3-6). In this way, the output of each residual block is normalized before being merged back into the main branch. As the network becomes deeper, the amplitude of the main branch will not accumulate.

[0070] FFN(X in )=X in +Norm(F out ) (3-6)

[0071] Among them, Norm (Normalization), in the field of machine learning and deep learning, usually refers to the normalization operation. Normalization is mainly used to convert the characteristics of data to a specific range, making the data comparable in different dimensions and helping to optimize the performance of the algorithm.

[0072] The structure of the Our Transformer Block module in this application is as follows Figure 5bAs shown in the figure, Our TransformerBlock module includes: an Attention module (Efficient Self-Attention), two Pos Norm modules, a Mix-D module and an Overlap PatchMerging module; wherein, the Attention module serves as the input end of the OurTransformer Block module, the Attention module is connected to the first Pos Norm module, the first Pos Norm module is connected to the Mix-D module, the Mix-D module is connected to the second Pos Norm module, the second Pos Norm module is connected to the Overlap PatchMerging module, and the Overlap PatchMerging module serves as the output end of the OurTransformerBlock module.

[0073] Among them, Overlap PatchMerging is an operation technique used in deep learning architectures to process images or sequence data. It mainly involves dividing the input data into overlapping patches and then merging these patches.

[0074] The decoder design of the Segformer model is relatively simple. It predicts segmentation by directly splicing features at each level. However, the resolution and semantic information of different feature levels vary greatly. Direct splicing will cause interference between features at each level, which cannot be better integrated and learned. It will also ignore many low-level features that are important for segmentation details, making many lesions undetectable. Therefore, this application designs a cascaded multi-scale feature fusion module and names it the CMSFFM module, which strengthens the multi-scale fusion learning of each feature with adjacent features, thereby making full use of the extracted shallow and deep features of potato diseases.

[0075] The core idea of ​​the cascaded multi-scale feature fusion module (CMSFFM module) is to use a progressive fusion method to perform multi-scale fusion learning on the feature information of each feature layer and its adjacent feature layer. The structure diagram of the CMSFFM module is shown in the figure. Figure 6As shown in the figure, the CMSFFM module includes: four HDF Block modules, and different outputs of the series-connected Our Transformer Block modules are used as different image feature layers (for example, image feature layers C1, C2, C3, and C4); the first image feature layer and the second image feature layer are used as inputs of the first HDF Block module, the second image feature layer, the third image feature layer, and the output of the first HDF Block module are used as inputs of the second HDF Block module, the third image feature layer, the fourth image feature layer, and the output of the second HDF Block module are used as inputs of the third HDF Block module, and the output of the fourth image feature layer and the third HDF Block module are used as inputs of the fourth HDF Block module; the output of each HDF Block module is a set of multi-scale image features (for example, multi-scale image features P1, P2, P3, and P4), and each set of multi-scale image features is input into the MLP Layer module.

[0076] The CMSFFM module can simultaneously output image features of different scales, and the semantic gap between non-adjacent image feature layers is large, especially the image feature layers C1 and C4. If the feature information between non-adjacent layers is directly fused, the effect will be poor. Therefore, this application adopts a progressive fusion method, and the image feature layers C1, C2, C3, and C4 are fused to form a set of multi-scale image features P1, P2, P3, and P4.

[0077] Taking the generation process of multi-scale feature P3 as an example, C1 and C2 are first fused, and then the fusion result is combined with C2 and C3 to obtain multi-scale feature P2. Since C1 and C2, C2 and C3 are adjacent layers, the above fusion method can narrow the semantic gap between C1 and C3, making the feature information of different fused layers closer, and avoiding poor fusion effect due to excessive semantic gap between non-adjacent layers.

[0078] The structure of the multi-scale hybrid hole fusion module (HDF Block module) designed by this application to fuse image features of different image feature layers is as follows Figure 7As shown, it includes: two Contact modules, two combination modules formed by a common convolution module with a convolution kernel size of 1×1, a BN module and a Gelu activation function, and a series of (different sizes) parallel convolutions of different scales of hole factor sizes; the first Contact module is used as the input end of the HDF Block module, the first Contact module is connected to the first combination module, the first combination module is connected to the above convolution, the convolution is connected to the second Contact module, the second Contact module is connected to the second combination module, and the second combination module is used as the output end of the HDF Block module. Among them, BN (Batch Normalization) is a technology for normalizing the input data of each layer in a neural network.

[0079] First, the image features of different levels entering the HDF Block module are spliced ​​to fully integrate the image features of the upper and lower levels. Then, a combination module consisting of a common convolution module with a convolution kernel size of 1×1, a BN module, and a Gelu activation function module is used. Since the image features of different levels have different scales, they need to be learned at different scales. Therefore, the fused image features need to be learned in a series of (different sizes) parallel convolutions of different scales of dilation factor sizes. The continuity of the receptive field is guaranteed by adopting mixed dilated convolution, the feature information contained in the image features is increased, and a set of dilated convolutions with a constraint relationship between the expansion rate values ​​of no more than 1 are selected, which effectively avoids the influence of the dilated convolution chess effect. Finally, the learned image features of each group of different fine-grained sizes are spliced, and finally output through a combination module consisting of a 1×1 common convolution module, a BN module, and a Gelu activation function module to restore the number of channels at this level.

[0080] By introducing the cascade feature fusion module proposed in this application into Segformer, the image features at each level can be fully integrated. The introduction of mixed dilated convolution enables the network to obtain more image features of different scales, while increasing the receptive field, retaining a lot of detail information, improving the model's ability to recognize leaves and spots, and enhancing the model's ability to handle segmentation tasks of different scales.

[0081] 1. The following is a potato disease detection experiment based on the AMW-YOLOv8 network model

[0082] This application mainly collects diseased potato leaves in real natural growth environments. The first stage of the algorithm image sources include field photography, open data from the journal Data in Brief, the scientific research database Zenodo, the PlantDoc-Dataset dataset released on github, Baidu, Google, etc. to collect potato disease pictures, and ensure that each picture contains at least one diseased leaf. After screening, pictures that meet the requirements were obtained, including 405 pictures of potato early blight, 458 pictures of potato late blight, 479 pictures of potato virus disease, and 437 pictures of potato wilt. The total number of images of each type was brought to nearly 1,300 through data enhancement methods. This paper uses precision, recall, mean average precision (mAP), etc. to evaluate detection accuracy; parameter quantity evaluates model complexity.

[0083] This application conducted an ablation experiment based on the YOLOv8n network model. By comparing the performance of the YOLOv8n network model after adding different improved modules, the contribution of each added module to the overall performance of the model can be more accurately evaluated. The experimental results are shown in Table 1.

[0084] Table 1 Ablation experiment of YOLOv8 improved method

[0085]

[0086] According to the results of the ablation experiment, the mAP@0.5 value of the basic network model YOLOv8n is 93.3%. First, based on the YOLOv8n model, the detection head improved by ASFF is introduced. By strengthening the fusion of multi-scale features, the mAP@0.5 value of the model is increased by 1.4%, and the number of parameters of the model increases by 1.373×106. Then, on the basis of improving YOLOv8n with the ASFF detection head, the mAP@0.5 value of the model is further increased by 0.4% by combining the lightweight MSBlock module in the C2f of the neck network, and the number of parameters is reduced. Finally, the loss function is optimized by WIoU. The mAP@0.5 value of the overall improved AMW-YOLOv8n model is 2% higher than that of the basic YOLOv8n model, and the number of model parameters increases from 3.006×106 to 4.178×106, an increase of 1.172×106, but a small increase in computational complexity has achieved a significant improvement in model recognition effect.

[0087] like Figure 8 PR curves of the YOLOv8n network model and the AMW-YOLOv8 network model. Figure 8From the PR curve in Figure 2, we can see that compared with the original YOLOv8n network model, the improved model has significantly improved the detection accuracy of potato early blight, potato late blight, potato virus disease and potato wilt. Among them, the detection of potato wilt has been improved the most, with the mAP@0.5 value increased by 3.9%. The improved AMW-YOLOv8 network model improves the overall performance of the model and also provides an efficient solution for potato disease detection tasks.

[0088] 2. Potato leaf disease classification experiment based on DCSegformer network model

[0089] This application studies potato early blight and potato late blight leaves with lesions among potato leaf diseases. First, the potato early blight and potato late blight leaf datasets in the complex environment used above are processed, mainly by extracting multiple diseased leaves contained in each picture by cropping, so that each image contains only a single early blight or late blight leaf, thereby simulating the situation where the diseased leaves have been detected in the previous chapter. After data enhancement, a dataset containing 2026 samples was formed. For the performance of the model, this application uses mean intersection over union (MIOU), pixel accuracy (PA), Dice coefficient, precision, recall rate, F1 value, parameter quantity and FLOPs for evaluation.

[0090] (1) Improved Transformer Block Analysis

[0091] In order to verify the effectiveness of the improved Transformer Block on the model performance, the following comparative experiments were conducted before and after the improvement, and the improved model was named DSegformer. The experimental data in Table 2 show that the improved Transformer Block model is 90.53%, 96.49%, 90.80%, 92.51%, 94.17%, and 92.87% in terms of MIoU, PA, Dice, Precion, Reall, and F1, respectively. Compared with the original Segformer model, MIoU increased by 1.1%, PA increased by 2.3%, Dice increased by 1%, and F1 increased by 0.8%, with obvious improvement effects. It is proved that the improved Transformer Block proposed by adding the context branch of the dilated convolution can obtain rich multi-scale spatial information, thereby more accurately distinguishing pixel-level details.

[0092] Table 2 Comparison before and after improving Transformer Block

[0093]

[0094] (2) Introducing CMSFFM analysis

[0095] Since the superposition of dilated convolutions with different dilation rates can improve the checkerboard effect to a certain extent, this experiment selected dilated convolutions with different dilation rates for testing based on the improved Transformer Block to find the best combination. Experiments 1 and 2 selected mixed dilated convolution modules with dilation rate combinations of (1,2,3,1,2,3) and (1,2,5,1,2,5) respectively. Experiments 3, 4, and 5 took into account the different sizes of input feature maps and designed mixed dilated convolution modules with different combinations for feature maps of different sizes. Specifically, the feature map size is limited to 320. For feature maps with a size less than 320, Experiments 3, 4, and 5 respectively select mixed dilation rate combinations of (2, 3, 3, 2, 3, 6), (1, 2, 3, 1, 2, 3), and (1, 2, 4, 1, 2, 4) for testing. For feature maps with a size greater than or equal to 320, Experiments 3, 4, and 5 respectively select mixed dilation rate combinations of (2, 3, 4, 2, 3, 6), (1, 2, 5, 1, 2, 5), and (1, 2, 3, 1, 2, 3) for testing.

[0096] Table 3 Comparison of mixed dilated convolution combinations with different dilation rates

[0097]

[0098] It can be concluded from Table 3 that in the experiment of the mixed dilated convolution module with dilated convolutions of different dilated rates, different mixed dilated convolutions are designed according to the size of the feature map, and the mixed dilated convolution module with dilated rates of (1,2,4,1,2,4) / (1,2,3,1,2,3) has the highest segmentation accuracy, with MIoU value reaching 91.01%, PA value reaching 96.65%, Dice value reaching 91.30%, and F1 value reaching 93.78%. The four evaluation indicators are the best values ​​in the five groups of experiments. Therefore, this paper selects the dilated convolution with dilated rates of (1,2,4,1,2,4) / (1,2,3,1,2,3) to form the HDF module.

[0099] In order to verify the effectiveness of the cascaded multi-scale feature fusion module proposed in this application, a comparative experiment was conducted by introducing only the hybrid void convolution module with the best void rate combination on the basis of the original Segformer, and the improved model was named CSegformer. The experimental results are shown in Table 4. From the data in Table 4, it can be seen that compared with the original Segformer model, the improved CSegformer model has an MIoU improvement of 1.1%, a PA improvement of 2.3%, a Dice improvement of 1%, and a F1 improvement of 0.8%. The parallel multi-scale hybrid void fusion module proposed in this paper can fully integrate the features of each level, preserve richer potato disease information, and thus improve the segmentation accuracy.

[0100] Table 4 Comparison before and after the introduction of the proposed CMSFFM module

[0101]

[0102] (3) Ablation experiment analysis

[0103] The segmentation model of potato leaves and lesions was trained under the same experimental conditions, and the ablation experiment was used to evaluate the performance of the Segformer potato disease leaf segmentation model. The ablation experiment results are shown in Table 5. The experiment uses the Segformer algorithm as the baseline model. First, the proposed cascade multi-scale void fusion module is introduced into the Segformer algorithm. The MIoU value of the model is improved by 1.11%, the Dice value is improved by 1.02%, the model parameters are increased from 27.35M to 28.54M, and the Flops are also increased by 0.72G. Then, on the basis of adding the cascaded multi-scale void fusion module, the proposed Our Transformer Block module was introduced by improving the Mix-FFN module and the position of LayerNorm in the model. The MIoU value of the model increased by 3.75% compared with the original Segformer, the Dice value increased by 3.18%, the model parameters increased from 27.35M to 31.24M, and the Flops also increased by 1.52G. However, the increase in a small amount of computational complexity can significantly improve the segmentation accuracy of the model. The two innovative improvement methods proposed enable the model to fully segment the features of the different sizes of lesions on potato leaves, thereby improving the segmentation accuracy.

[0104] Table 5 Ablation experiment of Segformer improved method

[0105]

[0106] Fig. 9 This is a comparison between the potato leaf and lesion segmentation method based on the improved Segformer proposed in this application and the training process before improvement. Fig. 9 The loss changes of the model on the training set and the test set are shown respectively. Fig. 9 It can be seen that the loss of the DCSegformer model on the training set and test set decreases faster, and tends to be smooth earlier than the original Segformer model, and the overall change process is also smoother.

[0107] (4) Visualization of heat map results

[0108] In order to more clearly demonstrate the effect of the improved model on the feature enhancement of diseased potato leaves and leaf contours, images were randomly selected from the data set and compared with the heat maps generated by the original Segformer model and the improved DCSegformer model, respectively, to fully verify the effectiveness of the improved Segformer model in extracting leaf and diseased features. Fig.10 The feature extraction effect is shown in the figure.

[0109] (5) Algorithm performance analysis

[0110] To verify the advantages of the improved model, this chapter conducts comparative experiments on the improved model DCSegformer and other semantic segmentation algorithms for segmenting leaves and lesions. The experimental results are shown in Table 6.

[0111] Table 6 Performance comparison of different algorithms

[0112]

[0113] It can be seen from the experimental data in Table 6 that compared with other semantic segmentation algorithms commonly used to segment leaves and spots, the improved DCSegformer algorithm has the highest MIoU value and the lowest Params and Flops values. It can achieve the highest segmentation accuracy while minimizing the amount of calculation, and the overall performance is more advantageous. The algorithm improvement effect is obvious.

[0114] 3. Disease classification results

[0115] At present, the severity of the disease on the leaves of plants is usually determined by calculating the relative area ratio. Specifically, the ratio of the area of ​​the lesions on the isolated potato leaves to the area of ​​the entire leaf is calculated. The ratio of the number of pixels of the lesions to the number of pixels of the leaves can be expressed according to formula (4-1):

[0116]

[0117] Among them, A d is the area of ​​the lesion, A lis the normal area of ​​the leaf, M(i, j) = 1 represents the pixel with a value of 1 in the image, ∑ij represents the sum of all pixel positions (i, j) in the image, and K represents the relative area ratio of the diseased spot to the entire leaf.

[0118] According to the standard reference table of potato early blight and late blight, the severity of potato diseases was assessed, as shown in Table 7, where K e Represents the standard reference for early blight, K l Represents the standard reference for late blight.

[0119] Table 7 Classification Standards

[0120] Disease level Disease severity <![CDATA[Ratio of early blight area of potato (K e / % <![CDATA[Ratio of the area of potato late blight (K l / %)]]> 0 normal <![CDATA[K e =0]]> <![CDATA[K l =0]]> 1 Relatively low <![CDATA[0<K e ≤5]]> <![CDATA[0<K l ≤5]]> 3 Relatively low <![CDATA[5<K e ≤10]]> <![CDATA[5<K l ≤10]]> 5 medium <![CDATA[10<K e ≤20]]> <![CDATA[10<K l ≤20]]> 7 Relatively high <![CDATA[20<K e ≤50]]> <![CDATA[20<K l ≤50]]> 9 high <![CDATA[K e ≥50]]> <![CDATA[K l ≥50]]>

[0121] In order to evaluate the effectiveness of the DCSegformer model in the task of grading the severity of actual potato diseases, this paper randomly selected 200 potato disease images from the dataset as grading experimental samples, calculated the ratio of the lesion area to the total leaf area through the above method, and graded the diseased leaves according to the disease grading standards in Table 7. Subsequently, the prediction results of these samples were compared with the actual labels. The disease grading evaluation results are shown in Table 8.

[0122] Table 8 Disease classification evaluation results

[0123]

[0124] From the data in Table 8, we can see that among the 200 samples, 190 samples had correct disease classification, and 10 samples had incorrect disease classification. The main factor leading to sample evaluation errors was under-segmentation and over-segmentation in the process of leaf and lesion segmentation. The overall average accuracy of the classification evaluation results can reach 95%, which can be applied in actual scenarios where disease classification is required.

[0125] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0126] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A potato leaf disease detection and classification method based on deep learning, characterized in that: The steps include: Step S110, collecting pictures of diseased potato leaves in a real natural growth environment; Step S120, input the collected pictures into the AMW-YOLOv8 network model for detection, so as to locate the position of the diseased leaves in each picture and obtain the diseased leaf pictures; Step S130: input the diseased leaf image into the DCSegFormer semantic segmentation model to obtain a potato disease classification image.

2. The potato leaf disease detection and classification method based on deep learning according to claim 1 is characterized in that: The AMW-YOLOv8 network model includes: an input end, a backbone network module, a neck network module and a head network module; the input end is connected to the backbone network module, the backbone network module is connected to the neck network, and the neck network is connected to the head module; The input end processes the image input to the input end through Mosaic data enhancement, and inputs the image features obtained by the input end processing into the backbone network module; The backbone network module includes: a Conv module, a C2f module and an SPPF module; the Conv module is connected to the input end, and the Conv module is connected to the C2f module, the C2f module is connected to the SPPF module, the C2f module and the SPPF module are connected to the neck network module, and the image features are processed by the Conv module, the C2f module and the SPPF module of the backbone network module and then input into the neck network module; The neck network module includes: an FPN network module and a PAN network module; the C2f module and the SPPF module of the backbone network module are connected to the FPN network module, the SPPF module is also connected to the PAN network module, the FPN network module and the PAN network module are both connected to the head network module, and the image features are processed by the FPN network module and the PAN network module of the neck network module and then input into the head network module; The head network module adopts a decoupled head structure to process the image features processed by the FPN network module and the PAN network module of the neck network module, locate the position of the diseased leaves in each image, and use the diseased leaf images as the output of the AMW-YOLOv8 network model.

3. The potato leaf disease detection and classification method based on deep learning according to claim 2 is characterized in that: The backbone network module includes: five Conv modules, four C2f modules and one SPPF module; the first Conv module is connected to the second Conv module, the second Conv module is connected to the first C2f module, the first C2f module is connected to the third Conv module, the third Conv module is connected to the second C2f module, the second C2f module is connected to the fourth Conv module, the fourth Conv module is connected to the third C2f module, the third C2f module is connected to the fifth Conv module, the fifth Conv module is connected to the fourth C2f module, and the fourth C2f module is connected to the SPPF module; and the first Conv module is connected to the input end, and the second C2f module, the third C2f module and the SPPF module are connected to the neck network module; The FPN network module includes: two C2f-MS modules, two Concat modules and two UpSampling modules; the first UpSampl ing module is connected to the first Concat module, the first Concat module is connected to the first C2f-MS module, the first C2f-MS module is connected to the second UpSampling module, the second UpSampl ing module is connected to the second Concat module, and the second Concat module is connected to the second C2f-MS module; and the first UpSampling module is connected to the SPPF module of the backbone network module, the first Concat module is connected to the third C2f module of the backbone network module, and the second Concat module is connected to the second C2f module of the backbone network module; The PAN network module includes: two CBS modules, two Concat modules and two C2f-MS modules; the first CBS module is connected to the first Concat module, the first Concat module is connected to the first C2f-MS module, the first C2f-MS module is connected to the second CBS module, the second CBS module is connected to the second Concat module, and the second Concat module is connected to the second C2f-MS module; and the first CBS module is connected to the second C2f-MS module of the FPN network module, the first Concat module is connected to the first C2f-MS module of the FPN network module, and the second Concat module is connected to the SPPF module of the backbone network module; The head network module includes: three Detect-ASFF modules; the first Detect-ASFF module is connected to the second C2f-MS module of the FPN network module, the second Detect-ASFF module is connected to the first C2f-MS module of the PAN network module, and the third Detect-ASFF module is connected to the second C2f-MS module of the PAN network module.

4. The potato leaf disease detection and classification method based on deep learning according to claim 3 is characterized in that: Replace the Bottleneck module in the C2f module with the MSBlock module to obtain the C2f-MS module; Add the ASFF module before the Detect module to get the Detect-ASFF module.

5. The potato leaf disease detection and classification method based on deep learning according to any one of claims 1 to 4, characterized in that: The DCSegFormer semantic segmentation model includes: Overlap Patch Embeddings module, multiple OurTransformer Block modules, CMSFFM module, MLP Layer module and MLP module; All Our Transformer Block modules are connected in series, the Overlap Patch Embeddings module is connected to the series-connected Our Transformer Block module, the series-connected Our Transformer Block module is connected to the CMSFFM module, the CMSFFM module is connected to the MLP Layer module, and the MLP Layer module is connected to the MLP module; The Overlap Patch Embeddings module serves as the input of the DCSegFormer semantic segmentation model, and the diseased leaf images are input into the Overlap Patch Embeddings module for lesion segmentation. The MLP module serves as the output of the DCSegFormer semantic segmentation model to output potato disease classification images.

6. The potato leaf disease detection and classification method based on deep learning according to claim 5, characterized in that: The Mix-FFN module in the Transformer Block module is replaced with a Mix-D module, and a Pos Norm module is set after the Attention module and the Mix-D module of the Transformer Block module to obtain the Our Transformer Block module.

7. The potato leaf disease detection and classification method based on deep learning according to claim 6, characterized in that: Our Transformer Block module includes: an Attention module, two Pos Norm modules, a Mix-D module and an Overlap PatchMerging module; Among them, the Attention module serves as the input end of the Our Transformer Block module, the Attention module is connected to the first Pos Norm module, the first Pos Norm module is connected to the Mix-D module, the Mix-D module is connected to the second PosNorm module, the second Pos Norm module is connected to the Overlap PatchMerging module, and the OverlapPatchMerging module serves as the output end of the Our Transformer Block module.

8. The potato leaf disease detection and classification method based on deep learning according to claim 6, characterized in that: The Mix-D module includes: two linear layers, a 3x3 convolution Conv module, two 1x1 convolution Conv modules, a 3x3 depth-separable convolution module and a sigmoid activation function module; wherein, the first linear layer serves as the input end of the Mix-D module, the first linear layer is connected to the 3x3 convolution Conv module and the first 1x1 convolution Conv module, the first 1x1 convolution Conv module is connected to the 3x3 depth-separable convolution module, the 3x3 depth-separable convolution module is connected to the second 1x1 convolution Conv module, and the second 1x1 convolution Conv module communicates with the sigmoid activation function module; the 3x3 convolution Conv module and the sigmoid activation function module are connected to the second linear layer, and the second linear layer serves as the output end of the Mix-D module.

9. The potato leaf disease detection and classification method based on deep learning according to claim 6, characterized in that: The CMSFFM module includes: four HDF Block modules, and the different outputs of the series-connected Our Transformer Block modules are used as different image feature layers; the first image feature layer and the second image feature layer are used as the input of the first HDF Block module, the second image feature layer, the third image feature layer and the output of the first HDF Block module are used as the input of the second HDF Block module, the third image feature layer, the fourth image feature layer and the output of the second HDF Block module are used as the input of the third HDF Block module, and the output of the fourth image feature layer and the third HDF Block module are used as the input of the fourth HDF Block module; the output of each HDF Block module is a set of multi-scale image features, and each set of multi-scale image features is input into the MLP Layer module.

10. The potato leaf disease detection and classification method based on deep learning according to claim 9, characterized in that: The HDF Block module includes: two Contact modules, two combination modules formed by a common convolution module with a convolution kernel size of 1×1, a BN module and a Gelu activation function, and a series of parallel convolutions with different scales of void factor sizes; the first Contact module serves as the input end of the HDF Block module, the first Contact module is connected to the first combination module, the first combination module is connected to the convolution, the convolution is connected to the second Contact module, the second Contact module is connected to the second combination module, and the second combination module serves as the output end of the HDF Block module.

Images