A leaf disease classification method based on YOLOv8 and a multi-task learning model

Abstract

This invention relates to the field of agricultural diseases, specifically providing a leaf disease classification method based on YOLOv8 and a multi-task learning model, comprising the following steps: S0, identifying diseased regions in the original image of diseased leaves based on the YOLOv8 model, and outputting a cropped image of the diseased leaf region; S1, constructing a segmentation-assisted classification multi-task learning model; S2, performing pixel-by-pixel segmentation of the detected leaves using the multi-task learning model, and classifying the segmented regions by disease type; S3, designing a custom total loss function combining segmentation loss, classification loss, and consistency loss, and training the multi-task learning model. This technical solution solves the problem of poor classification performance in agricultural plant disease detection, and constructs a highly efficient and accurate disease classification and lesion segmentation system.

Description

Technical Field

[0001] This invention relates to the field of agricultural diseases, specifically providing a leaf disease classification method based on YOLOv8 and a multi-task learning model. Background Technology

[0002] Rapid identification of crop diseases is crucial in modern agriculture, especially against the backdrop of climate change, rapid industrialization leading to arable land loss, and the global spread of pests (The global spread of crop pests and pathogens. Global Ecology and Biogeography, 2014; Forecasting plant and crop disease: An explorative study on current algorithms, 2021). Early identification of crop diseases can effectively reduce losses, increase crop yields, and thus ensure global food security. However, developing such an identification system faces numerous challenges. The agricultural environment itself is complex and variable; factors such as changing light, shading, and complex backgrounds all contribute to difficulties in image recognition. Furthermore, the visual differences between plant species and diseases are often very subtle, and the sheer number of disease types and plant species presents a significant challenge to accurate classification.

[0003] The widespread application of deep learning technology has had a profound impact on agriculture, especially in the field of plant disease detection. In the past, plant disease detection mainly relied on traditional machine learning (ML) techniques, particularly feature extraction methods (such as K-means clustering, gray-level co-occurrence matrix, etc.) and prediction methods based on classifiers (such as support vector machines, K-nearest neighbors, random forests, etc.) (Deep learning for plant identification and disease classification from leafimages: multi-prediction approaches, 2024). Although these methods can achieve a certain level of accuracy in some scenarios, their main problem is over-reliance on manual feature engineering and a tendency to overfit when data is insufficient, resulting in poor model generalization ability (Deepcrop: Deep learning-based crop diseaseprediction with web application, 2023).

[0004] With the rise of deep learning, especially breakthroughs in convolutional neural networks (CNNs), plant disease detection has entered a new stage. CNNs can automatically learn effective features from images, avoiding the tedious manual feature extraction steps required in traditional methods. The application of CNNs in plant pathology, particularly in image classification and detection tasks, has shown great potential. Many classic CNN architectures, such as VGG, ResNet, EfficientNet, DenseNet, Inception, and AlexNet, have been widely used in plant disease detection and have achieved excellent results (Plant diseasedetection and classification techniques: a comparative study of the performances, 2024). However, CNNs also have some limitations, especially when dealing with complex backgrounds, occlusions, or environmental changes (such as lighting and weather), where their performance is often affected. Therefore, overcoming these problems and improving robustness remains a research hotspot in this field.

[0005] To improve performance, more and more researchers are exploring new architectures, such as Vision Transformers (ViT) and Vision Mamba (ViM). ViT captures long-range dependencies in images through a self-attention mechanism, achieving excellent classification results (An image is worth 16x16 words: Transformers for image recognition at scale, 2020). However, due to the high computational complexity of the self-attention mechanism, ViT still faces challenges when processing large-scale data. To address this, He et al. proposed the Vision Mamba (ViM) architecture, which achieves linear scaling by optimizing the computational complexity of the self-attention mechanism, successfully solving the computational bottleneck of ViT when processing large-scale data (Surface vision mamba: Leveraging bidirectional state space model for efficient spherical manifold representation, 2025).

[0006] In this task, leaf images are often central to the analysis because leaves not only reflect the characteristics of the plant species but also exhibit typical symptoms of diseases. Traditionally, leaf disease classification and plant species identification have been considered independent tasks (Vmamba for plant leaf disease identification: design and experiment, 2025). However, with the development of deep learning technology, researchers have found that processing these tasks in an integrated and interdependent manner can bring several benefits: such as better interpretability, more efficient resource utilization, stronger knowledge sharing, and improved overall performance (Trends in machine and deep learning techniques for plant disease identification: A systematic review, 2024). This provides a theoretical basis for multi-task learning (MTL), especially in complex tasks such as plant disease detection, where MTL can enhance the synergistic effect between tasks through shared knowledge representations.

[0007] Researchers have proposed various MTL frameworks, such as Conditional Multitask Learning (CMTL), which effectively improves classification accuracy by making the learning of one task (such as disease classification) dependent on another task (such as species identification) (Conditional multitask learning for plant disease identification, 2021). Furthermore, other researchers, such as An et al., have achieved state-of-the-art performance on multiple tasks through multi-model ensembles using weights pre-trained on ImageNet (Hierarchical deep learning models using transfer learning for disease detection and classification based on small number of medical images, 2021).

[0008] In practical applications of plant disease detection, many researchers have explored combined methods that integrate segmentation and classification. For example, Ferentinos et al. proposed a method that uses the YOLO architecture to detect lesion regions and combines it with ResNet to classify specific diseases, significantly improving classification accuracy (Deep learning models for plant disease detection and diagnosis, 2019). Furthermore, Mzoughi and Yahiaoui proposed a segmentation-assisted method that improves classification performance by accurately identifying lesion regions, especially in situations with high background noise (Deep learning-based segmentation for disease identification, 2023).

[0009] However, despite its widespread application in many fields, MTL still faces several challenges. Existing MTL architectures often fail to effectively utilize shared generalizable knowledge, leading to negative transfer problems in the models. This makes these models more difficult to debug and optimize in practice (Mitigating negative transfer in multi-task learning with exponential moving average loss weighting strategies (student abstract), 2023). Furthermore, the scale of large models and the strong coupling between the backbone network and branch modules also reduce the modularity of MTL, limiting its flexibility and scalability.

[0010] To address these issues, this patent designs an architecture based on YOLO V8 and Vision Mamba Multi-Task Learning (VMTL). The Vision Mamba backbone is an efficient state-space model capable of capturing long-range dependencies with linear complexity, fully considering spatial details and global context. This architecture integrates semantic segmentation with disease classification. Through segmentation-assisted classification design, it not only achieves disease classification but also optimizes segmentation output, thereby improving the accuracy of spatial disease localization and ultimately helping to better identify and locate disease symptoms. Extensive experiments on the PlantDoc dataset demonstrate that VMTL exhibits state-of-the-art performance with a compact model size and real-time inference speed suitable for practical deployment. This is precisely the technological breakthrough of this patent: by combining segmentation with disease classification tasks and employing segmentation-assisted classification design, it not only improves the accuracy of disease classification but also optimizes the precision of segmentation results. This innovative method effectively improves the accuracy of spatial disease localization while ensuring real-time inference speed through efficient architectural design, filling a key gap in current industrial and technological applications. Summary of the Invention

[0011] To overcome the above-mentioned shortcomings, this invention is proposed. This invention provides a leaf disease classification method based on YOLOv8 and a multi-task learning model, comprising the following steps: S0. Identify the diseased areas in the original image of diseased leaves based on the YOLOv8 model, and output the cropped image of the diseased leaf area. S1. Construct a multi-task learning model for segmentation-assisted classification; S2. The detected leaves are segmented pixel by pixel using a multi-task learning model, and the segmented areas are classified according to disease type. S3. Design a custom total loss function by combining segmentation loss, classification loss and consistency loss, and train a multi-task learning model.

[0012] Furthermore, step S0 includes the following steps: S00, a diseased leaf detector is constructed based on the YOLOv8 Nano model, and the diseased leaf detector is trained; S01, Input the original image of diseased leaves of multiple plants, the diseased leaf detector outputs a set of candidate bounding boxes of diseased leaves, each bounding box includes bounding box coordinates, detection confidence and class label; S02, based on the bounding box output by the diseased leaf detector, crop the image of the diseased leaf region from the original image.

[0013] Furthermore, step S1 includes the following steps: Step S10: Construct the Vision Mamba backbone network, using ViM-Tiny as the shared feature extractor, outputting feature representations with dimensions [B, 197, 192]. Where B represents the batch size, and 197 includes 196 patch markers plus 1 CLS marker. 192 represents the feature dimension; Step S11: Output a segmentation map through image segmentation. ,in, This represents the feature representation of the output from the Vision Mamba backbone. A segmentation head mapped to pixel-wise logarithmic odds; Step S12, calculate the segmentation features of interest ,in, This represents a segmentation mask with two channels: background and disease. This represents the intermediate segmentation features derived from the segmentation head. This indicates the first convolution, which expands S into an 8-channel feature map. (·) indicates that non-linearity is introduced after the first convolution. (·) represents the second convolution, which reduces the 8 channels to a single-channel attention map; σ(·) represents the sigmoid activation function; and ⊙ represents element-wise multiplication. Step S13, by analyzing the segmentation image The process involves applying 2×2 adaptive average pooling, followed by 1×1 point convolution and ReLU nonlinear activation, and finally flattening the result into a vector to obtain the disease features. ; Step S14, define classification prediction ,in, Represents the logarithm of the disease category. (·) is the classification header. It is a CLS tag from the Vision Mamba backbone. From the segmentation mask and intermediate segmentation features Derived attention segmentation features, It is the feature vector of the disease.

[0014] Furthermore, step S2 includes the following steps: Step S20: The image of the diseased leaf area detected by the YOLOv8 model is cropped and resized to 224×224 pixels and then input into the multi-task learning model. In step S21, the multi-task learning model simultaneously performs pixel-by-pixel segmentation, delineates the lesion area, and classifies the lesion type.

[0015] Furthermore, define the total loss function. ,in, Indicates the loss from partitioning. Represents classification loss, Indicates consistency loss. , , These are their respective weighting coefficients, which are determined through grid search to obtain the best results; Consistency loss in, This represents the average proportion of pixels in sample i that are predicted to be diseased. This represents the softmax confidence score of the health class in sample i.

[0016] Furthermore, step S3 also includes using the AdamW optimizer and cosine annealing learning rate scheduling to improve the stability and effectiveness of the training process.

[0017] Furthermore, step S3 also includes incorporating an "online enhancement" strategy to enhance the model's generalization ability under actual field conditions.

[0018] The working principle and beneficial effects of this invention: In implementing the technical solution of this invention, an architecture for leaf detection, segmentation, and disease classification based on YOLOv8 and a multi-task learning model is proposed. This architecture combines YOLOv8-based leaf detection with a visual Mamba multi-task learning structure, simultaneously performing semantic segmentation and disease classification. It is suitable for plant disease diagnosis from leaf RGB images, constructing an efficient and accurate disease classification and lesion segmentation system, thus advancing precision agriculture and sustainable crop management. Attached Figure Description

[0019] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein: Figure 1 This is a schematic diagram of the main steps of a leaf disease classification method based on YOLOv8 and a multi-task learning model according to the present invention. Detailed Implementation

[0020] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0021] Figure 1This is a schematic diagram illustrating the main steps of a leaf disease classification method based on YOLOv8 and a multi-task learning model according to the present invention. Figure 1 As shown, the leaf disease classification method based on YOLOv8 and a multi-task learning model in this embodiment mainly includes the following steps S0-S3.

[0022] Step S0: Use the YOLOv8 model to locate diseased leaves. By identifying the diseased areas in the original image of the diseased leaves, output the cropped image of the diseased leaf area to simplify the subsequent disease diagnosis process.

[0023] In one implementation, step S0, which uses the YOLOv8 model to locate diseased leaves, includes the following sub-steps S00-S02.

[0024] Step S00: Construct a diseased leaf detector based on the YOLOv8 Nano model and train the diseased leaf detector.

[0025] In this embodiment, the YOLOv8 Nano model operates at a fixed input size of 640×640 pixels and is trained based on training data with bounding box annotations. To improve detection robustness, various data augmentation strategies (such as random scaling, rotation, and illumination perturbation) are introduced during training.

[0026] Step S01: In the inference phase, the input is a complex image containing multiple plants (i.e., the original image of diseased leaves), and the diseased leaf detector outputs a set of candidate bounding boxes for diseased leaves. Each bounding box is defined by coordinates... The results consist of detection confidence and category labels (disease-affected leaves). Simultaneously, the detection results can be visualized as an image with overlaid bounding boxes for qualitative analysis.

[0027] Step S02: Based on the bounding box output by the diseased leaf detector, crop the corresponding leaf region (i.e., the image of the diseased leaf region) from the original image.

[0028] Step S1: Construct a multi-task learning model for segmentation-assisted classification.

[0029] In one embodiment, the construction of the segmentation-assisted classification multi-task learning model in step S1 includes the following sub-steps S10-S14.

[0030] Step S10: Construct the Vision Mamba backbone network, using ViM-Tiny as a shared feature extractor. This extractor processes the 224×224 input image and outputs a feature representation with dimensions [B, 197, 192]. Where B is the batch size, and 197 includes 196 patch markers plus 1 CLS marker. 192 is the feature dimension.

[0031] Step S11: Through image segmentation, refined segmentation features in the leaf space are obtained. The segmentation features are used to guide and enhance classification decisions. Specifically, the segmentation output is the segmentation map (i.e., the predicted segmentation mask). ,in, This represents the feature representation of the output from the Vision Mamba backbone. The segmentation head is mapped to pixel-wise logarithmic odds.

[0032] Step S12, calculate the segmentation features of interest ,in It focuses on segmentation features. It is a segmentation mask with two channels (background and disease). These are intermediate segmentation features derived from the segmentation head. This indicates the first convolution, which expands S into an 8-channel feature map. (·) Introduces non-linearity after the first convolution. The · sign represents the second convolution, which reduces the 8 channels to a single-channel attention map. σ(·) is the sigmoid activation function, ensuring that the attention value is between 0 and 1. ⊙ represents element-wise multiplication, applying the attention map to the segmentation features. ; Step S13, by analyzing the segmentation image The process involves applying 2×2 adaptive average pooling, followed by 1×1 point convolution and ReLU nonlinear activation, and finally flattening the result into a vector to obtain the disease features. .

[0033] Step S14 combines three complementary information sources: first, semantic global context information, which provides a comprehensive understanding of plant health through ViM-Tiny's CLS tags; second, spatial disease information, where the segmentation features of interest highlight specific disease areas; and third, disease global context information, which captures lesion patterns and severity through disease features for downstream classification. The definition of classification prediction ,in, Represents the logarithm of the disease category. (·) is the classification header. It is a CLS tag from the Vision Mamba backbone. From the segmentation mask and intermediate segmentation features Derived attention segmentation features, It is the feature vector of the disease.

[0034] Step S2: The detected leaves are segmented pixel by pixel using a multi-task learning model, and the segmented areas are classified according to disease type to ensure accurate identification of disease type.

[0035] In one embodiment, the segmentation and classification of the detected blades using a multi-task learning model in step S2 includes the following sub-steps S20-S21.

[0036] Step S20: The image of the diseased leaf area detected by the YOLOv8 model is cropped and resized to 224×224 pixels and input into the multi-task learning (MTL) model.

[0037] In this embodiment, to adapt to the input requirements of the multi-task learning model, all cropped leaf images were resized and padded to a uniform size of 224 × 224 pixels. The resulting set of standardized leaf images was then input into the Vision Mamba (ViM-Tiny) multi-task learning model for further disease segmentation and classification.

[0038] In step S21, the multi-task learning model simultaneously performs pixel-by-pixel segmentation, delineates the lesion area, and classifies the lesion type. The segmentation output highlights the spatial regions of interest, helping the classification part to learn more discriminative features.

[0039] Step S3: Design a custom total loss function that combines segmentation loss, classification loss, and consistency loss to improve the stability and accuracy of the model.

[0040] In one implementation, designing a custom total loss function in step S3 includes the following sub-steps S30-S32.

[0041] Step S30 involves designing a comprehensive loss function that includes pixel-wise cross-entropy (for segmentation), classification cross-entropy (for classification), and a novel consistency loss (to ensure consistency between segmentation and classification outputs). The total loss function is as follows. ,in, It is a segmentation loss. It is classification loss. It is our new type of consistency loss. , , These are the respective weight coefficients, which are determined through grid search to obtain the optimal result. In the multi-task loss function proposed in this invention, the weight coefficients of each loss term are determined through grid search as follows: =1.0、 =0.7、 =0.3.

[0042] Consistency loss

[0043] in, The mismatch between the proportion of diseased regions obtained from penalized segmentation and the health confidence score of the classifier. Let... Let be the average proportion of pixels in sample i that are predicted to be diseased. is the softmax confidence score for the healthy class in sample i. This loss is averaged over batch size N and is activated only under conflict conditions (disease exists but is classified as healthy, or no disease exists but is classified as disease).

[0044] Step S31: Use the AdamW optimizer and cosine annealing learning rate scheduling to improve the stability and effectiveness of the training process.

[0045] Step S32: Incorporate an "on-the-fly" strategy to enhance the model's generalization ability under actual field conditions.

[0046] Based on steps S0-S3 above, this invention develops an architecture for leaf detection, segmentation, and disease classification based on YOLOv8 and a multi-task learning model. This architecture combines YOLOv8-based leaf detection with a visual Mamba multi-task learning structure, simultaneously performing semantic segmentation and disease classification. It is suitable for plant disease diagnosis from leaf RGB images, constructing an efficient and accurate disease classification and lesion segmentation system, thus advancing precision agriculture and sustainable crop management. This invention utilizes the Vision Mamba (ViM) backbone network for efficient linear complexity feature extraction, and combines it with segmentation-assisted classification design. By guiding and optimizing disease classification through segmentation output, it can achieve an accurate understanding of the spatial details and global background of diseases, and adapt to the differentiation and localization of diseases in complex backgrounds. To enhance the synergistic effect of multi-task training, this invention introduces a custom consistency loss function to force the consistency of segmentation and classification results, thereby improving the coordination and consistency of multi-task training results. Statistical verification has shown that this method effectively improves model performance.

[0047] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0048] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A leaf disease classification method based on YOLOv8 and a multi-task learning model, characterized in that, Includes the following steps: S0. Identify the diseased areas in the original image of diseased leaves based on the YOLOv8 model, and output the cropped image of the diseased leaf area. S1. Construct a multi-task learning model for segmentation-assisted classification; S2. The detected leaves are segmented pixel by pixel using a multi-task learning model, and the segmented areas are classified according to disease type. S3. Design a custom total loss function by combining segmentation loss, classification loss and consistency loss, and train a multi-task learning model.

2. The leaf disease classification method based on YOLOv8 and a multi-task learning model according to claim 1, characterized in that, Step S0 includes the following steps: S00, a diseased leaf detector is constructed based on the YOLOv8 Nano model, and the diseased leaf detector is trained; S01, Input the original image of diseased leaves of multiple plants, the diseased leaf detector outputs a set of candidate bounding boxes of diseased leaves, each bounding box includes bounding box coordinates, detection confidence and class label; S02, based on the bounding box output by the diseased leaf detector, crop the image of the diseased leaf region from the original image.

3. The leaf disease classification method based on YOLOv8 and a multi-task learning model according to claim 1, characterized in that, Step S1 includes the following steps: Step S10: Construct the Vision Mamba backbone network, using ViM-Tiny as the shared feature extractor, outputting feature representations with dimensions [B, 197, 192]. Where B represents the batch size, and 197 includes 196 patch markers plus 1 CLS marker. 192 represents the feature dimension; Step S11: Output a segmentation map through image segmentation. ,in, This represents the characteristic representation of the output of the VisionMamba backbone. A segmentation head mapped to pixel-wise logarithmic odds; Step S12, calculate the segmentation features of interest ,in, This represents a segmentation mask with two channels: background and disease. This represents the intermediate segmentation features derived from the segmentation head. This indicates the first convolution, which expands S into an 8-channel feature map. (·) indicates that non-linearity is introduced after the first convolution. (·) represents the second convolution, which reduces the 8 channels to a single-channel attention map; σ(·) represents the sigmoid activation function; and ⊙ represents element-wise multiplication. Step S13, by analyzing the segmentation image The process involves applying 2×2 adaptive average pooling, followed by 1×1 point convolution and ReLU nonlinear activation, and finally flattening the result into a vector to obtain the disease features. ; Step S14, define classification prediction ,in, Represents the logarithm of the disease category. (·) is the classification header. It is a CLS tag from the Vision Mamba backbone. From the segmentation mask and intermediate segmentation features Derived attention segmentation features, It is a disease feature vector.

4. The leaf disease classification method based on YOLOv8 and a multi-task learning model according to claim 1, characterized in that, Step S2 includes the following steps: Step S20: The image of the diseased leaf area detected by the YOLOv8 model is cropped and resized to 224×224 pixels and then input into the multi-task learning model. In step S21, the multi-task learning model simultaneously performs pixel-by-pixel segmentation, delineates the lesion area, and classifies the lesion type.

5. The leaf disease classification method based on YOLOv8 and a multi-task learning model according to claim 1, characterized in that, Define the total loss function ,in, Indicates the partition loss. Represents classification loss. Indicates consistency loss. , , These are their respective weighting coefficients, which are determined through grid search to obtain the best results; Consistency loss in, This represents the average proportion of pixels in sample i that are predicted to be diseased. This represents the softmax confidence score of the health class in sample i.

6. The leaf disease classification method based on YOLOv8 and a multi-task learning model according to claim 5, characterized in that, Step S3 further includes using the AdamW optimizer and cosine annealing learning rate scheduling to improve the stability and effectiveness of the training process.

7. The leaf disease classification method based on YOLOv8 and a multi-task learning model according to claim 5, characterized in that, Step S3 also includes incorporating an "online enhancement" strategy to enhance the model's generalization ability under actual field conditions.

Images