A real-time detection method for the severity of disease on leaves of various fruits

Abstract

A real-time detection method for the severity of disease on leaves of various fruits. This method belongs to the field of neural network target detection technology. It solves the technical problem of how to more accurately assess the severity of disease on fruit leaves. A real-time detection model for the location of leaf lesions is constructed. The structure of the real-time detection model for the location of leaf lesions is specifically as follows: the encoder part is constructed with a Transformer module as the main body, and an expanded feature module is provided in the feature extraction stage, and a feature weighted fusion module is provided in the feature fusion stage; the decoder part is constructed with a multi-layer perceptron as the main body. Compared with existing segmentation methods, the method of the present invention shows a better segmentation effect on different fruit leaf data sets.

Description

Technical Field

[0001] The invention belongs to the technical field of neural network target detection, and in particular relates to a real-time detection method for the severity of diseases of leaves of various fruits. Background Art

[0002] Fruits are rich in vitamins and other nutrients needed by the human body. However, due to environmental and weather conditions, leaf diseases frequently occur on fruit trees, significantly reducing their photosynthesis, affecting yields and fruit quality, and bringing severe economic impacts to fruit farmers.

[0003] Therefore, accurate analysis of fruit tree disease conditions and timely preventive measures can effectively reduce various losses for growers. Traditionally, diagnosing crop diseases involves visually observing symptoms and spots and seeking expert advice. This is difficult and labor-intensive for large, multi-species orchards, and also increases growers' financial costs.

[0004] Therefore, research on fruit tree disease detection is of great significance to agriculture. Image segmentation, by assigning labels to each pixel, facilitates a deeper understanding and analysis of disease conditions. Disease segmentation improves the accuracy of disease diagnosis and allows for real-time tracking and monitoring of disease trends. This effectively helps growers and plant protection personnel understand crop health status, develop effective control measures, prevent the spread of disease, and thus reduce pesticide use and ensure food safety.

[0005] In the early days of smart agriculture, traditional machine learning methods extracted features such as color, morphology, or texture from images and set specific thresholds to identify and locate target leaves and diseased spots. With the continuous development of deep learning, researchers have gradually applied deep learning to the agricultural field and achieved promising results.

[0006] However, there are still many difficulties in segmenting pathological images of different fruit tree leaves in real complex environments. In terms of leaves, (1) the curling and folding of leaves will cause shadows, which will increase the difficulty of segmentation. (2) The special shape of the leaf edge makes it challenging to extract edge features. (3) The alternating overlap between leaves makes it difficult to extract the global features of the leaves. In terms of disease spots, (1) the strong and dark changes in light intensity will cause the boundaries of the disease spots to be blurred, and the color of the spots will be similar to the color of the leaves, resulting in inaccurate segmentation results. (2) In areas with dense small disease spots, it is easy to miss spots and sticky segmentation. (3) The reflection after light in the post-rain environment makes the disease spot features blurred, making it difficult to extract the features of the light spots.

[0007] Therefore, how to efficiently overcome the various leaf and spot features and occlusion overlap problems in different environments to achieve precise segmentation and thus more accurately assess the severity of fruit leaf diseases is a current technical challenge in this field. Summary of the Invention

[0008] In order to solve the above technical problems, the present invention provides a real-time detection method for the severity of diseases on leaves of various fruits.

[0009] The method comprises the following steps:

[0010] S1. Data collection: Collect images of various fruit leaf lesions and perform preprocessing;

[0011] S2. Constructing a dataset: Dividing the pre-processed fruit leaf lesion images into a training set, a validation set, and a test set;

[0012] S3. Model construction: Construct a real-time leaf lesion location detection model. The structure of the real-time leaf lesion location detection model is specifically composed of two parts: an encoder and a decoder. The encoder part is constructed mainly with a Transformer module, and an expansion feature module is provided in the feature extraction stage, and a feature weighted fusion module is provided in the feature fusion stage; the decoder part is constructed mainly with a multilayer perceptron;

[0013] S4. Model training: Use the constructed data set to train the real-time detection model for leaf lesion locations and adjust the model parameters until the model meets the detection requirements;

[0014] S5. Use the trained leaf lesion location real-time detection model to perform real-time detection of lesion locations on the fruit leaves to be detected, and use the proportion of the area of ​​the lesion location on the fruit leaves to the total area of ​​the fruit leaves to estimate the severity of the disease on the fruit leaves.

[0015] Furthermore, the preprocessing is specifically as follows: uniformly adjusting the fruit leaf lesion images to the same size and manually labeling the categories; and performing random data enhancement operations on the fruit leaf lesion images.

[0016] Furthermore, the specific structure of the encoder is as follows: after the feature map is input, the data is divided into two paths, one path passes through the expanded feature module, and the other path passes through the overlapping patch embedding module and the Transformer module in sequence. The two feature maps generated after the two-path data processing are both input into the feature weighted fusion module for feature fusion. The fused features pass through three Transformer modules in sequence and then enter the decoder.

[0017] Furthermore, the specific structure of the feature expansion module is as follows: after the feature map is input, it first passes through the first layer 1×1 convolution module to generate a new feature channel, then passes through the void convolution module to expand the receptive field, and then passes through the second layer 1×1 convolution module to compress and integrate the information to output the feature map containing the disease, and then performs feature fusion of the input feature map and the feature map containing the disease; a SiLU activation function layer and a batch normalization layer are introduced between the first layer 1×1 convolution module and the void convolution module, and a SiLU activation function layer and a batch normalization layer are introduced between the void convolution module and the second layer 1×1 convolution module.

[0018] Furthermore, the specific structure of the Transformer module is as follows: after the feature map is input, it first passes through the deep multi-scale attention mechanism module to extract contextual semantic features and local features; then passes through the Mix-FFN module to enhance the feature expression ability of the Transformer module; finally, it enters the patch merging layer to downsample the input feature map, while gradually expanding the receptive field and integrating multi-scale information.

[0019] Furthermore, the specific structure of the deep multi-scale attention mechanism module is as follows: the input feature map first passes through the multi-head attention mechanism module to capture the texture and morphological features of different diseases and output output features with global context; then it is input into the deep multi-scale module, and the features are convolved using convolution kernels of different sizes to capture local features of different scales.

[0020] Furthermore, the data processing performed in the multi-head attention mechanism module is specifically as follows:

[0021] Generate query matrix through linear transformation , key matrix Sum Matrix ,pass Get attention weight ,in, express function, Represents the dimension of the attention head; then through Get output features with global context ,in, represents the random deletion of neurons. Indicates that the result is projected back to the original dimension through the linear layer. represents the transpose operation, Indicates the batch size, Represents a sequence, Indicates the number of channels, express The transposed matrix of .

[0022] Furthermore, the data processing performed in the deep multi-scale module is specifically as follows:

[0023] Use convolution kernels of different sizes to align input features Perform convolution, and the output results of multiple convolution operations are aggregated into a multi-scale convolution list , ,in, Denotes the convolution and size is When the input feature After convolution, add the output of all convolution kernels to get the aggregate feature , aggregate features Compressed into a vector through global average pooling , map S to a smaller dimension through a fully connected layer , through another fully connected layer Mapped to The same dimension, thus generating a weight matrix , using the weight matrix Each weight in performs a weighted combination of the features output by convolution kernels of different sizes to obtain a combined feature map, which is then output after passing through a fully connected layer.

[0024] Furthermore, the data processing performed in the feature weighted fusion module is specifically as follows:

[0025] Two input feature maps and The joint feature map is spliced ​​in the channel dimension to generate a joint feature map, and the joint feature map is processed by 1×1 convolution operation to generate a weighted weight map. The generated weighted weight map is processed by After the nonlinear transformation of the activation function, ensure that the weight value is between 0 and 1, and obtain the weighted weight map ; The feature map is weighted summed and Fusion is performed to obtain fusion features , ,in, Represents the batch normalization operation, and then the fusion features After performing 1×1 convolution and batch normalization operations in sequence, the final output feature map is obtained.

[0026] Furthermore, the decoder is composed of two multi-layer perceptron modules connected in sequence.

[0027] The beneficial effects of the method of the present invention are:

[0028] A real-time leaf lesion location detection model was constructed. This model uses a single-layer parallel fusion architecture to extract dense feature representations at different scales across two paths, significantly enhancing the ability to extract features of lesions of varying morphology and reducing missed lesions. The deep multi-scale attention mechanism module further extracts multi-level features by introducing a deep multi-scale module. The module also leverages the location information provided by the module to achieve global modeling while maintaining low computational complexity.

[0029] The system also innovatively proposes a feature expansion module that works in conjunction with the SegFormer module to extract global and local features, enabling more refined segmentation of leaf and lesion edges and effectively extracting tiny spots. The feature weighted fusion module uses an adaptive weighting strategy to fuse shallow features containing details and edge information with deep features rich in semantic information, achieving precise feature recovery.

[0030] Experimental results demonstrate that the proposed method achieves superior segmentation results compared to existing segmentation methods on various fruit leaf datasets, demonstrating the strong generalization and robustness of the real-time leaf lesion location detection model, providing effective technical support for pathological image analysis of various fruit leaves. Notably, the real-time leaf lesion location detection model outperforms most existing models in terms of params and FLOPs, and its inference speed on local servers and edge devices meets practical agricultural needs.

[0031] The researchers validated the model's performance in experiments using datasets containing six fruit varieties and seven diseased leaf diseases. Generalization tests on two grapevine diseases using the Plant Village dataset further verified the model's robustness and generalization capabilities. Future research will focus on further improving the model's accuracy and reducing computational costs, while also conducting extensive experiments on more diseased leaf datasets for crops to promote the method's practical application in smart agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of manually labeling raw data in an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of manual annotation of brown spot and black rot of grapes in an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the result of enhancing a dataset image in an embodiment of the present invention;

[0035] Figure 4 This is a structural diagram of a real-time detection model for leaf lesion locations in an embodiment of the present invention;

[0036] Figure 5 This is a structural diagram of the EFM expanded feature module in an embodiment of the present invention;

[0037] Figure 6 This is a structural diagram of the DM-Attention deep multi-scale attention mechanism module in an embodiment of the present invention;

[0038] Figure 7 This is a structural diagram of the FWFM feature weighted fusion module in an embodiment of the present invention;

[0039] Figure 8 This is a comparison chart of the disease segmentation visualization results of apple rust, apple leaf spot disease, and plum red spot disease using different models in the embodiments of the present invention;

[0040] Figure 9 This is a comparison chart of the visualization results of disease segmentation for grape white rot and pear scab using different models in an embodiment of the present invention;

[0041] Figure 10 This is a comparison chart of the disease segmentation visualization results of mango brown spot disease and pomegranate cercospora leaf spot disease in an embodiment of the present invention;

[0042] Figure 11 This is a comparison diagram of the disease segmentation visualization results of grape brown spot disease and grape black rot in an embodiment of the present invention;

[0043] Figure 12 This is a visualized heat map of the single attention area of ​​six fruit leaves and seven lesions in the embodiment of the present invention;

[0044] Figure 13 This is a visualized heat map of six fruit leaves and seven disease spots in the embodiment of the present invention;

[0045] Figure 14 The disease severity evaluation results of five fruit leaves with different severity levels in the embodiment of the present invention are shown;

[0046] Figure 15 This is the main interface diagram of the fruit tree leaf disease segmentation system in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] Example 1

[0049] This embodiment provides a real-time detection method for the severity of diseases on leaves of various fruits, characterized in that the method comprises the following steps:

[0050] S1. Data collection: Collect images of various fruit leaf lesions and perform preprocessing;

[0051] S2. Constructing a dataset: Dividing the pre-processed fruit leaf lesion images into a training set, a validation set, and a test set;

[0052] S3. Model construction: Construct a real-time leaf lesion location detection model. The structure of the real-time leaf lesion location detection model is specifically composed of two parts: an encoder and a decoder. The encoder part is constructed mainly with a Transformer module, and an expansion feature module is provided in the feature extraction stage, and a feature weighted fusion module is provided in the feature fusion stage; the decoder part is constructed mainly with a multilayer perceptron;

[0053] S4. Model training: Use the constructed data set to train the real-time detection model for leaf lesion locations and adjust the model parameters until the model meets the detection requirements;

[0054] S5. Use the trained leaf lesion location real-time detection model to perform real-time detection of lesion locations on the fruit leaves to be detected, and use the proportion of the area of ​​the lesion location on the fruit leaves to the total area of ​​the fruit leaves to estimate the severity of the disease on the fruit leaves.

[0055] Example 2

[0056] This embodiment is a further limitation of Example 1. Step S1 is further explained. In this embodiment, a total of 1,417 image data of 7 types of diseases of 6 kinds of fruits were collected, including images of 4 kinds of fruits in real environments and data of 2 kinds of fruits in experimental environments. The data under real environments were collected from the teaching and research base of Jilin Agricultural University. The collection time was from July to September 2024, during the fruit tree fruiting period. It was divided into two time periods, noon and evening, and photographed at a fixed distance of 5-10 cm. It also included image data after rain. In order to enhance the generalization of the model, disease images of two crops, mango and pomegranate, were also selected from the public crop disease and pest identification dataset PlantVillage. In addition, grape brown spot and black rot were selected as separate validation sets to further evaluate the generalization and robustness of the model.

[0057] Due to the inconsistent pixel size of the images during the acquisition process, to ensure data consistency, the images were uniformly resized to 512×512 pixels. The original data was manually annotated with the professional semantic segmentation and annotation software Labelme for model learning and training. The annotated dataset covers 7 diseases of 6 fruit crops, a total of 13 categories, and is saved in JSON format. The annotation style is as follows: Figure 1 As shown, (a)-(f) represent disease samples of different fruits, respectively. The colors of different categories correspond to the subcategories in the task, which are used to clearly mark the distinction between categories.

[0058] To verify the generalization ability of the model, we selected grape brown spot and black rot from the Plant Village dataset as the validation dataset. All images were resized to 256×256 pixels and individually annotated. The annotations for each image were saved in JSON format. For details on the classification, see Figure 2 , where (a) is grape brown spot disease and (b) is grape black rot disease, and the four colors represent different categories. Although the label colors in this dataset are similar to some of the colors in Figure 1, due to the different data sources, the two are independent of each other and therefore do not cause any interference or impact.

[0059] During target training, if the training dataset is small, deep learning networks are prone to overfitting, meaning the model performs well on the training data but poorly on unseen data, which reduces the network's generalization ability. To address this issue, we performed random data augmentation operations on the collected image data, such as rotation, cropping, adjusting contrast, and adding black 10 20 mask blocks, etc. This not only effectively increases the diversity of training data, but also minimizes the problem of uneven distribution of training samples, thereby improving the adaptability and robustness of the model in different scenarios. Figure 3 As shown, (a) is the random flipping of the image, (b) is the image contrast adjustment, (c) is the image cropping, and (d) is the addition of a black mask block.

[0060] Example 3

[0061] This embodiment further limits the embodiment 1. Step S3 is further described.

[0062] In view of the characteristics of various fruit leaf diseases, this paper proposes a real-time leaf lesion location detection model based on SegFormer (abbreviated as "Disease-Seg"). This model combines the advantages of CNN and Transformer in feature representation and aims to accurately identify different types of leaf diseases. Figure 4The overall structural design of Disease-Seg is presented. The model consists of two parts: an encoder and a decoder. The encoder part proposes a single-layer parallel fusion encoder structure. By combining the advantages of traditional CNN and Transformer, it effectively integrates local features with global context information without increasing network depth, while also minimizing missed features.

[0063] In detail, the feature maps generated by the two different encoders are all { } resolution. Each stage outputs a channel size of {16, 32, 64, 160, 256}. The EFM feature expansion module uses dilated convolution to expand the receptive field, improving the perception of local information without increasing the number of model parameters.

[0064] The FWFM feature weighted fusion module (Feature Weighted Fusion Module) adaptively weighted fusion can avoid the redundant information caused by simple splicing and enhance the effectiveness of feature expression. This paper also proposes a DM-Attention deep multi-scale attention mechanism. This mechanism leverages the traditional attention mechanism's ability to effectively extract global contextual information while combining multi-scale depthwise separable convolution and pointwise convolution. It also stacks depthwise separable convolutions of different scales. This not only effectively reduces the model's parameter count and computational overhead, but also significantly improves the ability to extract multi-scale features.

[0065] In the decoder part, the multi-scale feature maps from the encoder are extracted and The images are downsampled at a rate of , then fused and upsampled to their original size, resulting in Segmentation results. is the number of channels, which is 3 in this embodiment.

[0066] Example 4

[0067] This embodiment is a further limitation of embodiment 3. The EFM enlarged feature module is further described. To address the problem of feature ambiguity caused by small spots on fruit tree leaves or uneven illumination, an improved EFM enlarged feature module is proposed, aiming to improve the accuracy of disease area detection through a sophisticated network design. This module uses a series of designed convolution operations to capture and extract disease features of different scales, and is particularly suitable for early small disease spots on fruit tree leaves. The structural diagram of the EFM enlarged feature module is shown in Figure 5 .

[0068] First, the module uses 1×1 convolution to transform the input feature map into In terms of channels Mapped to This operation generates new feature channels by weighted summing of all input channels at each spatial position while keeping the spatial dimension unchanged. , and avoid excessive parameters while maintaining the richness of feature information.

[0069] Next, we introduce the dilation convolution (Dia Conv) and set the dilation rate to 2, and convert the input to The feature map is downsampled 2x times in the spatial layer, and the receptive field of the convolution kernel is significantly expanded without adding additional parameters, helping the model to effectively capture a wide range of local features.

[0070] In the network structure, the Sigmoid-Weighted Linear Unit (SiLU) activation function is introduced after each convolution layer to enhance nonlinear transformation capabilities and smoothly increase input values ​​from a range near 0. This feature makes the network more sensitive to subtle changes in leaf edges or disease boundaries, effectively improving segmentation accuracy. Especially when processing lesion edges, SiLU can make the model's segmentation in diseased areas more detailed and accurate.

[0071] In addition, a Batch Normalization (BN) layer is added to the network to standardize the feature maps. This operation helps reduce the statistical differences between different channels, thereby preventing overfitting and improving the model's robustness to changes in lighting or the presence of lesions on different leaf parts.

[0072] Finally, the module compresses and integrates the disease information in terms of channels through another layer of 1×1 convolution to generate an output feature map containing the disease, providing high-quality input for feature fusion and prediction of subsequent modules.

[0073] ;

[0074] in, is the input tensor, is the convolution kernel, For bias, then Perform BatchNormalization operation:

[0075] ;

[0076] Next, apply the SiLU activation function:

[0077] ;

[0078] After that, the dilated convolution operation is performed:

[0079] ;

[0080] is the convolution kernel, For bias.

[0081] Then apply Batch Normalization and SiLU activation function:

[0082] ;

[0083] Finally, a 1×1 convolution operation is performed to compress and integrate feature information:

[0084] .

[0085] This design enhances the model's ability to capture disease features through multi-level convolution and activation functions, especially when processing lesion details, which can effectively improve the model's segmentation accuracy and robustness.

[0086] Example 5

[0087] This embodiment further limits Example 3 and further explains the DM-Attention deep multi-scale attention mechanism module.

[0088] When considering the problem of identifying disease edges and detailed features, the attention mechanism in SegFormer uses a reduction ratio R to reduce the length of the sequence, thereby reducing computational complexity. However, different reduction ratios may lead to feature loss, especially for capturing details and edge information in the image.

[0089] Therefore, to overcome this problem, a new method called DM-Attention deep multi-scale attention mechanism is proposed. The core idea of ​​this method is to use depthwise separable convolution to independently process the input channels, thereby effectively reducing the computational complexity of the suppression model.

[0090] On this basis, the DMSM Deep Multi-scale Module is used to gradually extract multi-scale features, ultimately achieving a richer and more diverse feature representation while maintaining a low parameter count. Through this design, DM-Attention can not only reduce computational and memory consumption while maintaining efficient capture of details and edge information. This is especially true for tasks that require detailed feature representation, such as agricultural disease segmentation. Figure 6 .

[0091] First, the input features are processed by a multi-head attention mechanism. This mechanism generates a query matrix through linear transformation , key matrix Sum Matrix , and focus on the diseased area from different angles based on multiple heads.

[0092] This design enhances the model's ability to extract details in fruit tree leaf disease detection, effectively capturing the texture and morphological characteristics of different diseases, and is particularly adaptable to irregular disease shapes on leaves. Specifically, the model first transforms the input features into Mapping to query matrix , then, The dimension of is adjusted so that each attention head focuses on different feature channels. The formula is as follows:

[0093] ;

[0094] in, represents the transpose operation, represents the permutation operation, Represents a mapping operation. The three operations mentioned above implement the function of transforming the input Mapping to query matrix , Indicates the batch size, Represents a sequence, Indicates the number of channels, Indicates the number of heads.

[0095] Bond Matrix Sum Matrix The generation method and query matrix , two independent matrices are obtained through the linear layer and Divide.

[0096] ;

[0097] in, Represents the partition operation, query matrix and bond matrix After the dot product, it is scaled by the scaling factor and then by Function gets the attention weight :

[0098] ;

[0099] in, Represents the dimension of the attention head. By using the attention weight pair matrix The weighted result is then projected back to the original dimension through the linear layer to finally obtain the output feature with global context , the formula is as follows:

[0100] ;

[0101] in, represents the random deletion of neurons. Indicates that the result is projected back to the original dimension through the linear layer. Represents a transpose operation.

[0102] Next, to further improve the feature expression capability, DM-Attention introduces the DMSM deep multi-scale module, which convolves the features using convolution kernels of different sizes (1×1, 3×3, 5×5, 7×7) to capture local features of different scales. The output results of multiple convolution operations are aggregated into a multi-scale convolution list. , and its calculation formula is:

[0103] ;

[0104] Then the different scale features are added together to generate aggregate features , which is the superposition of all convolutional outputs, is used to represent the overall multi-scale information.

[0105] ;

[0106] in, Indicates the length of the multi-scale convolution list.

[0107] The feature map is compressed into a vector by global average pooling , thereby obtaining the global features of the input:

[0108] ;

[0109] in, and are the height and width of the feature map respectively.

[0110] Through a fully connected layer Will Mapping to a smaller dimension , reducing the computational burden while speeding up inference and guiding the subsequent weighting process:

[0111] ;

[0112] Through another fully connected layer Will Mapped to The same dimension, thus generating a weight matrix (Right now )

[0113] ;

[0114] Weight Matrix Each weight in Used to perform weighted combination of features of different convolution kernel sizes to obtain a combined feature map , the specific formula is as follows:

[0115] ;

[0116] Finally, the combined feature map After passing through a fully connected layer, the output is generated. DM-Attention further enhances the model's adaptability to different disease morphologies, textures, and structures, thereby improving the overall performance of fruit tree disease detection. It can more accurately detect diseased areas and capture subtle disease features on leaves.

[0117] Example 6

[0118] This embodiment is a further limitation of embodiment 3. The FWFM feature weighted fusion module is further described. In a single-layer parallel encoder structure, since different encoders focus on different aspects of feature extraction, the FWFM feature weighted fusion module aims to effectively integrate features from the two encoders in an adaptive manner. Figure 7 .

[0119] Two input feature maps and Concatenate along the channel dimension to generate a joint feature map that contains all the information from both encoders , the joint feature map is processed by 1×1 convolution operation to generate a weighted weight map, which is used to perform weighted fusion of input features in subsequent steps. The generated weighted weight map is processed by After the nonlinear transformation of the activation function, ensure that the weight value is between 0 and 1, and obtain the weighted weight map Specifically, each element of the weight map reflects the importance of the corresponding position in the final fusion feature map. and Fusion is performed to obtain fusion features , ,in, Represents the batch normalization operation, and then the fusion features After performing 1×1 convolution and batch normalization operations, the final output feature map is obtained. This weighted fusion strategy enables the module to flexibly adjust the fusion ratio of the two feature maps based on their contribution, thereby improving the expressiveness of the final fused feature map and enhancing the model's ability to model complex features.

[0120] Through this weighted fusion mechanism, the FWFM module can not only effectively combine feature maps from two different sources, but also capture more contextual information at multiple levels, ultimately optimizing the performance of the model, especially in complex tasks that require diverse feature extraction.

[0121] Example 7

[0122] This embodiment provides the test results of the Disease-Seg real-time detection model for leaf lesion locations according to the present invention on an experimental platform to further illustrate the superiority of its detection performance.

[0123] 1. Experimental platform and evaluation indicators:

[0124] To verify the effectiveness of the proposed Disease-Seg model, all experiments were conducted under consistent hardware and software environment conditions. The specific settings are as follows: The experiments were conducted on an NVIDIA TU102 [GeForce RTX 2080 Ti Rev. A] GPU, equipped with an 11th generation Intel® Core™ i5-11400F processor (2.60 GHz, 12 cores). The optimizer used the AdamW optimization algorithm with weight decay, the learning rate decay strategy was cosine decay, the momentum was set to 0.9, and the weight decay coefficient was , the batch size is 16, and the training epochs are set to 300.

[0125] To verify the robustness of the model, six evaluation metrics are used to evaluate the proposed model: Intersection over Union (IoU), Mean Intersection over Union (mIoU), Mean Pixel Accuracy (mPA), Accuracy (Acc), Floating Point Operations (FLOPs), and Parameters (Params). These metrics evaluate the model from various aspects.

[0126] The IoU indicator is used to evaluate the degree of overlap between the model's predicted segmentation results and the actual results on a single category, and is a key indicator for measuring the accuracy of single-category segmentation.

[0127] mIoU is used to evaluate the prediction effect of each category in the segmentation task. By measuring the degree of overlap between the predicted area and the true area, it can effectively reflect the overall performance of the model.

[0128] mPA is used to evaluate the proportion of correctly classified pixels in each category. It can measure the pixel-level prediction effect of each category.

[0129] Acc evaluates the classification accuracy of the model as a whole, that is, the proportion of all correctly predicted pixels to the total pixels.

[0130] Common metrics used in linear regression analysis include R² and RMSE. R² (coefficient of determination) measures the goodness of fit between the model's predicted values ​​and the actual values; values ​​closer to 1 indicate a more accurate model. RMSR (root mean square residual) indicates the average deviation between the predicted and actual values; smaller values ​​are better.

[0131] FLOPs measures the computational complexity of a model by measuring the amount of computation performed during inference. This ultimately determines its suitability for deployment on resource-constrained devices. Params, the sum of the number of parameters across all layers, measures the complexity and size of the model.

[0132] 2. Experimental results

[0133] This section compares the performance of the proposed Disease-Seg model with several current state-of-the-art and classic semantic segmentation techniques, including PSPNet, HRNetV2, U-Net, DeepLabV3+, SegFormer, SegNeXt, DANet, OCRNet, and UPerNet. Some models are based on the open-source semantic segmentation framework mmsegmentation. These techniques represent three different architectural paradigms: CNN, pure Transformer, and a fusion of CNN and Transformer. The experiments specifically focused on improving the boundary clarity between the leaf and its diseased area, as well as the accuracy of segmenting small points.

[0134] (1) Comparison of effects under different models

[0135] To better verify the performance of the Disease-Seg model in real-world scenarios, as shown in Tables 1 and 2, we performed a single-category segmentation on samples such as apples, grapes, and plums in the dataset, and analyzed the segmentation performance of each model on specific categories in detail. The highest performance indicators are shown in bold.

[0136] Table 1 shows the segmentation results for apple rust, leaf spot disease, grape white rot, and plum red spot disease in real-world outdoor environments. In the target lesion segmentation task, the proposed Disease-Seg model achieved the best performance across all seven lesion types. In the leaf segmentation task, its segmentation accuracy remained stable at approximately 98%. For example, in the segmentation of apple leaf spot disease, the Disease-Seg model achieved an Intersection over Union (IoU) value improvement of 33%, 44%, and 38% compared to OCRNet, UPerNet, and PSPNet, respectively. It also achieved a 14% improvement over SegNeXt, the best-performing comparison method.

[0137] The Disease-Seg model also performed exceptionally well in the other three disease spot segmentation tasks, achieving an average 13% higher segmentation accuracy than the top-performing SegFormer and U-Net models in these categories. In summary, the Disease-Seg model performs exceptionally well in complex real-world environments, effectively overcoming interference factors such as leaf folds, jagged edges, and varying lighting conditions, achieving accurate segmentation of leaves and disease spots.

[0138] Table 1:

[0139]

[0140] As shown in Table 2, the Disease-Seg model also performs well in the segmentation tasks of indoor pomegranate cercospora spot, indoor mango brown spot, and outdoor pear black spot.

[0141] For example, in the segmentation of pomegranate cercospora spots, models such as HRNetV2, UPerNet, and OCRNet all performed significantly worse than our proposed method in terms of leaf and lesion segmentation accuracy. In the mango disease category, although SegFormer performed best among the compared models, its IoU values ​​for leaf and lesion segmentation were still 1% and 16% lower than our proposed method, respectively. In the pomegranate category, UPerNet's object point segmentation performance also lagged behind our proposed method, with IoU values ​​for leaf segmentation being 6% lower and IoU values ​​for lesion segmentation being 15% lower.

[0142] PSPNet performed the worst in all comparison experiments. While its pyramid pooling module is advantageous in capturing global background features, it is less applicable to agricultural disease segmentation tasks that require a balance between local and global features. Further statistical analysis demonstrates that our method achieves higher accuracy and robustness in small object segmentation tasks, providing valuable reference and potential practical guidance for agricultural disease segmentation research.

[0143] Table 2:

[0144]

[0145] To more comprehensively validate the overall performance of our method in different scenarios, we compared it with a CNN-based method, a Transformer-based method, and a classic and efficient model based on the fusion of CNN and Transformer. Table 3 shows the performance of the Disease-Seg model and other comparison methods based on evaluation metrics such as mIoU, Acc, Params, FLOPs, and FPS, where bold and underlined columns indicate the highest and second-highest values, respectively.

[0146] CNN-based methods have the characteristics of shared weights, local connections, and inductive biases, which enable them to learn visual representations at a faster inference speed. However, due to the lack of CNN's ability to capture long-range dependencies, the segmentation performance of small diseases in complex backgrounds is poor. For example, the mIoU of DeepLabV3+, HRNetV2, and U-Net are 78.54%, 83.45%, and 82.61%, respectively, which are 11.78%, 6.87%, and 7.71% lower than the Disease-Seg model, respectively. Nevertheless, in terms of Params, the number of parameters of PSPNet is 2.41M lower than that of the Disease-Seg model, showing its certain lightweight advantage.

[0147] Transformer-based methods use a self-attention mechanism to calculate the correlation between each position in the feature sequence and other positions, thereby dynamically adjusting feature representations and capturing global context information. However, because weight distribution relies on global similarity, Transformer methods may ignore fine-grained structures or texture features in the absence of explicit neighborhood modeling.

[0148] For example, UPerNet's mIoU improves by 7.79% and 7.45% compared to SegFormer and SegNeXt, both of which use a lightweight decoder. Furthermore, the Disease-Seg model is 1.8 times faster than SegFormer in terms of FPS, but 0.7 times slower than SegNeXt. Experimental results demonstrate that the Disease-Seg model effectively captures local, fine-grained semantic information and demonstrates strong representational capabilities in high-FPS disease segmentation tasks.

[0149] Methods based on the fusion of Transformers and CNNs aim to complement global and local features. However, failing to effectively balance accuracy improvement with parameter size can lead to an imbalance in model performance and efficiency. The Disease-Seg model achieves an excellent trade-off between accuracy and computational effort. In comparison, OCRNet and UPerNet achieve mIoUs 18.60% and 25.67% lower than our model, respectively, while outperforming it by 7.3M and 37.17G in Params and 36.0M and 184.75G in FLOPs.

[0150] While the Disease-Seg model did not achieve optimal performance in terms of params, FLOPs, and FPS, its overall performance remained among the best, meeting the computing requirements of cloud server and edge device deployments. In summary, these experimental results fully demonstrate the effectiveness and applicability of the Disease-Seg model for agricultural fruit and leaf disease image segmentation, providing important technical support and theoretical reference for precision disease management in smart agriculture.

[0151] Table 3:

[0152]

[0153] exist Figure 8 The figure shows a detailed comparative analysis of disease segmentation visualizations for apple rust (A), apple leaf spot (B), and plum red spot (C) using different models. The white outlines indicate differences between the models. The results indicate that PSPNet's segmentation performance is unsatisfactory. This problem stems primarily from its multi-scale pyramid pooling structure, which can lead to loss of detailed information during downsampling, thus affecting segmentation accuracy.

[0154] For example, in the white-boxed area of ​​the apple rust segmentation result (A), while SegFormer, U-Net, and HR-NetV2 demonstrated some ability in segmenting smaller lesions, they still missed target areas compared to the label, failing to achieve complete segmentation. In the apple leaf blotch segmentation result (B), SegNeXt was able to segment leaves and lesions fairly accurately, closely matching the labeled image. However, in the white-boxed area, its judgment of lesions within overlapping leaves was inaccurate.

[0155] Other models, such as OCRNet, DeepLabV3+, and DAnet, failed to effectively distinguish between leaf edges and overlapping areas, primarily due to insufficient feature extraction of overlapping leaf edges of similar colors. In the segmentation results for plum red spot disease (C), despite the complex background and similar colors, most of the comparison models were able to accurately locate and segment the diseased spots. However, in the white frame area, models such as OCRNet, DeepLabV3+, and HR-NetV2 over-segmented the edges.

[0156] In summary, in the real environments shown in Figures (A), (B), and (C), with complex backgrounds, similar colors, and disease spots of varying sizes, the Disease-Seg model demonstrates significant advantages over other comparison models. It can accurately locate the target area and achieve fine segmentation of disease spots, significantly reducing the problems of missing target areas and loss of edge details, fully demonstrating the superiority and practical application potential of this method.

[0157] exist Figure 9 In the segmentation results of grape white rot (D) and pear scab (E) lesions shown in the figure, the leaves are affected by multiple factors, such as folding, jagged edges, irregular morphology, and background interference, resulting in certain deficiencies in the segmentation accuracy of existing deep learning models.

[0158] Taking the segmentation results of grape white rot (D) as an example, despite good lighting conditions, models such as DeepLabV3+, OCRNet, SegFormer, and DAnet exhibited certain deficiencies in the overall segmentation of leaf contours. This was primarily due to the complex and folded leaf morphology, as well as the influence of background noise, which prevented accurate segmentation. In particular, U-Net failed to effectively suppress background noise in the notched areas of the leaves, resulting in incomplete segmentation of the center. Furthermore, HRNetV2 failed to accurately depict the jagged edges of the grape leaves.

[0159] When it comes to lesion segmentation, most models miss lesions due to their irregular morphology and dense distribution. This is particularly true for the white boxes in the grape white rot (D) segmentation results. For example, models like DeepLabV3+, U-Net, and OCRNet all failed to effectively identify all lesions.

[0160] While SegNext did not miss disease spots, its detailed depiction of lesion morphology was far inferior to that of the Disease-Seg model. Similarly, in the segmentation results for pear scab (E), the sharp edges of the pear leaves posed a significant challenge for accurate leaf segmentation. Comparison models such as OCRNet, DAnet, and U-Net failed to effectively extract the details of the pear leaf edges, resulting in blurred segmentation boundaries.

[0161] Furthermore, the small disease spots within the white frame are not accurately extracted by most models, particularly DeepLabV3+ and HRNetV2, which fail to recognize these local features. In contrast, the Disease-Seg model is able to accurately extract these subtle disease features. This is due to its single-layer parallel fusion architecture, which effectively extracts global contextual features and integrates local details, thereby achieving precise segmentation of leaves and disease spots. In summary, the Disease-Seg model demonstrates superiority in handling complex leaf morphology and disease spot distribution, particularly in capturing details and suppressing complex backgrounds, significantly improving segmentation accuracy.

[0162] exist Figure 10The figure shows the segmentation results of mango brown spot (F) and pomegranate cercospora spot (G) in a laboratory setting. The main challenges come from noise interference in the image and the irregular morphology of the lesions. The folded mango leaves, the small and diverse lesions, and the diverse morphology of the pomegranate leaves, with the lesions similar in color to the background, all make accurate segmentation extremely difficult.

[0163] As shown in the white box in the segmentation result for mango brown spot disease (F), models such as DeepLabV3+, U-Net, and OCRNet only extracted the differential features between the spots and the leaves, but failed to effectively extract the shape of the spots. The Disease-Seg model, on the other hand, not only accurately demarcates the pixel regions between the leaves and the spots, but also meticulously captures the morphological characteristics of the spots, demonstrating a significant advantage over the other comparison models. The white box in the segmentation result for pomegranate Cercospora spot disease (G) clearly demonstrates the differences between the various models.

[0164] Because the lesions of pomegranate cercospora leaf spot have similar characteristics to the background, models such as OCRNet, SegFormer, and U-Net failed to effectively suppress background noise, resulting in omissions of the lesion area. In particular, in the lesion area to the right of the white frame, these models failed to accurately distinguish between the lesions and the background, resulting in a large range of missed detections. DeepLabV3+ made an incorrect judgment in the upper left corner of the white frame, further affecting the segmentation results. In contrast, the Disease-Seg model can effectively suppress noise and accurately distinguish between lesions and background, significantly improving segmentation accuracy, especially in capturing details and extracting local features.

[0165] Overall, the Disease-Seg model demonstrates significant advantages in the task of segmenting disease spots in mango and pomegranate by effectively suppressing noise and accurately extracting morphological features. In particular, it exhibits higher robustness and accuracy when dealing with complex backgrounds and small disease spots.

[0166] (2) Generalization experiment

[0167] To validate the overall generalization of the model, we compared it with classic and advanced methods on the Plant Village dataset, including CNN-based methods, Transformer-based methods, and a CNN-Teensy fusion method. We also adjusted the image resolution to differ from that of the training set to verify the model's overall performance at different resolutions. This is shown in Table 4, where bold and underlined columns indicate the highest and lowest values, respectively.

[0168] The results show that the Disease-Seg model has significant advantages over other models. Specifically, HRNetV2 achieved the best overall performance among the comparison models, but its mIoU and Acc metrics were 1.96% and 0.12% lower than the Disease-Seg model, respectively. Furthermore, HRNetV2's parameter count (Params) and computational overhead (FLOPs) were 24.76M and 18.70G higher than the Disease-Seg model, respectively. U-Net and DeepLabV3+, while performing relatively well in comparison, still have room for improvement in segmentation accuracy, with mIoU and Acc being 4.65% and 0.86% lower than the Disease-Seg model, and 3.48% and 0.81% lower, respectively.

[0169] Furthermore, these two models also lag behind the Disease-Seg model in terms of parameter size and inference speed. While PSPNet performs well in terms of Params and FLOPs efficiency, this is primarily due to the scalability of its pyramid pooling module, which effectively reduces computational overhead and improves processing speed while maintaining performance through a smaller pooling scale. However, PSPNet struggles when segmenting small objects, particularly in the agricultural disease image segmentation task, where the pyramid pooling module's focus on global features limits its ability to capture the detailed features of small objects. In contrast, SegNeXt, a Transformer fusion architecture, achieves mIoU and Accuracy scores that are 28.84% and 1.16% lower than those of the Disease-Seg model, respectively.

[0170] OCRNet and UPerNet, as CNN and Transformer fusion architectures, fail to strike a balance between overall segmentation accuracy and model efficiency. Experimental results demonstrate that the Disease-Seg model effectively compensates for the loss of detailed information while maintaining a low computational cost, resulting in more accurate segmentation of small diseased targets. In summary, the Disease-Seg model performs well across multiple evaluation metrics, demonstrating its comprehensive advantages in complex disease segmentation tasks.

[0171] Table 4:

[0172]

[0173] In order to more intuitively demonstrate the superiority of the proposed method on the Plant Village dataset, the segmentation results of grape brown spot and grape black rot in the dataset are visualized and listed. Figure 11The results show a detailed comparative analysis of the segmentation visualization results of grape brown spot (H) and grape black rot (I) using different models. In the segmentation results of grape brown spot (H), as shown in the lower left corner of the white dotted box, DeepLabV3+, U-Net, and HRNetV2 failed to accurately extract the detailed features of the adhesion between the two lesions. In contrast, the Disease-Seg model can effectively compensate for the loss of fine-grained information caused by the fusion of features of different resolutions. In the segmentation results of grape black rot (I), as shown in the center of the white dotted box, the Disease-Seg model shows a more refined segmentation effect in handling the shape of the lesions compared to SegFormer and SegNeXt. The Disease-Seg model performs particularly well in handling subtle leaf lesions, especially in the segmentation accuracy of the edge texture and detailed information of the subtle lesions.

[0174] (3) Attention Contrast

[0175] To verify the DM-Attention deep multi-scale attention mechanism's ability to effectively handle complex agricultural scenarios, experiments compared several currently high-performing attention mechanisms (such as SE, CBAM, COT, SK, Triplet, and Global Context) to fully demonstrate DM-Attention's superiority. As shown in Table 5, the baseline model achieved a mean Intersection Over Union (MIoU) of 81.70%, mean Detection Perception (MPA) of 87.90%, and Accuracy (ACC) of 99.09%. After introducing DM-Attention, model performance significantly improved, with mIoU reaching 85.98%, mPA increasing to 91.31%, and Accuracy reaching 99.11%.

[0176] Compared to the baseline model, the three metrics improved by 4.28%, 3.41%, and 0.02%, respectively. Among the compared models, CBAM's performance closely followed DM-Attention, with mIoU and mPA reaching 84.52% and 88.83%, respectively, demonstrating its superior feature modeling capabilities. However, SE-Attention's performance was slightly inferior to CBAM, with mIoU and mPA reaching 82.57% and 87.57%, respectively. In contrast, COT, SK, and Triplet performed poorly, demonstrating their respective shortcomings in modeling complex features. Triplet achieved the lowest mIoU and mPA, at only 69.39% and 76.87%, respectively.

[0177] Global Context performed the worst, with mIoU and mPA of 67.51% and 75.46%, respectively. This may be due to its overemphasis on global features, resulting in insufficient capture of local information. Comprehensive comparative results demonstrate that DM-Attention has significant performance advantages in complex agricultural scenarios. Its outstanding performance not only validates the method's effectiveness in modeling complex features but also demonstrates its outstanding ability to handle the interaction between local and global features, providing strong support for agricultural disease detection and segmentation tasks.

[0178] Table 5:

[0179]

[0180] To further verify the effectiveness of the proposed DM-Attention, the Grad-CAM method was used to compare and analyze the segmentation performance of DM-Attention with other classic and efficient attention mechanisms from the perspective of the effectiveness of the region of interest. Specifically, Figures (a) to (f) show the visualized heat maps of the single attention regions of six fruit leaves and seven lesions.

[0181] Figure 12 (a)-(f) show heatmap visualizations of diseased leaves from six fruit varieties. Figures (a) and (b) illustrate the diseased attention regions on pomegranate and mango leaves. Compared to SE, SK, and Triplet methods, DM-Attention effectively focuses on the entire leaf outline and clearly distinguishes the lesion from the leaf's edge. Figure (b) compares GlobalContext, CBAM, and CoT methods, demonstrating that DM-Attention accurately distinguishes features similar to those in the background and lesion regions, precisely focusing on the target diseased region. Figures (c) and (d) demonstrate disease recognition on plum and pear leaves in complex outdoor environments.

[0182] Due to factors such as occlusion and background clutter, the edges of the leaves are not clear. Compared with SE, SK, Triplet, and GlobalContext, DM-Attention can accurately capture the edge features of the leaves. Although CBAM can also notice the edges of the leaves, it fails to effectively distinguish the critical area between the lesions and the leaf edges, resulting in poor recognition of the target area. Figure (e) shows the lesion area of ​​a grape leaf, where the leaf is folded and the edges are jagged, and the background is relatively complex. Both CBAM and DM-Attention can effectively focus on the target area, but DM-Attention shows stronger robustness when dealing with complex backgrounds.

[0183] Figure (f) shows an apple leaf affected by both half-leaf drop disease and rust. Compared to other attention mechanisms, DM-Attention more accurately focuses on the entire leaf's feature area and effectively distinguishes between the two diseased areas, significantly outperforming other comparison methods. Further analysis shows that the multiple depthwise separable convolutions stacked within the DM-Attention module play a significant role in enhancing the model's feature extraction capabilities. Depthwise separable convolutions effectively reduce computational complexity while preserving rich detail information, improving the model's ability to distinguish between different diseased areas.

[0184] Results show that DM-Attention enhances the model's local feature extraction capabilities through the introduction of depthwise separable convolutions. It also restores inter-channel connections through point-by-point convolutions, further improving the expressive power of feature representation. Furthermore, by utilizing spatial dimensionality reduction, convolutions are used to fuse features at different scales, enhancing the model's robustness against complex backgrounds and occlusions. By introducing a weighted mechanism for convolutional outputs, the model can more accurately capture the boundaries between lesions and leaves, avoiding the ambiguity inherent in traditional convolutional methods when segmenting diseased areas.

[0185] like Figure 13 (a)-(f) show the visualized heat maps of seven disease spots. These images clearly show the performance differences of different attention mechanisms in disease area segmentation.

[0186] (a)-(f) represent heat maps of disease spots on pomegranate, mango, plum, pear, grape, and apple, respectively. (a) and (b) show samples collected in a laboratory setting. Due to the similar color of the disease spots and the background, the edge features are blurred. Traditional attention mechanisms such as SE, SK, CoT, and Triplet fail to focus on the target disease spots in this situation, being distracted by the background. While CBAM can accurately locate the disease spots, its ability to perceive edge details is still insufficient. In contrast, the DM-Attention proposed in this paper not only accurately captures the spatial location of the disease spots but also effectively extracts their edge features, thereby improving segmentation accuracy.

[0187] In (c), (d), and (e), the samples were collected in a real outdoor environment, significantly affected by varying lighting and climatic conditions. Under these conditions, Global Context did not clearly focus on the characteristics of plum red spot disease, performing poorly compared to other methods. CoT and Global Context failed to effectively address external interference, while SE and SK failed to focus on the small diseased spots on pear leaves due to insufficient local information perception. Furthermore, the diseased spots on grape leaves are easily distracted by background noise. In this case, CoT, Triplet, and Global Context overly focus on non-target areas, resulting in suboptimal segmentation results.

[0188] DM-Attention demonstrates significant advantages in handling complex backgrounds and illumination changes. It can not only accurately locate target disease spots, but also effectively distinguish between target and non-target areas. Figure (f) shows the coexistence of rust and leaf spot disease on apple leaves, which places higher demands on the model's multi-category pixel segmentation capabilities. (1) In the rust area, Triplet fails to effectively focus on the target disease spots, while CoT, Global Context, and SK are significantly distracted by the background and deviate significantly from the target area. (2) In the leaf spot disease area, the DM-Attention proposed in this paper can more accurately focus on the target disease area than other methods, demonstrating excellent attention allocation capabilities.

[0189] In summary, DM-Attention demonstrates significant advantages in both laboratory and real-world complex outdoor environments. It effectively perceives the location of target disease spots while extracting edge features, and demonstrates greater robustness and accuracy in segmenting multi-class co-morbid diseases. Experimental results demonstrate that DM-Attention significantly outperforms traditional attention mechanisms across diverse disease types and scenarios, providing a more accurate and reliable solution for agricultural disease detection.

[0190] (4) Ablation experiment

[0191] Six sets of ablation experiments were designed to verify the effectiveness of the proposed Disease-Seg model. The experiment used a control variable method to gradually introduce EFM, DM-Attention, and FWFM modules to explore their effects on the adaptability of the Transformer architecture and the optimization of detailed features. The specific experimental results are shown in Table 6. Test1 is the baseline model. After the introduction of DM-Attention, the segmentation accuracy of Test2 is significantly improved compared with Test1, with mIoU, mPA, and Acc increased by 4.28%, 3.41%, and 0.02%, respectively. After the introduction of EFM in Test3, the Params and FLOPs changed slightly, but the accuracy was improved, indicating that the EFM module can effectively extract local and global context features.

[0192] Test 4 combined the EFM and DM-Attention modules to further improve segmentation accuracy, achieving increases of 6.23%, 3.97%, and 0.35% in mIoU, mPA, and Acc, respectively, compared to the baseline model. This result demonstrates the critical role of expanding the local receptive field and fusing local and global features in addressing fruit tree leaf diseases in complex agricultural scenarios. Test 5, building on Test 3, also introduced the FWFM module, effectively mitigating feature loss through a weighted fusion strategy, improving model performance.

[0193] Test6 is the final architecture of Disease-Seg, which improves mIoU, mPA, and Acc by 8.62%, 5.75%, and 0.43% respectively compared to the baseline model, while increasing Params and FLOPs by only 1.06M and 2.59G. In summary, Disease-Seg, by integrating CNN and Transformer architectures, successfully addresses the problem of fruit tree leaf diseases in complex agricultural environments. It not only achieves high segmentation accuracy but also strikes a good balance between parameter count and computational complexity. Experimental results validate the effectiveness of the proposed model and demonstrate that the complete model significantly improves both accuracy and performance compared to using a single module.

[0194] Table 6:

[0195]

[0196] (5) Deployment experiment

[0197] To verify the practical application of the proposed model, we conducted inference time evaluation on the resource-constrained mobile device Jetson Nano and compared Disease-Seg with other models. Specific data is shown in Table 7. The results show that the proposed Disease-Seg demonstrates faster inference speed on the Jetson Nano platform. Although PSPNet performs the fastest in inference speed, its accuracy is relatively low.

[0198] It should be noted that although models such as DeepLabV3+, U-Net, DANet, PSPNet, and UPerNet have performed well in many semantic segmentation tasks, their model structures are relatively complex and require high computation and storage, resulting in slower inference speeds on mobile platforms. In contrast, Disease-Seg optimizes computation and storage requirements, allowing it to run efficiently on platforms with lower computational power, such as the Jetson Nano.

[0199] In summary, the combination of preprocessing and data augmentation successfully created a dataset suitable for training the Disease-Seg model and verified the model's superiority in semantic segmentation tasks. Disease-Seg not only surpassed other models in accuracy but also enabled real-time inference on mobile platforms with limited computing power, significantly facilitating practical applications.

[0200] Table 7:

[0201]

[0202] (6) Disease severity assessment experiment

[0203] In order to more clearly illustrate the process of disease severity assessment, five fruit leaves with different severity levels are listed, e.g. Figure 14 As shown in Figure 1, disease coverage is a key indicator for measuring the extent of disease spread. It can intuitively reflect the spread of the disease and provide an effective basis for assessing the severity of the disease. The application of this indicator helps growers to accurately manage and control the disease. The calculation method of the disease ratio is as follows:

[0204] ;in, represents the disease coverage rate, Indicates the area of ​​the lesion on the fruit leaf, It represents the area of ​​non-lesional parts of fruit leaves.

[0205] In order to test the practical applicability of the Disease-Seg model in fruit leaf disease segmentation, a fruit leaf disease segmentation system was established. The main interface of the system is as follows: Figure 15 shown.

[0206] In the present invention, the main challenge is how to efficiently overcome the various leaf and spot features and occlusion overlap problems in different environments to achieve accurate segmentation to meet the actual needs of agricultural production. The proposed Disease-Seg model compares the mainstream methods of various architectures and shows the best performance in the segmentation and diagnosis of fruit tree leaf diseases. Therefore, the Disease-Seg model can provide an effective solution for fruit tree leaves and disease spots in different scenarios. At the same time, the FPS of the Disease-Seg model reached 69, which shows that the model can process images quickly. At the same time, when tested in edge devices, the inference speed of 49ms meets the requirements of real-time processing while maintaining accuracy.

Claims

1. A real-time detection method for the severity of disease on leaves of various fruits, characterized in that: The method comprises the following steps: S1. Data acquisition: Collect images of various fruit leaf lesions and perform preprocessing; S2. Constructing a dataset: Dividing the pre-processed fruit leaf lesion images into a training set, a validation set, and a test set; S3. Model construction: Construct a real-time leaf lesion location detection model. The structure of the real-time leaf lesion location detection model is specifically composed of two parts: an encoder and a decoder. The encoder part is constructed mainly with a Transformer module, and an expansion feature module is provided in the feature extraction stage, and a feature weighted fusion module is provided in the feature fusion stage; the decoder part is constructed mainly with a multilayer perceptron; The specific structure of the encoder is as follows: after the feature map is input, the data is divided into two paths, one path passes through the expanded feature module, and the other path passes through the overlapping patch embedding module and the Transformer module in sequence. The two feature maps generated after the two paths of data processing are both input into the feature weighted fusion module for feature fusion. The fused features pass through three Transformer modules in sequence and are then input into the decoder. The specific structure of the feature expansion module is as follows: after the feature map is input, it first passes through the first layer 1×1 convolution module to generate a new feature channel, then passes through the hollow convolution module to expand the receptive field, and then passes through the second layer 1×1 convolution module to compress and integrate the information to output the feature map containing the disease, and then the input feature map and the feature map containing the disease are fused. A SiLU activation function layer and a batch normalization layer are introduced between the first layer 1×1 convolution module and the dilated convolution module, and a SiLU activation function layer and a batch normalization layer are introduced between the dilated convolution module and the second layer 1×1 convolution module; The specific structure of the Transformer module is as follows: after the feature map is input, it first passes through the deep multi-scale attention mechanism module to extract contextual semantic features and local features; then passes through the Mix-FFN module to enhance the feature expression capability of the Transformer module; finally, it enters the patch merging layer to downsample the input feature map, while gradually expanding the receptive field and integrating multi-scale information; The specific structure of the deep multi-scale attention mechanism module is as follows: the input feature map first passes through the multi-head attention mechanism module to capture the texture and morphological features of different diseases and output output features with global context; then it enters the deep multi-scale module, which convolves the features with convolution kernels of different sizes to capture local features at different scales; S4. Model training: Use the constructed data set to train the real-time detection model for leaf lesion locations and adjust the model parameters until the model meets the detection requirements; S5. Use the trained leaf lesion location real-time detection model to perform real-time detection of lesion locations on the fruit leaves to be detected, and use the proportion of the area of ​​the lesion location on the fruit leaves to the total area of ​​the fruit leaves to estimate the severity of the disease on the fruit leaves.

2. The method for real-time detection of disease severity of various fruit leaves according to claim 1, characterized in that: The preprocessing specifically includes: uniformly adjusting the fruit leaf lesion images to the same size and manually marking the categories; and performing random data enhancement operations on the fruit leaf lesion images.

3. The real-time detection method for the severity of disease on leaves of various fruits according to claim 2, characterized in that: The data processing performed in the multi-head attention mechanism module is specifically as follows: Generate query matrix through linear transformation , key matrix Sum Matrix ,pass Get attention weight ,in, express function, Represents the dimension of the attention head; then through Get output features with global context ,in, represents the random deletion of neurons. Indicates that the result is projected back to the original dimension through the linear layer. represents the transpose operation, Indicates the batch size, Represents a sequence, Indicates the number of channels, express The transposed matrix of .

4. The method for real-time detection of disease severity of various fruit leaves according to claim 3, characterized in that: The data processing performed in the deep multi-scale module is specifically as follows: Use convolution kernels of different sizes to align input features Perform convolution, and the output results of multiple convolution operations are aggregated into a multi-scale convolution list , ,in, Denotes the convolution and size is When the input feature After convolution, add the output of all convolution kernels to get the aggregate feature , aggregate features Compressed into a vector through global average pooling , map S to a smaller dimension through a fully connected layer , through another fully connected layer Mapped to The same dimension, thus generating a weight matrix , using the weight matrix Each weight in performs a weighted combination of the features output by convolution kernels of different sizes to obtain a combined feature map, which is then output after passing through a fully connected layer.

5. The real-time detection method for the severity of diseases of various fruit leaves according to claim 4, characterized in that: The data processing performed in the feature weighted fusion module is specifically as follows: Two input feature maps and The joint feature map is spliced ​​in the channel dimension to generate a joint feature map, and the joint feature map is processed by 1×1 convolution operation to generate a weighted weight map. The generated weighted weight map is processed by After the nonlinear transformation of the activation function, ensure that the weight value is between 0 and 1, and obtain the weighted weight map ; The feature map is weighted summed and Fusion is performed to obtain fusion features , ,in, Represents the batch normalization operation, and then the fusion features After performing 1×1 convolution and batch normalization operations in sequence, the final output feature map is obtained.

6. The method for real-time detection of disease severity of various fruit leaves according to claim 5, characterized in that: The decoder consists of two multi-layer perceptron modules connected in sequence.

Images