Tobacco disease segmentation method based on improved double attention mechanism

Abstract

The present application relates to the field of image processing, and proposes a tobacco disease segmentation method based on an improved double attention mechanism to improve the detection accuracy of small target lesions. In view of the problem that the traditional semantic segmentation method is difficult to identify small lesions, the present application introduces cross-feature map attention and double-branch attention mechanism in the segmentation network to enhance the attention degree and feature expression ability of small target regions. In addition, in order to solve the problem of tobacco disease data scarcity, an image splicing strategy is used to construct a training dataset, and experiments are carried out on various baseline networks. The results show that the present application can improve the mIoU by 4.75%, while the computing overhead increases by only 15.07%, which can effectively preserve image details while improving the accuracy of disease identification, and provides an efficient and reliable solution for tobacco disease detection.

Description

Technical Field

[0001] This invention is a method for tobacco leaf disease segmentation based on an improved dual-attention mechanism. Specifically, it uses the UNet semantic segmentation network and a modified UNet network to identify tobacco leaf lesion images in a field environment, obtaining the lesion type and leaf surface area ratio. This invention belongs to the field of smart agriculture technology. Background Technology

[0002] Tobacco has extremely high economic value, but it is often attacked by various diseases during the field period, resulting in a significant decline in its yield and quality. Traditional disease diagnosis mainly relies on human experience, which is not only inefficient, but also often too crude in its control measures. For example, farmers usually use large-scale spraying of pesticides for treatment, which not only causes pesticide abuse, but also leads to serious ecological pollution[1]. In contrast, the application of computer vision technology can more quickly and accurately identify tobacco diseases, providing scientific basis and decision support for disease control.

[0003] In the past, most of the methods for solving the problem of tobacco leaf disease identification were image classification or target detection, and certain results were achieved. In terms of image classification, Lin et al. [2] proposed CAMFF-Net, which designed a multi-feature fusion (MFF) module to reduce the loss of rich feature information in the model, and achieved an accuracy of 89.71% in the field tobacco disease classification task; SUN et al. [3] used ResNet-101 to classify the occurrence of tobacco aphids in images, and combined data augmentation technology and a 3-level classification method to improve the accuracy of tobacco aphid identification.

[0004] Sladojevic[4] proposed using deep convolutional neural networks (CNN) to classify plant leaf images and identify plant diseases. The model achieved an accuracy of 91% to 98% in individual class tests, with an average of 96.3%. Bharali et al.[5] used a small neural network, trained on a small dataset of only 1400 images, and achieved efficient modeling with the help of Keras and TensorFlow. They still achieved a high classification accuracy of 96.6% under limited data conditions. Lin et al.[6] used few-shot learning (FSL) to identify tobacco leaf anomalies. Through instance embedding and task adaptation, the accuracy reached 66.04% in the 5-way, 1-shot task of PlantVillage, and 45.5% and 56.5% in two settings of the TLA dataset, respectively.

[0005] In terms of target detection, Lin et al. [7] proposed YOLOX-Tiny, which is an improved detection network. By introducing hierarchical hybrid scale units (HMU) in the neck network, information interaction and feature refinement between channels are realized. It achieved an average accuracy of 80.56% in the detection task of tobacco brown spot disease under natural conditions. Zhang et al. [8] used deep residual networks to replace VGG16 for feature extraction. At the same time, k-means clustering was used to optimize the candidate boxes for detecting healthy tomato leaves and four diseases, which improved the identification accuracy of disease detection by 2.71%.

[0006] However, from an image processing perspective, neither image classification nor object detection techniques can achieve fine-grained identification of multiple types of lesions on the same leaf. Therefore, this paper employs a semantic segmentation method to identify tobacco diseases.

[0007] Semantic segmentation technology classifies each pixel in tobacco leaf images using a deep neural network model, and can accurately divide different types of disease areas. Compared with image classification and object detection, semantic segmentation technology can not only identify tobacco leaf disease types, but also accurately mark the spatial distribution of lesions, providing more granular recognition results. In addition, by classifying tobacco leaf disease areas at the pixel level, semantic segmentation technology can also distinguish between healthy tobacco leaf areas and lesion areas at the same time, providing more effective data support for disease diagnosis. For example, Chen et al. [9] proposed the MD-UNet algorithm, which innovatively designed a multi-scale convolution module (MC) and a dense residual dilated convolution module (MVCR), providing a theoretical basis and technical support for the accurate segmentation of tobacco leaf lesions and other plant diseases. Zhang et al.

[10] proposed two efficient and accurate segmentation methods to address the problems of overlapping occlusion and edge blurring in tobacco leaf segmentation during the bushing stage. By introducing a feature fusion layer and a hybrid attention mechanism, they improved the segmentation algorithm based on the Mask R-CNN model, which improved the MIoU index by 11.10%. Ou et al.

[11] proposed a semantic segmentation network for tobacco diseases that combines convolutional block attention modules (CBAM) and skip connections. This network effectively enhances feature extraction and fusion capabilities and achieves an average crossover union ratio (mIoU) of 64.99% in the tobacco disease segmentation task.

[0008] This study focuses on tobacco leaf lesions, which are all small target objects. Therefore, this paper proposes a method that simultaneously introduces two attention mechanisms, cross-feature map attention and bi-branch attention, to improve the segmentation effect of small target lesions. Summary of the Invention

[0009] The technical problem to be solved by this invention is:

[0010] 1) Complex background interference: In natural field environments, tobacco leaf lesions usually have similar colors and textures to healthy leaves, soil, weeds and other backgrounds, making it difficult for traditional segmentation methods to accurately distinguish lesion areas.

[0011] 2) Difficulty in identifying small target lesions: The lesion area often occupies a very small proportion on the surface of tobacco leaves. Traditional UNet and its variant networks are not capable of detecting small target lesions, which can easily lead to missed or false detection of small lesions.

[0012] 3) Limited feature representation capability: Traditional UNet mainly relies on encoder-decoder structure for feature extraction and reconstruction, lacks effective modeling of local and global information, and is difficult to accurately depict lesion areas of different shapes.

[0013] This invention focuses on a method for segmenting tobacco diseases based on an improved dual-attention mechanism. The objective of this invention is achieved as follows:

[0014] 1) Step 1: Collect the dataset.

[0015] The dataset used in this paper comes from the field environment in Dali, Qujing and Honghe, Yunnan Province, and all images were taken on-site.

[0016] The training set contains 100 images of frog eye disease, 70 images of climate spot disease, and 68 images of wildfire disease, for a total of 238 images. 2) Step 2: Data preprocessing.

[0017] Due to the limited amount of data, this study used image stitching to expand the dataset. Each time, one 512×512 pixel image of each of the three types of lesions was randomly selected and stitched together to form a 512×1536 pixel image, with a random seed set to ensure data consistency. Finally, the training set, validation set, and test set were divided into three sets in an 8:1:1 ratio, containing 8000, 1000, and 1000 images respectively.

[0018] 3) Step 3: Select a baseline network.

[0019] This study selected three classic semantic segmentation networks—UNet, DeepLab, and HRNet—as baseline networks. The network was trained on these three baseline networks, and the best-performing UNet baseline network was selected for further improvement.

[0020] 4) Step 4: Introduce attention mechanism.

[0021] We introduced two attention mechanisms, AFMA and DA, into the UNet baseline network with the best performance, and designed ablation experiments to further verify the effectiveness of the module. We improved the performance through quantitative index analysis.

[0022] 5) Step 5: Testing and evaluation.

[0023] The stitched images were tested using training models with different module combinations, and the effectiveness of the model improvement was analyzed by comparing the visualization results.

[0024] Compared with the prior art, the advantages of the present invention are as follows:

[0025] 1) Image stitching training strategy: This study proposes a data augmentation method based on stitching. By randomly stitching together images of tobacco leaf lesions of different categories, the method improves the pixel imbalance problem between existing data classes and simulates the situation of multiple diseases occurring at the same time as much as possible, thus providing effective data support for tobacco leaf disease identification.

[0026] 2) Dual Attention Mechanism Design: This study designs a hierarchical feature enhancement framework, introducing two different attention mechanisms: Dual-branch Attention (DA) and Across Feature Map Attention (AFMA). The DA focuses on capturing fine-grained local features, achieving effective inter-class feature differentiation with fewer parameters; the AFMA enhances multi-scale feature representation by cascading features from different feature map layers. The improvements from these two attention mechanisms significantly enhance the segmentation performance of small target lesions.

[0027] 3) Lightweight Dual Attention Mechanism: The proposed dual attention mechanism has fewer parameters and can be integrated as a plugin into various mainstream baseline networks. The proposed combination strategy demonstrates superior performance on baseline networks such as UNet, DeepLab, and HRNet by adding fewer parameters. Attached Figure Description

[0028] Figure 1 The three types of small-scale tobacco leaf lesions are shown: (a) frog eye disease, (b) climate spot, and (c) wildfire disease.

[0029] Figure 2 It is a process of piecing together three types of lesions.

[0030] Figure 3 It is a structure that combines cross-feature map attention and bi-branch attention in UNet.

[0031] Figure 4 It is a cross-feature map attention (AFMA) module framework structure.

[0032] Figure 5 It is a two-branch attention structure.

[0033] Figure 6 This is a visualization of the test results for the 9012th image using UNet.

[0034] Figure 7 This is a visualization of the test results for the 9012th image using DeepLab.

[0035] Figure 8 This is a visualization of the test results for the 9012th image using HRNet.

[0036] Figure 9 This is a visualization of the 9014th image tested using UNet.

[0037] Figure 10 This is a visualization of the test results for the 9014th image using DeepLab.

[0038] Figure 11 This is a visualization of the test results for the 9014th image using HRNet. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0040] Step 1: Collect the dataset.

[0041] The specific steps for collecting the dataset in Step 1 are as follows:

[0042] The dataset used in this invention originates from the field environments of Dali, Qujing, and Honghe in Yunnan Province, and all images were taken on-site. The training set contains 100 images of frog eye disease, 70 images of climate spot disease, and 68 images of wildfire disease, totaling 238 images. Figure 1 These are illustrations of the three types of lesions involved in this invention.

[0043] Step 2: Data preprocessing.

[0044] The specific steps of data preprocessing in Step 2 are as follows:

[0045] Due to limited data volume, this study used image stitching to expand the dataset. Each time, one 512×512 pixel image of each of the three types of lesions was randomly selected and stitched together to form a 512×1536 pixel image (e.g., ...). Figure 2 As shown in the figure, a random seed was set to ensure data consistency. Finally, the training set, validation set, and test set were divided in an 8:1:1 ratio, containing 8000, 1000, and 1000 images respectively.

[0046] Step 3: Select a baseline network.

[0047] The specific steps for selecting the baseline network in Step 3 are as follows:

[0048] Step 3.1: Select the baseline network:

[0049] This invention selects three classic semantic segmentation networks—UNet, DeepLab, and HRNet—as baseline networks for training and splicing tobacco leaf lesion datasets. The mIoU metric is used to measure the overall performance of the segmentation networks; a higher mIoU value indicates that the model's predictions are closer to the actual situation. The formula is as follows:

[0050]

[0051] Where N is the total number of categories, A i B is the predicted segmentation region of class i. i It is the true segmentation region of class i. A i ∩B i It is the intersection of the i-th predicted region and the true region (the number of correctly predicted pixels). A i ∪B i It is the union of the i-th predicted region and the true region (total number of pixels).

[0052] Step 3.2: Use the same hardware configuration and set the same training parameters:

[0053] All experiments in this invention were conducted on an NVIDIA GeForce RTX4090 (as shown in Table 1), and the same training parameters were used as shown in Table 2 to ensure that all baseline networks had the same training environment during the experiments.

[0054] Table 1: Training Hardware Configuration

[0055] Configuration items illustrate CPU Intel(R) Xeon(R) Platinum 8336C (32 cores, 2.30GHz) GPU NVIDIA GeForce RTX4090 Memory 251GiBDRAM

[0056] Table 2: Training Hyperparameter Settings

[0057]

[0058] Step 3.3: Determine UNet as the final baseline network based on the mIoU metric:

[0059] Based on the comparative experiments in Tables 3 and 4, it can be seen that, except for a slightly lower mIoU index of 3.51% in the climate spot, the mIoU index is better than the other two baseline networks in the three categories of frog eye disease, wildfire disease and background. Therefore, UNet is determined to be the baseline network for further improvement.

[0060] Table 3: Baseline Network Performance Comparison

[0061]

[0062]

[0063] Table 4: Performance Comparison of Attention Modules

[0064] Model Frogeye (%) Weatherfleck (%) Wildfire (%) Background (%) Miou (%) UNet+AFMA+DA 41.37 26.34 47.76 99.49 53.74 DeepLab+AFMA+DA 24.08 5.99 21.44 99.15 37.67 HRNet+AFMA+DA 34.92 17.34 43.69 99.47 48.86

[0065] Step 4: Introduce an attention mechanism and perform quantitative comparison.

[0066] The frameworks of the AFMA module and the DA module are as follows: Figure 3 ,and Figure 4 As shown, the UNet framework combines two modules as follows: Figure 5 As shown.

[0067] The specific steps for introducing the attention mechanism in Step 4 are as follows:

[0068] Step 4.1: Embedding the first-level feature map. Input color image I (I∈R) H×W×3 Feature maps are extracted using convolutional layers. First, a 7×7 convolutional layer with 64 filters is used. and a 3×3 convolutional layer with 1 filter The input image I is embedded into a new feature map. This process is represented by formula (1):

[0069]

[0070] Here R img ∈R H×W×1 .

[0071] Step 4.2: Transform the feature map of layer i. Use a 1×1 convolutional layer (Conv2) to transform the feature map F of layer i. i Transform into features for each category. Specifically, generate a feature containing N c Feature maps of each category R ith , where N c It is the number of categories. The process is formalized as follows:

[0072] R ith =Conv2(F i (3)

[0073] here Represents the i-th layer feature map related to the category, each It only contains information related to the k-th category.

[0074] Step 4.2: R img The image is split into fixed-size patches. To calculate the relationships between each image patch, the original image R is first... img The process involves splitting the material into multiple fixed-size patches. This process can be represented by the following formula:

[0075] P img =φ(R) img,d)(4)

[0076] Where φ is the image segmentation operation, which segments R... img ∈R H×W×1 Reconstructed into multiple two-dimensional image blocks P of size d×d. img .

[0077] Step 4.4: Split each channel into patches. Similar to Step 3, in this stage, the feature map R of the i-th layer is... ith Each class feature map in It is also split into multiple two-dimensional patches. Specifically, this is expressed in formulas (4) and (5):

[0078]

[0079] here, Let || denote the two-dimensional patch sequence of the k-th category, and || denote the connection.

[0080] Step 4.5: Calculate the relationship between image patches and feature map patches. In this stage, each patch P of the original image... img Patch of feature maps associated with each category in layer i The dot product between them is calculated to obtain the relationship between them. This is specifically expressed in formulas (6) and (7):

[0081]

[0082] in This represents the relationship between the original image patch and the k-th class feature map patch. It is a patch of the feature map for each category, A ith The relationship between the two is established by calculating the dot product.

[0083] Step 4.6: Adjust the size using pooling. Use average pooling to adjust the size of the predicted mask M output by the decoder. mask Compressed to R ith The same size. Specifically, this is represented by formulas (9) and (10):

[0084]

[0085] here ψ(input,k H ,k W ,s H ,s W ) represents average pooling, with a kernel size of (k H ,k W ), step size is (s) H ,sW ).

[0086] Step 4.7: Divide the mask feature map into patches. Similar to steps 3 and 4, in this stage, we will... mask Each channel is split into flat two-dimensional patches. This is represented by formulas (12) and (13):

[0087]

[0088] here

[0089] Step 4.8: Apply the attention map. In this stage, attention map A is applied... ith Application to mask P mask The above is used for spatial modulation, thereby enhancing the segmentation effect on small objects. The formula is as follows:

[0090] This represents the mask generated after spatial modulation.

[0091] Step 4.9: Fold to the final size. The resulting... Fold into With the same size, the final prediction result is obtained:

[0092]

[0093] Here, O ith This represents the modulated output of the i-th layer feature map, and the final output is represented as Pre:

[0094] Pre=M mask +O ith (17)

[0095] here

[0096] Step 4.10: Calculate the AFMA (Average Downward Monetary Price) for gold. It is based on ground truth (gt), where each value represents the golden ratio between a block in the original image and a block in the feature map. During training, to further supervise the quality of the attention map, a golden AFMA (A gt The calculation process is shown in formulas (19)(20)(21):

[0097]

[0098] By calculating the relationship between each image patch and the feature maps associated with each category, the AFMA plugin can effectively improve the segmentation performance of small objects.

[0099] Step 4.11: The dual-branch attention module feeds the same input feature maps into two feature extraction layers: a positional feature extraction block (PAM) and a channel feature extraction block (CAM). Finally, the two different features are fused, as shown in the specific structure below. Figure 3 As shown in the figure. This paper uses dual-branch attention as a feature extraction module to enhance the feature representation ability of the encoder, thereby improving the performance of the segmentation network.

[0100] The dual attention module consists of two parts: the first part primarily uses PAM (Portable Attention) and the second part primarily uses CAM (Portable Attention Module). The first component takes the input features, performs a single convolution, and then performs a dimensionality-reducing convolution on the input features, scaling the number of channels by 1 / 16 to obtain the simplified feature α. 1 and α 2 These serve as inputs for PAM and CAM, respectively.

[0101] • After PAM feature extraction, convolution is performed to obtain α. 1 :

[0102] α 1 =Conv(input)(21)

[0103]

[0104] The component CAM is the same; replace PAM with CAM in formula (27), as shown in the following formula:

[0105] α 2 =Conv(input)(23)

[0106]

[0107] • Extracting α from two layers of attention 1 and α 2 Then, by aggregating and summing the two attention layers, and restoring the number of channels in a single convolution, the output is obtained:

[0108]

[0109] DA-Block seamlessly integrates the ability to extract positional and channel features, providing the model with a deep fusion of global and local semantic information.

[0110] Step 5: Testing and evaluation.

[0111] The specific steps for testing and evaluation in Step 5 are as follows:

[0112] The stitched images were tested using training models with different module combinations, and the effectiveness of the model improvement was analyzed by comparing the visualization results.

[0113] Step 5.1: AFMA Module Effectiveness Analysis. To verify the effectiveness of the AFMA module in the fusion attention, we designed a performance comparison based on a tobacco disease dataset after applying two modules to the baseline networks respectively. As shown in Table 5, the model performance varied after applying the AFMA module to the three baseline networks: DeepLab, HRNet, and UNet. On the DeepLab baseline network, the AFMA module improved the IoU of frog eye disease, climate spot, and wildfire disease by 3.08%, 2.09%, and 20.77%, respectively, and the mIoU by 6.49%. On the HRNet baseline network, the AFMA module improved the IoU of climate spot by 1.61% and the mIoU by 0.12%. For the UNet network, the AFMA module slightly decreased the IoU of frog eye disease and wildfire disease, resulting in a 0.58% decrease in mIoU, but improved the IoU of climate spot by 3.68%, indicating that the AFMA module improved the recognition accuracy of climate spot. Furthermore, in synergy with the DA module, the UNet baseline network ultimately achieved the highest IoU and the highest mIoU for a single lesion class, further validating the effectiveness of the AFMA module.

[0114] Table 5: AFMA Module Effectiveness Experiment

[0115]

[0116]

[0117] Step 5.2: DA Module Effectiveness Analysis. To verify the effectiveness of the DA module in the fused attention model, we designed a performance comparison based on a tobacco disease dataset after applying two modules to the baseline networks respectively. Table 6 shows the performance comparison after applying the DA module to the three baseline networks. In the UNet baseline network, after introducing the DA module, although the mIoU decreased by 1.14% and the IoU of wildfire disease decreased by 30.11%, the IoU of frog eye disease increased by 3.23%, and the IoU of climate spot increased by 22.14%. This indicates that the DA module has a gain effect on the identification of these two types of lesions and provides the possibility for its synergistic improvement with AFMA to improve the final identification accuracy. After the synergistic effect, the IoU and mIoU of the four types of lesions are significantly better than the baseline network and the network applying a single attention module. In the DeepLab baseline network, the application of the DA module increased the IoU of frog eye disease by 9.82%, the IoU of wildfire disease by 11.76%, and the IoU of climate spot decreased slightly by 0.24%, with an overall mIoU improvement of 5.33%. In the HRNet baseline network, the DA module slightly decreased mIoU by 0.19%, climate spot IoU by 1.32%, but improved wildfire IoU by 0.98%. Nevertheless, the synergistic effect of the dual attention modules resulted in a higher final mIoU than the baseline network. In contrast, on the UNet baseline network, the synergistic effect of the dual attention modules had a more significant impact on the network's recognition performance.

[0118] Table 6: DA Module Effectiveness Experiment

[0119] Model Frog eye disease (%) Climate spots (%) Wildfire disease (%) background(%) Miou (%) UNet 38.60 14.11 43.77 99.49 48.99 UNet+DA 41.83 36.25 13.66 99.66 47.85 DeepLab 25.97 2.34 18.43 99.32 36.52 DeepLab+DA 35.79 2.10 30.19 99.36 41.85 HRNet 33.82 17.62 43.24 99.41 48.52 HRNet+DA 33.37 16.30 44.22 99.42 48.33

[0120] Step 5.3: Discussion on the Application Depth of AFMA. The AFMA module models the relationship between small and large targets by calculating the cross-correlation between the original image patch and the feature map patch, thus compensating for information loss during feature propagation. To verify the effectiveness of different feature layers in reconstructing the relationship between small and large targets, we conducted a comparative experiment of AFMA on different feature layers with the DA module applied four times. According to the results in Table 7: 1. Applying the AFMA module at different depths can improve the performance of the baseline segmentation model, indicating that utilizing the relationship between the original image and the feature map helps to steadily improve the segmentation effect. 2. The IoU of the three types of lesions—frog eye disease, climate spots, and wildfire disease—is improved at different depths, and the mIoU reaches its highest at a depth of 2. Therefore, the model with a depth of 2 is selected as the final solution.

[0121] Table 7: Application of AFMA attention in feature maps of different layers

[0122] UNet+AFMA+DA Frog eye disease (%) Climate spots (%) Wildfire disease (%) background(%) Miou (%) AFMA_depth = 1 42.95 25.09 45.48 99.48 53.25 AFMA_depth=2 41.37 26.34 47.76 99.49 53.74 AFMA_depth=3 41.19 25.92 48.24 99.49 53.71 AFMA_depth=4 43.10 24.37 47.32 99.51 53.57 AFMA_depth=5 42.06 23.84 47.76 99.48 53.28

[0123] Step 5.4: Visual Comparison. To visually demonstrate the effectiveness of the proposed AFMA+DA attention module combination, Figure 8-1 Figure 3 shows representative segmentation results of three baseline methods applied with different attention modules on the tobacco disease test set. In the prediction results, frog eye disease is visualized as green (0,255,0), climate spot as yellow (255,255,0), and wildfire disease as blue (0,0,255). We manually added rectangles above the prediction results to facilitate comparison of differences between networks, where white rectangles represent poor results and red rectangles represent good results.

[0124] Figure 8 This is a visualization of the results after testing the 9012th image using the UNet baseline network. Comparing the rectangles at the top of the image shows that neither the UNet baseline network nor the network with a single module correctly identified wildfire disease, but the combined module successfully identified the wildfire disease indicated by the rectangles. Comparing the rectangles at the bottom shows that the UNet baseline network incorrectly identified the weather spots within the rectangles as frog eye disease, while the weather spots were successfully identified regardless of whether AFMA, DA, or AFMA+DA were used.

[0125] Figure 9 This is a visualization of the results after testing the 9012th image using the DeepLab baseline network. By comparing the rectangles below, we can see that neither the DeepLab baseline network nor the network with a single module correctly identified the weather patches. However, the combination of modules successfully identified some weather patches, which is consistent with the slight improvement in IoU from 2.34% to 5.99%.

[0126] Figure 10 This is a visualization of the results after testing the 9012th image using the DeepLab baseline network. As can be seen from the area marked by the rectangle above, the baseline network HRNet identified some wildfire disease as weather spots, and whether using a single module AFMA or DA, or a combination of two modules, the wildfire disease in the box was successfully identified.

[0127] Figure 11This is a visualization of the results after testing the UNet baseline network on image 9014. As shown in the rectangles below, the UNet baseline network only identified a portion of the lesions within the cluster of climate spots, incorrectly classifying them as frog eye disease. Introducing the AFMA module significantly identified more lesions, with only one spot in the center being misclassified as frog eye disease. Both the DA module and the dual-module approach successfully identified most climate spots without misclassification. Notably, the UNet+DA model incorrectly classified all frog eye diseases as climate spots, but the dual-module combination avoided this, further demonstrating the effective synergy between the two modules.

[0128] The visualization analysis above shows that the effectiveness of dual-module collaboration is repeatedly confirmed on the three baseline networks, and UNet+AFMA+DA always achieves the best results, which is consistent with the IoU index for the three types of lesion identification.

[0129] References

[0130] [1]Martinelli, Federico, Riccardo Scalenghe, Salvatore Davino, StefanoPanno, Giuseppe Scuderi, Paolo Ruisi, Paolo Villa et al. "Advanced methods of plant disease detection.Areview." Agronomy for Sustainable Development 35, no.1 (2015): 1-25. [2]Lin J, Chen Y, Pan R, et al. CAMFFNet: A novel convolutional neural network model for tobacco disease image recognition [J]. Computers and Electronics in Agriculture, 2022, 202: 107390.

[0131] [3]SUN Jiazhao,LIQunling,LIN Xiaoxing,et al.Development of ImageRecognition System for Tobacco Aphid Based on Resnet-101 Model[J].PLANTHEALTH AND MEDICINE,2024,3(4):26-31.doi:10.13718 / j.cnki.zwyx.2024.04.004

[0132] [4]Sladojevic S,Arsenovic M,Anderla A,et al.Deep neural networksbased recognition of plant diseases by leaf image classification[J].Computational intelligence and neuroscience,2016,2016(1):3289801.

[0133] [5]Bharali P,Bhuyan C,Boruah A.Plant disease detection by leaf imageclassification using convolutional neural network[C] / / Internationalconference on information,communication and computing technology.Singapore:Springer Singapore,2019:194-205.

[0134] [6]Lin H,Qiang Z,Tse R,et al.A few-shot learning method for tobaccoabnormality identification[J].Frontiers in Plant Science,2024,15:1333236.

[0135] [7]Lin J,Yu D,Pan R,et al.Improved YOLOX-Tiny network for detectionof tobacco brown spot disease[J].Frontiers in Plant Science,2023,14:1135105.

[0136] [8]Zhang Y,Song C,Zhang D.Deep learning-based object detectionimprovement fortomato disease[J].IEEE access,2020,8:56607-56614.

[0137] [9]Chen Z,Peng Y,Yang Y,et al.MD-Unet:used for the segmentation oftobacco leaflesions[C] / / 2024 9th International Conference on IntelligentComputing and SignalProcessing(ICSP).IEEE,2024:1785-1791.

[0138]

[10] Zhang W,Wang Y,Shen G,et al.Tobacco leaf segmentation based onimproved maskRCNN Algorithm and SAM model[J].IEEE Access,2023.

[0139]

[11] Ou J L,Lin H,Qiang Z P,et al.Tobacco Leaf Disease Segmentationbased onTDSSNet[C] / / 2023 Twelfth International Conference on Image ProcessingTheory,Toolsand Applications(IPTA).IEEE,2023:1-5。

Claims

1. A method for segmenting tobacco diseases based on an improved dual-attention mechanism, characterized in that, The method specifically includes the following steps: 1) Step 1: Collect the dataset. 2) Step 2: Data preprocessing. 3) Step 3: Train the baseline network. 4) Step 4: Improve the baseline network. 5) Step 5: Testing and evaluation.

2. The method for segmenting tobacco diseases based on an improved dual-attention mechanism according to claim 1, characterized in that: The specific details of obtaining the tobacco leaf disease dataset in Step 1 are as follows: On-site shooting in a field environment, continuously photographing the same tobacco field with approximately one-month intervals between each session, helps us obtain the pathological characteristics of various lesions at each stage of the disease process, enabling better identification of lesions later. During the shooting process, we try to avoid the influence of weather; on sunny days, we use tools such as sun umbrellas to avoid strong light reflections in the images, which facilitates subsequent computer processing.

3. The method for segmenting tobacco diseases based on an improved dual-attention mechanism according to claim 1, characterized in that: In Step 2, the specific steps for preprocessing the tobacco disease dataset are as follows: Step 2.1: Data Filtering. Filter the images of tobacco leaf lesions taken with the camera, removing data that is difficult to use for training, such as those with poor focus, excessively bright or dark lighting, or severely damaged lesions. Step 2.2: Data Classification. The tobacco leaf disease data retained after data screening is classified. The types of lesions identified in this invention include frog eye disease, climate spot, and wildfire disease. Further screening is performed during classification, retaining images with obvious lesion characteristics and clear lesions. Step 2.3: Data Stitching. Before image stitching, the images are scaled down, and the resolution of the three types of lesion images is set to 512×512 pixels. Then, a random stitching method is used, randomly selecting one 512×512 pixel image from each of the three types of lesions and stitching them together to form a 512×1536 pixel image. A random seed is set to ensure data consistency. Finally, the training set, validation set, and test set are divided into training set, validation set, and test set in an 8:1:1 ratio, containing 8000, 1000, and 1000 images respectively. Step 2.4: Data Annotation. Annotation was performed using the Segment Anything tool, with frog eye disease visualized in green, climate spots in yellow, and wildfire disease in blue.

4. The method for segmenting tobacco diseases based on an improved dual-attention mechanism according to claim 1, characterized in that: In Step 3, the specific steps for training the baseline network are as follows: Step 3.1 Initialize the training environment: Set up computing devices (CUDA device visibility), and configure random seeds to ensure the reproducibility of the experiment. Determine whether to use distributed training mode, and initialize process groups in a multi-GPU environment. If using Synchronous Batch Normalization (SyncBN) or Mixed Precision Training (FP16), adjust the model parameters accordingly. Step 3.2 Construct the U-Net baseline network: Select the backbone network of the U-Net structure (such as ResNet50), and choose whether to load pre-trained weights according to requirements. If pre-trained weights are loaded, perform model weight matching and loading; for mismatched parts, use random initialization. Step 3.3 Define training hyperparameters: Set the number of classification classes, input image size, learning rate, optimizer type (e.g., Adam or SGD), weight decay, and learning rate decay strategy (e.g., cosine decay). Set hyperparameters for both frozen and unfrozen training phases, including initial learning rate, batch size, and number of training epochs. Step 3.4 Dataset Loading and Processing: Read the training and validation sets and perform data preprocessing according to U-Net training requirements, such as normalization, data augmentation, and batch processing. If distributed training is used, set a distributed sampling strategy to ensure that the data is evenly distributed across different GPUs. Step 3.5 Optimizer and Learning Rate Scheduling: Initialize the model optimizer according to the set optimizer type and set a dynamic learning rate adjustment strategy. Adaptively adjust the learning rate based on the current batch size to ensure optimization stability. Step 3.6 Training Process: A frozen training strategy is adopted. In the first few training rounds, only the parts of the U-Net network outside the backbone are optimized to prevent weight perturbation. During training, all network parameters are unfrozen according to the set number of rounds, and the batch size and learning rate are adjusted to allow the model to fully learn the data features. When using mixed precision training (FP16), gradient scaling is used to prevent numerical overflow. Loss functions (such as cross-entropy loss, Dice Loss, Focal Loss) are calculated, and forward and backpropagation are performed to optimize the parameters. Step 3.7 Model Evaluation and Saving: Evaluate the model performance on the validation set at a specified number of training epochs, and calculate splitting metrics such as IoU and mIoU. Save the model weights at set intervals, and record information such as training loss and learning rate changes to support training process backtracking. After training is complete, close the logger and store the final trained U-Net model weights.

5. The method for segmenting tobacco diseases based on an improved dual-attention mechanism according to claim 1, characterized in that: In Step 4, the specific steps for improving the baseline network are as follows: Step 4.1 Obtain the encoder: Select ResNet50 as the encoder and introduce AFMA (Adaptive Feature Attention Module) and DA (Dual Attention Module) to enhance feature extraction capabilities. Step 4.2 Initialize the encoder: Set encoder_name to "resnet50", encoder_depth to 5, encoder_weights to "imagenet", and att_depth to 3 to control the depth of the attention module. Call the get_encoder method to load the encoder, with the input channel number in_channels set to 3 (suitable for RGB images), and output feature maps and attention information at multiple levels. The encoder uses ResNet50 and combines AFMA and DA for feature enhancement to improve segmentation accuracy. Step 4.3 Decoder Construction: UnetDecoder is used for decoding, with decoder_channels set to (256, 128, 64, 32, 16). The SCSE mechanism is used for attention enhancement. The `center = True if encoder_name.startswith("vgg")else False` setting ensures compatibility with the VGG network structure. Additional attention parameters are passed to the decoder structure for feature reconstruction, utilizing AFMA and DA mechanisms to improve segmentation accuracy. The decoder employs the SCSE attention mechanism and introduces an attention depth parameter `att_depth = 3` to optimize feature reconstruction. Step 4.4 Initialize the model: Initialize the decoder weights using `init.initialize_decoder(self.decoder)`. If a classification task is involved, define an additional `ClassificationHead` for classification task initialization. Step 4.5 Forward Propagation: After inputting the image x, the encoder first extracts features and attentions, which are then passed to the decoder. The decoder combines the attentions to generate the final output for the segmentation task. The inference process `predict(x)` performs gradient-free computation using `torch.no_grad()`, improving computational efficiency. Gradient-free inference is performed using `predict(x)`, outputting the segmentation result. The segmentation result and attention feature map are obtained using `present(x)`.

Images