Method for performing phenotype analysis and severity quantification on potato tuber diseases by using double-end CNN-Transform network

Abstract

The invention discloses a method for phenotypic analysis and severity quantification of potato tuber diseases by using a double-ended CNN-Transform network, and belongs to the technical field of potato disease detection. The invention aims to realize accurate classification of potato tuber diseases. The method comprises the steps that a potato disease data set is constructed, and an Anylaging tool is used for labeling; performing data enhancement to obtain a potato disease data set after data enhancement, and dividing the potato disease data set into a training set and a test set; the method comprises the following steps: constructing a double-end hybrid CNN-Transform architecture, a CNN encoder for local feature extraction, a Transform bottleneck layer for global context modeling and a double-end output structure coupled through a novel SAP mechanism, wherein the CNN encoder, the Transform bottleneck layer and the double-end output structure are connected in sequence; and constructing a composite multi-task loss function to train the double-headed mixed CNN-Transform architecture, and carrying out automatic diagnosis and detection on the potato diseases by adopting a test set.

Description

Technical Field

[0001] This invention belongs to the field of potato disease detection technology, specifically involving a method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network. Background Technology

[0002] Accurate and interpretable detection of potato tuber diseases is crucial for postharvest quality assessment and disease management. However, existing models often rely on background correlations (“shortcut learning”) rather than true pathological features. Furthermore, current methods typically treat disease classification and lesion segmentation as independent tasks, failing to model their inherent spatial dependencies. Moreover, a common but often overlooked challenge in deploying deep learning models in agriculture remains – “shortcut learning.” Standard classifiers often rely on spurious correlations (such as background color, table texture, or lighting conditions) to infer disease categories, rather than focusing on the pathological lesions themselves. This makes the model a “black box”: performing well on specific test sets but struggling to generalize to real-world environments with varying backgrounds. In addition, most existing multi-task frameworks treat disease classification and lesion segmentation as parallel, independent data streams. This design ignores the biologically logical dependencies: disease identification should be determined entirely by the visual characteristics of infected tissue, not by the surrounding healthy epidermis or background environment. Failure to model this dependency limits the interpretability and biological effectiveness of the system. Summary of the Invention

[0003] The problem this invention aims to solve is to achieve accurate classification, lesion segmentation, and interpretable quantitative analysis of potato tuber diseases. It proposes a method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network includes the following steps:

[0006] S1. Combine independently collected potato disease images with publicly available data resources to construct a potato disease dataset, and use Anylabeling tools to label all images in the potato disease dataset;

[0007] S2. The images in the potato disease dataset obtained in step S1 are augmented using the image augmentation library alburementations to obtain the augmented potato disease dataset, which is divided into a training set and a test set.

[0008] S3. Construct a dual-head hybrid CNN-Transformer architecture, consisting of a CNN encoder for local feature extraction, a Transformer bottleneck layer for global context modeling, and a dual-head output structure coupled through a novel SAP mechanism;

[0009] S4. Construct a composite multi-task loss function, and use the training set obtained in step S2 to train the dual-head hybrid CNN-Transformer architecture constructed in step S3 to obtain the trained dual-head hybrid CNN-Transformer architecture. Use the test set to perform automatic diagnosis and detection of potato diseases.

[0010] Furthermore, the specific implementation method of step S1 includes the following steps:

[0011] S1.1. Independently collect potato disease images in the field, including images of potato diseases such as black scurf, common scab, pink rot, powdery scab and psoriasis;

[0012] S1.2. Select potato disease images from publicly available data resources, including publicly available benchmark datasets, newly created expert-annotated datasets from Heilongjiang Province, China, and independent datasets from Norway, to establish samples of different severity stages of dry rot, and divide dry rot internal type, dry rot surface type and severe dry rot type into three independent categories.

[0013] S1.3. Construct a potato disease dataset based on the images from steps S1.1 and S1.2, label the images using Anylabeling tools, and generate pixel-level disease masks for all images by drawing polygonal lesion boundaries.

[0014] Furthermore, the specific implementation method of step S2 includes the following steps:

[0015] S2.1. Perform a rigid geometric transformation on the images in the potato disease dataset obtained in step S1, setting the probability of horizontal / vertical flipping to 0.5 / 0.3, the probability of translation, scaling and rotation to 0.7, and the rotation angle to ±25°.

[0016] S2.2. Perform a non-rigid geometric transformation on the image processed in step S2.1. The probability of hitting the elastic / mesh distortion is 0.4. Set the control pixel displacement amplitude α=40 and the control spatial smoothness of the deformation σ=8.

[0017] S2.3. Perform photometric transformation on the image processed in step S2.2, set the probability of brightness / contrast to 0.6, set the limit range to ±0.2, then set the probability of limiting contrast adaptive histogram equalization to 0.3, and set the cropping limit to 3.0;

[0018] S2.4. Perform environmental simulation on the image processed in step S2.3, set the probability of shadow to 0.5, then perform idle blur processing, set the probability of Gaussian / motion / median blur to 0.4, set the convolution kernel size to 3-5, and obtain a data-augmented potato disease dataset with an image size of 256*256.

[0019] S2.5. Divide the augmented potato disease dataset into training and test sets according to a 9:1 ratio.

[0020] Furthermore, the specific implementation method of step S3 includes the following steps:

[0021] S3.1. Set up a CNN encoder for local feature extraction. Use a ResNet-34 backbone network pre-trained on ImageNet as the feature encoder. Remove the last classification layer and pooling layer. Input the potato disease image into the CNN encoder for local feature extraction to obtain the feature map output by the encoder.

[0022] S3.2. Set the Transformer bottleneck layer used for global context modeling to consist of 4 Transformer encoder modules, input the feature map output by the encoder to the Transformer bottleneck layer used for global context modeling, and obtain the feature map output at the Transformer bottleneck.

[0023] S3.3. A novel SAP mechanism-coupled dual-head output structure is set up, which includes a classification head for disease identification using softmax activation and a segmentation decoder for generating lesion masks using a deconvolution structure combined with sigmoid activation. By implementing segmentation-aware pooling hard attention constraints, the feature map is selectively gated, and an explicit structural dependency is established between the segmentation and classification tasks.

[0024] Let F represent the feature map output from the bottleneck of the Transformer, M be the predicted soft segmentation mask, and calculate the classification feature vector v_cls as follows:

[0025] v_cls = GAP(F ⊙ Norm(M))

[0026] Where ⊙ represents element-wise multiplication, Norm(M) represents normalizing the soft mask to form a spatial attention map derived from the segmentation input, and GAP is global average pooling, used to compress spatial feature maps into classification feature vectors.

[0027] Furthermore, the specific implementation method of step S4 includes the following steps:

[0028] S4.1. The network is optimized using the composite multi-task loss function L_total, with the following expression:

[0029] L_total = λ_cls · L_CE + λ_seg · (L_Dice + L_BCE)

[0030] Where L_CE represents the cross-entropy loss used in the classification task, L_BCE represents the binary cross-entropy loss, L_Dice represents the Dice loss, and the classification target weight coefficient λ_cls = 1.0 and the segmentation target weight coefficient λ_seg = 1.0 are set.

[0031] S4.2. The network training uses the Adam optimizer, with an initial learning rate set to 1e. -4 The weight decay rate is 1e -2 The training batch size is set to 4, and the validation batch size is set to 1. A decoupling strategy for learning rate adjustment and model selection is adopted: the ReduceLROnPlateau scheduler is used to dynamically adjust the learning rate based on the validation set loss.

[0032] If the training set loss does not decrease for 5 consecutive epochs, the learning rate will be decayed by a factor of 0.9.

[0033] S4.3. The selection criterion for the optimal model checkpoint is to maximize the Dice similarity coefficient of the training set. By monitoring the Dice score, if the Dice score of the training set does not improve for 40 consecutive epochs, the early stopping mechanism is triggered.

[0034] The beneficial effects of this invention are:

[0035] This invention describes a method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network, integrating the local texture extraction capabilities of CNNs with the global context modeling advantages of Transformers. To ensure that the decision-making process is strictly based on biopathological features, a segmentation-perceptual pooling (SAP) mechanism is introduced. The SAP hard attention constraint strategy explicitly gates classification features through predicted lesion masks, forcing the model to ignore environmental background. Although this strict constraint results in only a slight loss in classification accuracy (95.6% vs. 95.8%), it significantly improves the model's effectiveness and generalization potential. Furthermore, the coupled architecture establishes a supervised feedback loop, optimizing lesion localization capabilities and achieving state-of-the-art correlation in disease severity estimation (R² = 0.997). This interpretable dual-head CNN-Transformer network enables objective and reproducible quantification of disease severity, suitable for automated sorting lines, breeding phenotypic analysis, and post-harvest decision support.

[0036] This invention presents a method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network. This method effectively integrates disease classification and lesion segmentation tasks in potato tuber images, achieving a balance between high accuracy and strong interpretability. The model can accurately quantify the area of ​​lesion regions, and its predictions maintain a high correlation with actual measurements, demonstrating its robustness in automated disease phenotypic analysis. Compared to previous agricultural models based on CNN or Transformer, this framework improves both generalization ability and interpretability, laying a practical foundation for the actual deployment of precision agriculture and intelligent detection systems. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating a method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network, as described in this invention.

[0038] Figure 2 Photos of eight types of potato diseases;

[0039] Figure 3 The data augmentation strategy visualizations for the two selected potato disease categories are shown in the first row, where the comparison charts for powdery rot and common scab are shown in the second row.

[0040] Figure 4 The training process of the dual-head hybrid CNN-Transformer architecture of the present invention is shown in (a) for loss training and (b) for index training.

[0041] Figure 5 The quantitative evaluation of the baseline hybrid CNN-Transformer model on an unseen test dataset is shown, where (a) is the row-normalized confusion matrix and (b) is the scatter plot analysis.

[0042] Figure 6 Qualitative visualization comparison images of the segmentation results of eight potato diseases in this invention;

[0043] Figure 7 This is a comparative visualization diagram of the attention mechanism in the dual-head hybrid CNN-Transformer architecture of the present invention, wherein the first row is an image of internal dry rot and the second row is an image of pink rot;

[0044] Figure 8 The quantization performance of the dual-head hybrid CNN-Transformer architecture of the present invention is shown in (a) as the normalized confusion matrix of the classification accuracy of the test set, and (b) as the lesion severity estimation analysis curve.

[0045] Figure 9This is a comparative visualization diagram of the attention mechanism in the dual-head hybrid CNN-Transformer architecture of the present invention, where the first row is an image of moderate dry rot and the second row is an image of severe dry rot. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0047] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0048] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 - Appendix Figure 9 Detailed explanation is as follows:

[0049] Example 1:

[0050] A method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network includes the following steps:

[0051] S1. Combine independently collected potato disease images with publicly available data resources to construct a potato disease dataset, and use Anylabeling tools to label all images in the potato disease dataset;

[0052] Furthermore, the specific implementation method of step S1 includes the following steps:

[0053] S1.1. Independently collect potato disease images in the field, including images of potato diseases such as black scurf, common scab, pink rot, powdery scab and psoriasis;

[0054] S1.2. Select potato disease images from publicly available data resources, including publicly available benchmark datasets, newly created expert-annotated datasets from Heilongjiang Province, China, and independent datasets from Norway, to establish samples of different severity stages of dry rot, and divide dry rot internal type, dry rot surface type and severe dry rot type into three independent categories.

[0055] Furthermore, the publicly available benchmark dataset was constructed by Faria et al. in 2023; publicly available potato disease images significantly enhance the model's generalization ability across different imaging domains. A total of 425 high-resolution RGB images covering eight different disease categories were collected. To alleviate the shortage of category samples and introduce environmental diversity, this embodiment supplements the self-developed dataset with the publicly available dataset (Faria et al., 2023), particularly for samples at different severity stages of dry rot (caused by Fusarium). The detailed composition of each type of sample in the dataset (stratified by source) is summarized in Table 1. Dry rot (internal type), dry rot (surface type), and dry rot (severe type) are divided into three independent categories because their visual characteristics differ significantly and cannot be reliably modeled as single-category labels.

[0056] Table 1

[0057]

[0058] S1.3. Construct a potato disease dataset based on the images from steps S1.1 and S1.2, label the images using Anylabeling tools, and generate pixel-level disease masks for all images by drawing polygonal lesion boundaries.

[0059] S2. The images in the potato disease dataset obtained in step S1 are augmented using the image augmentation library alburementations to obtain the augmented potato disease dataset, which is divided into a training set and a test set.

[0060] Furthermore, the specific implementation method of step S2 includes the following steps:

[0061] S2.1. Perform a rigid geometric transformation on the images in the potato disease dataset obtained in step S1, setting the probability of horizontal / vertical flipping to 0.5 / 0.3, the probability of translation, scaling and rotation to 0.7, and the rotation angle to ±25°.

[0062] S2.2. Perform a non-rigid geometric transformation on the image processed in step S2.1. The probability of hitting the elastic / mesh distortion is 0.4. Set the control pixel displacement amplitude α=40 and the control spatial smoothness of the deformation σ=8.

[0063] S2.3. Perform photometric transformation on the image processed in step S2.2, set the probability of brightness / contrast to 0.6, set the limit range to ±0.2, then set the probability of limiting contrast adaptive histogram equalization to 0.3, and set the cropping limit to 3.0;

[0064] S2.4. Perform environmental simulation on the image processed in step S2.3, set the probability of shadow to 0.5, then perform idle blur processing, set the probability of Gaussian / motion / median blur to 0.4, set the convolution kernel size to 3-5, and obtain a data-augmented potato disease dataset with an image size of 256*256.

[0065] S2.5. Divide the augmented potato disease dataset into training and test sets according to a 9:1 ratio.

[0066] Furthermore, Figure 3 This visualization demonstrates an example of a data augmentation workflow used to enhance the diversity of a potato disease image dataset. The figure presents original images of two different disease categories (pink rot and common scab), along with five augmented variants for each original image. These augmented images are generated by combining geometric transformations (such as rotation and flipping) and photometric adjustments (such as changes in brightness and contrast). By generating multiple variants from a single source image, this augmentation workflow significantly expands the size of the training dataset, helps prevent overfitting, and ultimately improves the model's generalization ability to novel images.

[0067] S3. Construct a dual-head hybrid CNN-Transformer architecture, consisting of a CNN encoder for local feature extraction, a Transformer bottleneck layer for global context modeling, and a dual-head output structure coupled through a novel SAP mechanism;

[0068] Furthermore, the specific implementation method of step S3 includes the following steps:

[0069] S3.1. Set up a CNN encoder for local feature extraction. Use a ResNet-34 backbone network pre-trained on ImageNet as the feature encoder. Remove the last classification layer and pooling layer. Input the potato disease image into the CNN encoder for local feature extraction to obtain the feature map output by the encoder.

[0070] Furthermore, the encoder can effectively capture high-frequency local features, such as the scaly texture of common scabies or the sclerotium structure of mole.

[0071] S3.2. Set the Transformer bottleneck layer used for global context modeling to consist of 4 Transformer encoder modules, input the feature map output by the encoder to the Transformer bottleneck layer used for global context modeling, and obtain the feature map output at the Transformer bottleneck.

[0072] Furthermore, the Transformer bottleneck layer used for global context modeling can capture long-range dependencies, enabling the model to analyze the spatial distribution characteristics of lesions on the tuber surface—which is crucial for distinguishing diffuse infections such as pink rot from localized spot lesions.

[0073] S3.3. A novel SAP mechanism-coupled dual-head output structure is set up, which includes a classification head for disease identification using softmax activation and a segmentation decoder for generating lesion masks using a deconvolution structure combined with sigmoid activation. By implementing segmentation-aware pooling hard attention constraints, the feature map is selectively gated, and an explicit structural dependency is established between the segmentation and classification tasks.

[0074] Let F represent the feature map output from the bottleneck of the Transformer, M be the predicted soft segmentation mask, and calculate the classification feature vector v_cls as follows:

[0075] v_cls = GAP(F ⊙ Norm(M))

[0076] Where ⊙ represents element-wise multiplication, Norm(M) represents normalizing the soft mask to form a spatial attention map derived from the segmentation input, and GAP is global average pooling, used to compress spatial feature maps into classification feature vectors.

[0077] S4. Construct a composite multi-task loss function, and use the training set obtained in step S2 to train the dual-head hybrid CNN-Transformer architecture constructed in step S3 to obtain the trained dual-head hybrid CNN-Transformer architecture. Use the test set to perform automatic diagnosis and detection of potato diseases.

[0078] Furthermore, the specific implementation method of step S4 includes the following steps:

[0079] S4.1. The network is optimized using the composite multi-task loss function L_total, with the following expression:

[0080] L_total = λ_cls · L_CE + λ_seg · (L_Dice + L_BCE)

[0081] Where L_CE represents the cross-entropy loss used in the classification task, L_BCE represents the binary cross-entropy loss, L_Dice represents the Dice loss, and the classification target weight coefficient λ_cls = 1.0 and the segmentation target weight coefficient λ_seg = 1.0 are set.

[0082] S4.2. The network training uses the Adam optimizer, with an initial learning rate set to 1e. -4 The weight decay rate is 1e -2 The training batch size is set to 4, and the validation batch size is set to 1. A decoupling strategy for learning rate adjustment and model selection is adopted: the ReduceLROnPlateau scheduler is used to dynamically adjust the learning rate based on the validation set loss.

[0083] If the training set loss does not decrease for 5 consecutive epochs, the learning rate will be decayed by a factor of 0.9.

[0084] S4.3. The selection criterion for the optimal model checkpoint is to maximize the Dice similarity coefficient of the training set. By monitoring the Dice score, if the Dice score of the training set does not improve for 40 consecutive epochs, the early stopping mechanism is triggered.

[0085] All experiments in this embodiment were implemented using the PyTorch framework and executed on a workstation equipped with an NVIDIA V100 GPU (16GB VRAM). After data augmentation, the dataset was partitioned into 90 / 10 stratified sets to ensure that the class distribution was maintained across the training and validation sets. This stratification was performed after data augmentation to ensure that all augmented variants of the same original image were assigned to only one of the training or testing sets, thus preventing data leakage.

[0086] To comprehensively evaluate the performance of the proposed framework, different metrics were used during the training monitoring and final test set evaluation phases. Training monitoring and model selection: During the training phase, the Dessie similarity coefficient and cross-union ratio were calculated for each epoch based on the test set. The Dessie score served as the primary criterion for early stopping mechanisms and checkpoint selection, ensuring that the model prioritizes accurate lesion localization.

[0087] Quantitative evaluation using a test set: On an unseen test dataset, focus on two key performance metrics related to automated classification: Classification accuracy. Defined as the ratio of the number of correctly classified tuber images to the total number of test images:

[0088]

[0089] in, The number of correctly classified tuber images, Total number of test images;

[0090] The correlation (R²) between severity estimation and prediction reflects the predicted lesion area (A). pred ) and the actual lesion area (A gt The relationship between the two factors is as follows. The coefficient of determination (R²) measures the degree of agreement between the model's estimate based on the segmentation results and the expert annotations, reflecting the model's ability to reproducibly measure disease severity as a continuous trait.

[0091]

[0092] Among them, A gt This represents the mean true lesion area across all test images. A high R² value (approaching 1) indicates a high degree of consistency between the predicted and actual lesion areas, validating the model's reliable ability to reproducibly quantify infection severity in research, commercial, or breeding applications.

[0093] To verify the effectiveness of the SAP mechanism, gradient-weighted class activation mapping (SEM) was employed. This technique was applied to the last convolutional block before the Transformer layer in the classification task (corresponding to the model encoder layers [-1][-2]) and the second-to-last encoder block in the segmentation task (corresponding to the model encoder layers [-2][-1]). Gradient distributions were calculated based on these feature maps using a channel-level average activation method.

[0094] The training process of the obtained dual-head hybrid CNN-Transformer architecture is as follows: Figure 4 As shown, the model exhibits stable convergence characteristics, with the classification loss decreasing rapidly within the first 50 epochs. Simultaneously, the segmentation evaluation metrics—Intersection over Union (IoU) and Dissemination Similarity Coefficient (DSC)—show a steady logarithmic increase. The small difference between the training and validation losses indicates that, despite the high complexity of the task and the limited sample size for specific categories (such as potato flour scab), the employed data augmentation strategies effectively suppressed overfitting.

[0095] Then, the performance of the baseline hybrid CNN-Transformer model (without gating) was first evaluated on the reserved test set. Figure 5 As shown, the baseline model achieved a robust classification accuracy of 95.8%. Regarding severity quantification, regression analysis between the predicted lesion region and the ground truth labeled region showed a high coefficient of determination (R² = 0.995). Although these results indicate that the hybrid architecture effectively extracts spatial features, further analysis of the attention map (Section 3.3) revealed that the model partially relies on environmental context information, thus prompting the development of an SAP fine-tuning strategy. Figure 5The quantitative evaluation of the model on the unseen test dataset is presented in the following sections: (a) The row-normalized confusion matrix shows the model's classification performance for eight potato diseases. Diagonal elements represent the recall (sensitivity) of each category, while off-diagonal elements show the proportion of misclassified samples. (b) Scatter plot analysis shows the correlation between the actual lesion area (%) and the model-predicted lesion area (%).

[0096] Figure 6 This provides a qualitative visualization of the segmentation results for eight potato diseases. Each example set includes two comparison images: the left side shows the labeled ground truth segmentation mask (overlaid in green), and the right side shows the model-predicted segmentation mask (overlaid in cyan). The text labels above each image indicate the disease category, the model classification confidence score (Conf), and the percentage of the diseased area relative to the potato tuber surface area in both the ground truth (GT) and model predictions (Pred).

[0097] To explore the decision-making mechanism of the baseline model, a class activation map (Grad-CAM) was visualized. In the absence of an explicit gating mechanism, the classification head exhibited a "shortcut learning" tendency. This behavior limited the model's generalization ability under realistic and varied lighting and background conditions. Figure 7 As shown, the attention map of the classification task frequently activates non-pathological background regions (such as table edges or shadows). This indicates that although the baseline model achieves high numerical accuracy (95.8%), its predictions are partly driven by spurious background associations rather than being entirely based on biopathological features. Figure 7 A comparative visualization of the attention mechanism in the hybrid CNN-Transformer model. This figure shows the Grad-CAM saliency maps of two representative disease samples: dry rot (inside) at the top, and pink rot at the bottom. From left to right in each group are: the original input image (left), and the attention map generated by the classification head (encoder bottleneck layer) (middle).

[0098] To address the shortcut learning phenomenon observed in the baseline model, the proposed SAP mechanism was used to fine-tune the model. Quantitative results for the hybrid model with SAP enabled are as follows: Figure 8As shown, the most significant improvement lies in the quantitative accuracy of disease severity estimation. The SAP-enabled hybrid model achieves state-of-the-art correlation (R²=0.997, an improvement from the previous 0.995). The scatter plot shows a tighter clustering of data points along the baseline (y=x), particularly in potato samples with low to moderate infection severity (<20%) – a critical range for commercial grading. The SAP-enabled hybrid model achieves a classification accuracy of 95.6%. While this represents a slight decrease of 0.2% compared to the baseline model, it reflects that the model has eliminated "cheating" features from the background, ensuring that decisions are strictly based on the lesion characteristics themselves. Figure 8 Quantitative performance of the hybrid CNN-Transformer model integrating SAP. (a) Normalized confusion matrix of classification accuracy on the test set. The model achieves perfect recall (1.00) on six of the eight disease categories, demonstrating robust discriminative ability even across different severity stages of dry rot. (b) Severity estimation analysis: comparison of true values ​​vs. predicted lesion area (%).

[0099] The effectiveness of the SAP mechanism lies in Figure 9 This was intuitively validated in practice. Unlike the baseline model, the hybrid model using SAP exhibits a highly concentrated attention distribution in its classification head (middle image), strictly corresponding to the segmentation output (right image). Taking the dried rot (surface) sample as an example, the model completely ignores healthy epidermis and background interference, focusing only on the necrotic depression area. This confirms that the hard attention mechanism successfully filters out environmental noise, ensuring the biological validity of the prediction results. Figure 9 This is a visualization illustrating the attention mechanism in the hybrid CNN-Transformer model for enabling SAP. The figure shows Grad-CAM saliency maps for two representative disease samples: top for dry rot (surface) and bottom for severe dry rot. From left to right, each group consists of: the original input image (left) and the attention map generated by the classification head (middle).

[0100] To place the findings in a broader context, the proposed framework was benchmarked against dual-task versions of standard architectures: a pure CNN model (ResNet34 backbone), a pure vision Transformer (ViT), and the standard U-Net. Table 2 summarizes the comparison results. The single-modal benchmark models exhibited clear performance trade-offs: CNNs excelled in segmentation tasks but lacked the global contextual information needed for robust classification, while ViT was adept at classification but failed to generate accurate lesion boundaries. The hybrid model with SAP enabled was the only architecture to achieve a balanced superiority, maintaining top-tier classification accuracy while achieving the highest severity estimation accuracy (R²=0.997).

[0101] Table 2

[0102]

[0103] Table 2 reveals the delicate trade-off between standard accuracy metrics and model robustness. The baseline hybrid model achieved slightly higher classification accuracy than the SAP-enhanced model (95.8% vs. 95.6%), but qualitative analysis shows that the baseline model boosted confidence by exploiting spurious background relevance—a phenomenon known as the "Smart Hans" effect (Geirhos et al., 2020). By mathematically masking the background through the SAP mechanism, the model is forced to eliminate these learning shortcuts. While this constraint results in a negligible decrease in test set accuracy, it significantly enhances the model's credibility and generalization ability. In real-world agricultural scenarios (with variable backgrounds), models focusing solely on disease characteristics are far more practically valuable than those that achieve high accuracy through memorizing lab benches.

[0104] While disease classification is a prerequisite for disease management, the quantification of severity is the economic driver of the potato industry. In breeding projects and packaging plants, the distinction between Grade A potatoes (1% defect rate) and Grade B potatoes (5% defect rate) directly determines their market value. The model equipped with the SAP mechanism achieved a determination coefficient of R²=0.997, significantly outperforming the standard baseline model. This level of accuracy validates the superiority of semantic segmentation technology over object detection (bounding box methods) (Arshaghi et al., 2023). Bounding boxes inevitably include healthy tissue within rectangular areas, while pixel-level methods can accurately measure the necrotic surface area, providing a reproducible standard for automated grading.

[0105] It is worth noting that the design mechanism of coupling the dual-task heads itself improves the segmentation quality. In standard multi-task learning, different tasks often have competing objectives; however, in the framework equipped with the SAP mechanism, the classification head must rely on the segmentation mask to "observe" lesions. This forms a supervised feedback loop: if the segmentation head generates a noisy mask, the classifier will be unable to identify lesion features, resulting in a high loss value and thus penalizing the segmentation decoder. This structural dependency effectively forces the segmentation network to generate clearer and more semantically explicit masks, proving that imposing interpretability constraints can proactively improve model performance.

[0106] Although attention maps for the single-modal baseline models (pure CNN, pure ViT) were not shown, their architecture lacked mechanisms to suppress irrelevant background information. Visual Transformers, in particular, readily absorb global contextual information from shallow network layers, inducing shortcut learning. By explicitly zeroing out non-lesion regions before classification, the hybrid model equipped with SAP severed these spurious associations. Therefore, the model achieved higher biological validity and better adapted to imaging environments with significantly varying background conditions. These findings demonstrate the potential of interpretable AI technologies to empower industrial grading, breeding, and disease monitoring processes.

[0107] The findings highlight the inherent trade-offs of unimodal architectures: CNN-based models (U-Net, pure CNN) typically perform well in segmentation tasks (with high Dice / mIoU metrics) because convolutional operations effectively preserve local spatial details such as edges and textures. However, these models often fall short in classification consistency due to their lack of a global receptive field, making it difficult to associate lesion patterns at greater distances.

[0108] Transformer-based models (ViT) exhibit the opposite trend. They excel in classification tasks by capturing global context (i.e., the "whole potato" view) but struggle with fine-grained segmentation. Due to the fixed patch size used in their tokenization process, pixel-level predictions often exhibit "blocky" or coarse characteristics.

[0109] The hybrid architecture achieves synergistic performance by effectively integrating the advantages of both approaches, outperforming the single-modal baseline model and delivering more balanced results. This architecture utilizes a CNN encoder to extract high-fidelity texture features (crucial for distinguishing between black nevus and common scabies), while simultaneously modeling long-range dependencies through a Transformer bottleneck layer (essential for understanding the spread of dry rot). This allows the model to achieve a dual optimization of classification accuracy and segmentation precision.

[0110] Several data augmentation strategies are designed to eliminate the risk of shortcut learning and enhance the ability to extract features centered on lesions. Rotational augmentation combined with reflection filling avoids artificial black borders that resemble non-lesion backgrounds—without this processing, the model might incorrectly associate these artifacts with specific disease categories. Elastic deformation into lesions provides biologically accurate deformation patterns, reducing the risk of the model overfitting the tuber outline or overall geometry rather than lesion features.

[0111] Since many potato diseases are defined by local microtexture (such as the scaly, rough surface of common scab or the fine, silvery appearance of psoriasis), the Limiting Contrast Adaptive Histogram Equalization (CLAHE) technique is used to enhance spatial local texture features. This method forces the classifier and segmentation head to rely on high-frequency epidermal features rather than global intensity differences for judgment.

[0112] Finally, by implementing at least one enhancement process for each sample, we ensured that both the training and test sets used the enhanced image variants, while the original, unmodified images were only used in the final testing phase. This approach prevented information leakage and also verified the optimized model's true generalization ability to novel image distributions.

[0113] While numerous segmentation and detection models exist, dual-task-head models hold unique value. Although object detection architectures (such as the YOLO series) and instance segmentation frameworks (such as Mask R-CNN) have become mainstream solutions in the field of fruit counting for agricultural robots, they are not suitable for the precise quantitative analysis of potato pathology. Potato diseases such as black scab and common scab typically present as amorphous, irregular textures rather than clearly defined, countable objects. If severity assessment relies on bounding box methods (such as YOLO), the geometric rigidity of the rectangular region inevitably mixes healthy tissue with necrotic lesions in the statistics, leading to an overestimation of severity indicators. Similarly, instance segmentation models (such as Mask R-CNN) treat each individual lesion as a separate entity. Given that a single potato tuber may present hundreds of tiny, scattered sclerotia, treating them as independent instances would impose a huge computational burden and ignore the overall texture background required for diagnosis.

[0114] Therefore, this problem is defined as a semantic segmentation task, treating defects as global surface textures. This method not only achieves accurate severity rating at the pixel level (R²=0.997), but also generates a unified binary mask required for an innovative segmentation-aware SAP mechanism—a structural dependency that region-based proposal networks cannot natively support.

[0115] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network, characterized in that, Includes the following steps: S1. Combine independently collected potato disease images with publicly available data resources to construct a potato disease dataset, and use Anylabeling tools to label all images in the potato disease dataset; S2. The images in the potato disease dataset obtained in step S1 are augmented using the image augmentation library alburementations to obtain the augmented potato disease dataset, which is divided into a training set and a test set. S3. Construct a dual-head hybrid CNN-Transformer architecture, consisting of a CNN encoder for local feature extraction, a Transformer bottleneck layer for global context modeling, and a dual-head output structure coupled through a novel SAP mechanism; S4. Construct a composite multi-task loss function, and use the training set obtained in step S2 to train the dual-head hybrid CNN-Transformer architecture constructed in step S3 to obtain the trained dual-head hybrid CNN-Transformer architecture. Use the test set to perform automatic diagnosis and detection of potato diseases.

2. The method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network according to claim 1, characterized in that, The specific implementation method of step S1 includes the following steps: S1.

1. Independently collect potato disease images in the field, including images of potato diseases such as black scurf, common scab, pink rot, powdery scab and psoriasis; S1.

2. Select potato disease images from publicly available data resources, including publicly available benchmark datasets, newly created expert-annotated datasets from Heilongjiang Province, China, and independent datasets from Norway, to establish samples of different severity stages of dry rot, and divide dry rot internal type, dry rot surface type and severe dry rot type into three independent categories. S1.

3. Construct a potato disease dataset based on the images from steps S1.1 and S1.2, label the images using Anylabeling tools, and generate pixel-level disease masks for all images by drawing polygonal lesion boundaries.

3. The method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network according to claim 2, characterized in that, The specific implementation method of step S2 includes the following steps: S2.

1. Perform a rigid geometric transformation on the images in the potato disease dataset obtained in step S1, setting the probability of horizontal / vertical flipping to 0.5 / 0.3, the probability of translation, scaling and rotation to 0.7, and the rotation angle to ±25°. S2.

2. Perform a non-rigid geometric transformation on the image processed in step S2.

1. The probability of hitting the elastic / mesh distortion is 0.

4. Set the control pixel displacement amplitude α=40 and the control spatial smoothness of the deformation σ=8. S2.

3. Perform photometric transformation on the image processed in step S2.2, set the probability of brightness / contrast to 0.6, set the limit range to ±0.2, then set the probability of limiting contrast adaptive histogram equalization to 0.3, and set the cropping limit to 3.0; S2.

4. Perform environmental simulation on the image processed in step S2.3, set the probability of shadow to 0.5, then perform idle blur processing, set the probability of Gaussian / motion / median blur to 0.4, set the convolution kernel size to 3-5, and obtain a data-augmented potato disease dataset with an image size of 256*256. S2.

5. Divide the augmented potato disease dataset into training and test sets according to a 9:1 ratio.

4. The method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network according to claim 3, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. Set up a CNN encoder for local feature extraction. Use a ResNet-34 backbone network pre-trained on ImageNet as the feature encoder. Remove the last classification layer and pooling layer. Input the potato disease image into the CNN encoder for local feature extraction to obtain the feature map output by the encoder. S3.

2. Set the Transformer bottleneck layer used for global context modeling to consist of 4 Transformer encoder modules, input the feature map output by the encoder to the Transformer bottleneck layer used for global context modeling, and obtain the feature map output at the Transformer bottleneck. S3.

3. A novel SAP mechanism-coupled dual-head output structure is set up, which includes a classification head for disease identification using softmax activation and a segmentation decoder for generating lesion masks using a deconvolution structure combined with sigmoid activation. By implementing segmentation-aware pooling hard attention constraints, the feature map is selectively gated, and an explicit structural dependency is established between the segmentation and classification tasks. Let F represent the feature map output from the bottleneck of the Transformer, M be the predicted soft segmentation mask, and calculate the classification feature vector v_cls as follows: v_cls = GAP(F ⊙ Norm(M)) Where ⊙ represents element-wise multiplication, Norm(M) represents normalizing the soft mask to form a spatial attention map derived from the segmentation input, and GAP is global average pooling, used to compress spatial feature maps into classification feature vectors.

5. The method for phenotypic analysis and severity quantification of potato tuber diseases using a dual-head CNN-Transformer network according to claim 4, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. The network is optimized using the composite multi-task loss function L_total, with the following expression: L_total = λ_cls · L_CE + λ_seg · (L_Dice + L_BCE) Where L_CE represents the cross-entropy loss used in the classification task, L_BCE represents the binary cross-entropy loss, L_Dice represents the Dice loss, and the classification target weight coefficient λ_cls = 1.0 and the segmentation target weight coefficient λ_seg = 1.0 are set. S4.

2. The network training uses the Adam optimizer, with an initial learning rate set to 1e. -4 The weight decay rate is 1e -2 The training batch size is set to 4, and the validation batch size is set to 1. A decoupling strategy for learning rate adjustment and model selection is adopted: the ReduceLROnPlateau scheduler is used to dynamically adjust the learning rate based on the validation set loss. If the training set loss does not decrease for 5 consecutive epochs, the learning rate will be decayed by a factor of 0.

9. S4.

3. The selection criterion for the optimal model checkpoint is to maximize the Dice similarity coefficient of the training set. By monitoring the Dice score, if the Dice score of the training set does not improve for 40 consecutive epochs, the early stopping mechanism is triggered.

Images