Plant leaf disease severity assessment method based on position-guided lesion representation learning

Abstract

The present invention relates to the field of computer vision technology, and in particular to a method for assessing the severity of plant leaf diseases based on position-guided lesion representation learning. The method comprises the following steps: obtaining a diseased leaf image; inputting the diseased leaf image into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, the real training set being leaf images labeled with disease severity; the hierarchical lesion fusion assessment network model comprises a feature encoder with prior knowledge, a hierarchical lesion feature fusion module, and an assessment module; the feature encoder with prior knowledge is obtained by inputting an image pair into a position-guided lesion representation learning network for training, the image pair representing the healthy and diseased states of the same leaf; the leaf disease state is generated from a healthy leaf using a diffusion model to identify a diseased leaf. The present invention enables more accurate positioning and severity assessment of lesion areas.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for assessing the severity of plant leaf diseases based on position-guided lesion representation learning. Background Art

[0002] Crop yields are frequently threatened by various plant diseases, resulting in annual yield reductions of approximately 13% to 22%, leading to significant agricultural losses. Deep learning technology has made significant progress in plant disease identification, achieving accuracy rates exceeding 90% due to its powerful feature extraction and automated learning capabilities. However, this technology is limited to disease identification and cannot provide precise guidance on pesticide dosage. This limitation leads to reduced crop yields, environmental pollution, economic losses, and missed optimal treatment periods. Therefore, accurately assessing the severity of plant diseases is crucial for achieving intelligent crop management and increasing food production.

[0003] Traditional methods for assessing plant disease severity often rely on the experience and judgment of agricultural experts or laboratory biochemical analysis. While effective in certain situations, these methods are limited by the subjectivity and labor-intensive nature of human judgment, making them unsuitable for large-scale assessments. With advances in computer vision, image processing technology is now widely used to assess plant disease severity, and machine learning methods can automate image analysis and process disease information. However, existing machine learning methods struggle to accurately assess disease severity in complex field environments because their performance relies heavily on high-quality feature engineering.

[0004] With the rapid development of computer vision in recent years, convolutional neural networks (CNNs) and visual transformers (ViTs) have become increasingly widely used in plant disease severity assessment, demonstrating greater automation and accuracy. These methods extract image features through deep models, overcoming the traditional reliance on manual feature extraction and demonstrating greater potential. However, these methods primarily rely on overall image features, making them prone to misidentifying non-lesion areas, such as background information, as lesion areas, leading to biased severity assessments.

[0005] Therefore, developing a method that can more accurately locate and assess the severity of lesion areas is of great significance for improving agricultural intelligent management and crop health monitoring. Summary of the Invention

[0006] The purpose of the present invention is to provide a plant leaf disease severity assessment method based on position-guided lesion representation learning, which is suitable for severity assessment of various plant leaf diseases and can be migrated to fruit disease severity assessment.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] The plant leaf disease severity assessment method based on location-guided lesion representation learning includes:

[0009] Acquire images of diseased leaves;

[0010] The diseased leaf image is input into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, and the real training set is a leaf image with annotated disease severity. The hierarchical lesion fusion assessment network model is constructed based on a feature encoder with prior knowledge, and the feature encoder with prior knowledge is obtained by inputting image pairs into a position-guided lesion representation learning network for training.

[0011] Optionally, obtaining a feature encoder with prior knowledge includes:

[0012] Constructing a position-guided lesion representation learning network model, wherein the position-guided lesion representation learning network model is an encoder-decoder based model, the encoder is a dual-branch feature encoder, and the decoder is a differential positioning decoder;

[0013] The position-guided lesion representation learning network model is trained using the image to obtain the trained position-guided lesion representation learning network model, and the trained dual-branch feature encoder is frozen as the feature encoder with prior knowledge.

[0014] Optionally, using the image to train the position-guided lesion representation learning network model includes:

[0015] Inputting a healthy leaf image into a diffusion model to generate a diseased leaf image, and constructing the image pair based on the healthy leaf image and the corresponding diseased leaf image;

[0016] Inputting the image pair into a dual-branch feature encoder to perform feature extraction to obtain features of the image pair;

[0017] Inputting the features of the image pair into the differential positioning decoder, locating the lesion area by calculating the difference between the image pair, and obtaining a lesion area prediction result;

[0018] Based on the lesion area prediction result and the actual lesion area, the position-guided lesion representation learning network model is trained through the constraints of the loss function.

[0019] Optionally, inputting the image pair into a dual-branch feature encoder for feature extraction includes:

[0020] Extracting hierarchical fine-grained features from the image pair using a hierarchical information extractor to obtain fine-grained semantic features;

[0021] extracting high-level semantic features from the image pairs using an information aggregator to obtain high-level semantic features;

[0022] The fine-grained semantic features and the high-level semantic features are input into a cross-layer feature fusion module for feature fusion to obtain fused features, wherein the cross-layer feature fusion module is obtained based on several cross-attention modules, and the fused features and the hierarchical fine-grained features are features of the image pair.

[0023] Optionally, inputting the fine-grained semantic features and the high-level semantic features into a cross-layer feature fusion module for feature fusion includes:

[0024] Multiplying the fine-grained semantic features of each layer by the high-level semantic features and dividing the result by the scaling value to obtain the attention weight;

[0025] The attention weight is multiplied by the fine-grained semantic features of each layer to obtain the attention matrix of each layer and connected along the channel dimension to obtain the fusion feature.

[0026] Optionally, obtaining the lesion area prediction result includes:

[0027] Calculating the features of the image pair using a difference method and performing fusion and convolution operations to obtain differential features;

[0028] Aggregating the differential features through cascaded upsampling and addition operations and restoring them to the original resolution;

[0029] The original resolution feature map is converted into a binary map through an activation function and a preset threshold to generate the lesion area prediction result.

[0030] Optionally, inputting the diseased leaf image into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result includes:

[0031] Inputting the diseased leaf image and the copied diseased leaf image into the feature encoder with prior knowledge to perform feature extraction to obtain diseased leaf features;

[0032] Inputting the diseased leaf features into a hierarchical lesion feature fusion module for feature fusion and enhancement to obtain fusion-enhanced features;

[0033] The fused enhanced features are input into an evaluation module to obtain the leaf disease severity evaluation result.

[0034] Optionally, inputting the diseased leaf features into a hierarchical lesion feature fusion module for feature fusion and enhancement includes:

[0035] The diseased leaf features are fused by an addition operation, and the fused diseased leaf features are input into a channel attention module to enhance the channel features to obtain a channel feature map;

[0036] The channel feature map is input into the spatial attention module to enhance the spatial features and obtain a fusion enhancement map.

[0037] Optionally, inputting the fused enhanced features into an evaluation module to obtain the leaf disease severity evaluation result includes:

[0038] The fused enhanced features are processed in sequence by convolution layer, batch normalization and ReLu activation function, residual block, adaptive average pooling layer, fully connected layer and softmax function to obtain the probability distribution of disease severity at each level, and the disease severity level corresponding to the highest probability is used as the leaf disease severity assessment result.

[0039] Compared with the prior art, the present invention has the following advantages and technical effects:

[0040] The present invention addresses the problem that traditional deep learning methods rely on global image features and are prone to mistaking background information for lesion areas, resulting in deviations in severity assessment. A location-guided lesion representation learning evaluation strategy is proposed. This strategy focuses on extracting features from the lesion area and further assessing the severity of the disease based on these features. Specifically, the present invention first uses a diffusion model to generate diseased leaves from healthy leaves to construct image pairs of healthy and diseased leaves. Next, the location-guided disease representation learning network trains the network to identify and locate the lesion area on the leaf by comparing healthy and diseased leaves, thereby obtaining an encoder with prior knowledge. Finally, the hierarchical disease fusion evaluation network shares and freezes the encoder, extracts features focused on the lesion area, and fuses and classifies these features through a hierarchical strategy to achieve a quantitative assessment of the severity of the disease. Through the above method, the present invention effectively eliminates the interference of background information, accurately learns the features of the lesion area, and thus significantly improves the accuracy of disease severity assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1This is a diagram of the overall network architecture for implementing a method for assessing plant leaf disease severity based on location-guided lesion representation learning according to an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of a position-guided representation learning network according to an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the structure of a hierarchical lesion fusion assessment network according to an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of the visualization results of locating three plant lesion areas according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] This embodiment provides a method for assessing plant leaf disease severity based on location-guided lesion representation learning, including:

[0049] Acquire images of diseased leaves;

[0050] The diseased leaf image is input into the hierarchical lesion fusion assessment network model to obtain the leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, and the real training set is a leaf image marked with the disease severity. The hierarchical lesion fusion assessment network model includes a feature encoder with prior knowledge, a hierarchical lesion feature fusion module and an evaluation module. The feature encoder with prior knowledge is obtained by inputting image pairs into the position-guided lesion representation learning network model for image pair training.

[0051] Specifically, the training of this embodiment requires two datasets in total, one is a dataset of plant leaf health or disease in a real environment for the final severity assessment, and the other is an image pair constructed based on a generative model.

[0052] Realistic dataset construction: This dataset was selected from the real world. After the data was selected, the severity of the dataset was manually labeled. During the training process, the present invention set the ratio of the training set to the test set to 8:2 to train the hierarchical lesion fusion assessment network.

[0053] Image pair construction: Use the trained diffusion model to generate plant disease images from healthy leaf images, and make a one-to-one correspondence with healthy leaves to form image pairs. The training set and test set are divided into a ratio of 8:2, and the training locations are used to guide the lesion representation learning network.

[0054] Furthermore, obtaining a feature encoder with prior knowledge includes:

[0055] Constructing a position-guided lesion representation learning network model, wherein the position-guided lesion representation learning network model is an encoder-decoder based model, the encoder is a dual-branch feature encoder, and the decoder is a differential positioning decoder;

[0056] The image is used to train the position-guided lesion representation learning network model, the trained position-guided lesion representation learning network model is obtained, and the trained dual-branch feature encoder is frozen as a feature encoder with prior knowledge.

[0057] Specifically, the position-guided lesion representation learning network: the constructed image pairs are input into the position-guided lesion representation learning network, the features are extracted using a dual-branch feature encoder, and the lesion area is decoded through a differential positioning decoder. After learning, a feature encoder with prior knowledge can be obtained.

[0058] Furthermore, using images to guide the location of lesion representation learning network models to train includes:

[0059] The healthy leaf image is input into the diffusion model to generate the diseased leaf image, and an image pair is constructed based on the healthy leaf image and the corresponding diseased leaf image;

[0060] Input the image pair into the dual-branch feature encoder for feature extraction to obtain the features of the image pair;

[0061] The features of the image pair are input into the differential localization decoder, and the lesion area is located by calculating the difference between the image pairs to obtain the lesion area prediction result;

[0062] Based on the lesion area prediction results and the actual lesion area, the lesion representation learning network model is guided by the constrained training position of the loss function.

[0063] Specifically, the location-guided lesion representation learning network is trained: To constrain the model, binary cross entropy and Dice loss are used to optimize lesion localization performance. By training the network with the constraints of the loss function, lesion localization is achieved and a two-branch feature encoder with prior knowledge of the lesion region is obtained. The weights of these encoders are stored as the encoder for the hierarchical lesion fusion assessment network.

[0064] Furthermore, the image is input into the dual-branch feature encoder for feature extraction, including:

[0065] Use the hierarchical information extractor to extract hierarchical fine-grained features from image pairs and obtain fine-grained semantic features;

[0066] Using information aggregator to extract high-level semantic features from image pairs to obtain high-level semantic features;

[0067] The fine-grained semantic features and high-level semantic features are fed into a cross-layer feature fusion module for feature fusion to obtain fused features. The cross-layer feature fusion module is based on several cross-attention modules. The final output of the encoder is an image pair feature that includes the fused features and the hierarchical fine-grained features.

[0068] Specifically, the hierarchical fine-grained feature extraction: The hierarchical feature extractor is similar in structure to ResNet18, preserving visual semantic information at all levels while maintaining computational efficiency. It uses a strategy trained from scratch, specifically designed to extract detailed information from the input image, generating features at different resolutions: 256×256, 128×128, and 64×64.

[0069] Extraction of high-level semantic information: The information aggregator uses a ResNet18 architecture pre-trained on 421,133 plant disease images and fine-tuned during model training to adapt to the new task. Its output is a feature of size 32×32.

[0070] Furthermore, the fine-grained semantic features and high-level semantic features are input into the cross-layer feature fusion module for feature fusion, including:

[0071] Multiply the fine-grained semantic features of each layer with the high-level semantic features and divide them by the scaling value to obtain the attention weight;

[0072] Multiply the attention weights with the fine-grained semantic features of each layer to obtain the attention matrix of each layer and connect them along the channel dimension to obtain the fusion features.

[0073] Specifically, the cross-layer feature fusion module consists of three cross-attention blocks. It refines hierarchical features by using high-level features to guide the enhancement of the focus area. Fine-grained and high-level semantic features are combined using an attention matrix. Specifically, the fine-grained information at each layer is multiplied by the high-level semantic information and then divided by a scaling value to obtain the attention weights. The weights are multiplied by the fine-grained information at each layer to obtain the attention matrix for each layer, which is then concatenated along the channel dimension.

[0074] Furthermore, obtaining the lesion area prediction results includes:

[0075] The features of the image pair are calculated using the difference method and then fused and convolved to obtain the differential features;

[0076] The differential features are aggregated through cascaded upsampling and addition operations and restored to the original resolution;

[0077] The original resolution feature map is converted into a binary image through the activation function and the preset threshold to generate the lesion area prediction result.

[0078] Specifically, the features of the extracted image pairs are input into the differential positioning decoder, and the feature differences between healthy leaves and diseased leaves are directly calculated to locate the lesion area. Then, the original resolution is restored through upsampling and fusion, and the lesion area prediction result is obtained after activation.

[0079] Furthermore, the diseased leaf image is input into the hierarchical lesion fusion assessment network model to obtain the leaf disease severity assessment results including:

[0080] The diseased leaf image and the replicated diseased leaf image are input into a feature encoder with prior knowledge to extract features and obtain the diseased leaf features;

[0081] The diseased leaf features are input into the hierarchical lesion feature fusion module for feature fusion and enhancement to obtain fusion-enhanced features;

[0082] The fused enhanced features are input into the evaluation module to obtain the leaf disease severity assessment results.

[0083] Specifically, the severity of the leaf disease is assessed by inputting the diseased leaf image into the Hierarchical Lesion Fusion Assessment Network. This network integrates a pre-trained two-branch feature encoder for feature extraction, a Hierarchical Lesion Fusion module for feature fusion and enhancement, and finally an assessment module to determine the corresponding grade.

[0084] Furthermore, the diseased leaf features are input into the hierarchical lesion feature fusion module for feature fusion and enhancement, including:

[0085] The diseased leaf features are fused through addition operations, and the fused diseased leaf features are input into the channel attention module to enhance the channel features and obtain the channel feature map;

[0086] The channel feature map is input into the spatial attention module to enhance the spatial features and obtain the fused enhanced map.

[0087] Specifically, feature fusion enhancement: the hierarchical lesion information fusion module is used to fuse the two types of extracted features, and channel attention and spatial attention are cascaded to enhance the focus on key areas.

[0088] Feature fusion: Through adaptive adjustment, the extracted fine-grained hierarchical features and the high-level semantic features extracted and enhanced by the cross-layer feature fusion module are adjusted to the same scale, and then the two types of features are fused through an addition operation.

[0089] Channel Attention: The channel attention mechanism assigns weights to feature map channels, highlighting important channels. Specifically, the fused feature maps undergo global max and average pooling to capture channel context, which is then passed through a fully connected layer. The resulting weights are fused and applied to the fused feature map from the feature fusion step to enhance key channels.

[0090] Spatial Attention: Spatial attention highlights important locations in the feature map. Specifically, the map after the channel attention step is compressed along the channel dimension through max and average pooling, resulting in a spatial feature map. These maps are added to generate attention weights, which are applied to the spatial dimensions of the feature map obtained by the channel attention step.

[0091] Furthermore, the fused enhanced features are input into the evaluation module to obtain the leaf disease severity assessment results including:

[0092] The fused enhanced features are processed in sequence by convolutional layer, batch normalization and ReLu activation function, residual block, adaptive average pooling layer, fully connected layer and softmax function to obtain the probability distribution of disease severity at each level, and the disease severity level corresponding to the highest probability is used as the leaf disease severity assessment result.

[0093] Specifically, level evaluation: the step output features are mapped through 3×3 convolution to capture local spatial information. At the same time, batch normalization and ReLu activation function are used to ensure the stability of training and introduce nonlinearity. Then, the residual block is used to further enhance the capture of deep semantic information to ensure that the model can learn complex feature relationships at a deeper level. Subsequently, adaptive average pooling is applied to downsample the feature map to a size of 1×1 to unify the dimension of the features. Finally, the features are projected to the corresponding hierarchical space through the fully connected layer, and the results are normalized using the softmax function to obtain the probability distribution of each level. The highest probability is the severity level of the assessment.

[0094] In this embodiment, the severity of plant diseases is divided into six levels, corresponding to a multi-class classification task. In order to optimize the hierarchical lesion fusion assessment network, a cross entropy loss is applied.

[0095] The method of this embodiment is further described in detail below:

[0096] The network model used in the plant leaf disease severity assessment method based on position-guided lesion representation learning in this embodiment consists of three main parts: an image generation network, a position-guided lesion representation learning network, and a hierarchical lesion fusion assessment network. Figure 1 . The image generation network uses a diffusion model to generate diseased leaves from healthy leaves to construct the paired images required by the location-guided lesion representation learning network. The location-guided lesion representation learning network uses an encoder-decoder architecture to identify and locate lesion regions on leaves by comparing healthy-lesion paired photos, and the encoder is trained using prior knowledge of plant lesion regions. The hierarchical lesion fusion assessment network uses an encoder trained by the learning network to extract lesion-specific features, and then uses a hierarchical strategy to fuse and classify these features to provide a quantitative assessment of disease severity.

[0097] The plant leaf disease severity assessment method based on position-guided lesion representation learning specifically includes the following steps:

[0098] Step 1: Construct a dataset. The training of this embodiment requires two datasets: one is a dataset of plant leaf health or disease in a real environment for the final severity assessment, and the other is an image pair constructed based on a generative model.

[0099] Step 1.1: Real-world dataset construction. This dataset was selected from the real world. After the data was selected, the dataset was manually labeled according to severity. The grading criteria used are shown in Table 1. During the training process, the present invention sets the training set to test set at a ratio of 8:2.

[0100] Step 1.2: Image pair construction: Use the trained image generation network (diffusion model) to generate plant disease images from healthy leaf images and establish a one-to-one correspondence with healthy leaves to form image pairs.

[0101] Table 1

[0102]

[0103] Step 2: Location-guided lesion representation learning, see Figure 2 The image pair is input into the position-guided lesion representation learning network, which is an encoder-decoder based model. Specifically, a dual-branch feature encoder is used to extract features, and the lesion area is decoded by a differential localization decoder. After learning, a feature encoder with prior knowledge can be obtained.

[0104] Step 2.1. Feature Extraction. The image pair is input into a dual-branch feature encoder to obtain two types of feature information. A parallel hierarchical feature extractor and an information aggregator (also known as an information refiner) are used to extract fine-grained and high-level semantic features from the image pair, respectively. These features are then integrated through a cross-layer feature fusion module to enhance focus on the lesion area.

[0105] Step 2.1.1: Hierarchical fine-grained features The hierarchical feature extractor is similar to ResNet18 in structure and can retain visual semantic information at all levels while maintaining computational efficiency. Specifically, the hierarchical feature extractor uses the first three layers of ResNet18 to generate features of different resolutions. 256×256, 128×128 and 64×64. The strategy adopted is to train from scratch, which is specifically used to extract detail information from the input image. Step 2.1.2, high-level semantic information F A The information aggregator uses a four-layer ResNet18 network, which uses the ResNet18 architecture pre-trained on 421,133 plant disease images and fine-tuned during model training to adapt to the new task. Its output is a feature F of size 32×32 A .

[0106] Step 2.1.3, feature enhancement. The cross-layer feature fusion module consists of three cross-attention blocks, which refines hierarchical features by using high-level features to guide the enhancement of the focus area. Using the attention matrix combination and F A The output features of each layer Respectively with F A Multiply and divide by the scale value Get the attention weight, multiply the weight by The attention matrices of each layer can be obtained and then connected along the channel dimension. The process is defined as:

[0107] F C =Concat(F 1 ,F 2 ,F 3 )

[0108]

[0109] Among them, Concat(·) represents the connection in the channel dimension, F i Represents three levels of fine-grained features With high-level semantic features F A The result after cross attention, F C It means concatenating the results of the three attention operations along the channel dimension.

[0110] Step 2.2: Decode the lesion area. The features of the image pair are input into the differential localization decoder, which directly calculates the feature differences between the healthy and diseased leaves to locate the lesion area. The original resolution is then restored through upsampling and fusion, and the lesion area prediction result is obtained through activation.

[0111] Specifically, calculate the output of the dual-branch feature encoder, that is, the four-layer healthy leaf feature and leaf characteristics of lesions The difference between the two layers is obtained by using the difference method, and the original features are connected and fused with the difference features. Then, the convolution operation is performed on the fused features to obtain the difference features of each layer. Then, through the cascade upsampling and addition operations, the differential features from deep to shallow layers are gradually aggregated and finally restored to the original resolution H×W. Finally, the original resolution feature map is converted to Convert it into a binary image and generate the prediction result P of the lesion area. The specific process is defined as follows:

[0112]

[0113] Among them, MLP is a three-layer convolutional network, |·| represents the absolute value operation, and US(·) is a convolution with a kernel size of 1×1, which is used to reduce the channel dimension and improve the resolution.

[0114] Step 2.3: Position-guided lesion representation learning network training. To constrain the model, binary cross entropy and Dice loss are used to optimize the localization performance of the lesion area. Binary cross entropy loss L bceIt is used to measure the accuracy of the model in the binary classification task so that the model network can better identify the pixel-level lesion area. It is defined as follows:

[0115]

[0116] Among them, P represents the predicted value, Y represents the label value, N is the total number of pixels, P i Represents the prediction result of the i-th pixel, Y i Represents the true value of the pixel.

[0117] Dice loss L dice It is used to solve the problem of imbalanced data sets, especially when the ratio of lesion area to background area is unbalanced. It is defined as follows:

[0118]

[0119] Here, ε is a smoothing term used to avoid division by zero.

[0120] The objective function is composed of binary cross entropy loss and Dice loss to form a hybrid loss function, which can effectively balance the pixel classification accuracy and region overlap optimization in the segmentation task, and is defined as follows: L LGRL =L bce (P,Y)+L dice (P,Y)

[0121] By training the network under the constraints of the loss function, the lesion area can be located, and a two-branch feature encoder with prior knowledge of the lesion area can be obtained, and its weights are saved as the encoder of the hierarchical lesion fusion evaluation network.

[0122] Step 3: Severity Assessment, see Figure 3 The severity of the leaf disease can be assessed by inputting the diseased leaf image into the hierarchical lesion fusion assessment network. It integrates a pre-trained two-branch feature encoder for feature extraction, uses a hierarchical lesion fusion module to fuse and enhance the features, and finally uses an assessment module to determine the corresponding grade.

[0123] Step 3.1, Feature Extraction: Feature extraction is performed directly using the pre-trained two-branch feature encoder, and its parameters are frozen to preserve prior knowledge of the lesion area, thereby achieving efficient localization of the diseased area. Severity assessment only requires a single image of the diseased leaf as input. To maintain the encoder's original input image pair structure, a paired input is constructed by duplicating the diseased leaf image.

[0124] Step 3.2, feature fusion enhancement: Use the hierarchical lesion information fusion module to fuse the two types of features extracted by the encoder, and cascade channel attention and spatial attention to enhance attention to key areas.

[0125] Step 3.2.1, Feature Fusion: Through adaptive adjustment, the fine-grained hierarchical features extracted from the hierarchical feature extractor are The high-level semantic features F extracted from the information aggregator and enhanced by the cross-layer feature fusion module C Adjust to the same scale, and then fuse the two types of features through addition. The detailed process is as follows:

[0126]

[0127] Where DS(·) represents adaptive downsampling.

[0128] Step 3.2.2, Channel Attention: The channel attention mechanism assigns weights to feature map channels to highlight important channels. Specifically, the fused feature map undergoes global max and average pooling to capture channel context, which is passed through the fully connected layer. The resulting weights are fused and applied to the original feature map to enhance key channels:

[0129]

[0130] Among them, MaxPool(·) and AvgPool(·) correspond to the maximum pooling and average pooling operations in the spatial dimension respectively, the symbol σ(·) represents the sigmoid activation function, and FC(·) represents the fully connected layer. Represents the dot product operation.

[0131] Step 3.2.3, Spatial Attention: Spatial attention highlights important locations in the feature map. Specifically, the map after channel attention is compressed along the channel dimension through maximum and average pooling, resulting in a spatial feature map. These maps are added to generate attention weights, which are applied to the spatial dimension:

[0132]

[0133] Among them, MaxPool(·) and $AvgPool(·) correspond to the maximum pooling and average pooling operations along the channel dimension, respectively.

[0134] Step 3.3, Grade Evaluation: Input feature F SAThrough 3×3 convolution mapping, local spatial information is captured. At the same time, batch normalization and ReLu activation function are used to ensure the stability of training and introduce nonlinearity. Then, the residual block is used to further enhance the capture of deep semantic information to ensure that the model can learn complex feature relationships at a deeper level. Subsequently, adaptive average pooling is applied to downsample the feature map to a size of 1×1 to unify the dimension of the features. Finally, the features are projected to the corresponding hierarchical space through the fully connected layer, and the results are normalized using the softmax function to obtain the probability distribution of each level. The highest probability is the severity level of the assessment, which is defined as follows:

[0135] level=Softmax(FC(AvgPool(Block(BN(Relu(Conv(F SA )))))))

[0136] Among them, Conv(·) represents the convolution operation, Relu(·) represents the ReLu activation function, BN(·) represents the batch normalization process, Block(·) represents the residual block, AvgPool(·) corresponds to the adaptive average pooling operation, and FC(·) is called the fully connected layer.

[0137] Step 3.4: Training the hierarchical lesion fusion assessment network. Plant disease severity is divided into six levels, corresponding to a multi-class classification task. To optimize the network, a cross-entropy loss is applied, which is defined as follows:

[0138]

[0139] Where N is the number of categories, y i is the true label, is the probability distribution predicted by the model.

[0140] Step 4. Network Evaluation: Given that the two trained networks perform different tasks, targeted evaluation metrics are selected to measure their performance. For the location-guided lesion representation learning network, two commonly used evaluation metrics are used: intersection over union (IoU) and overall accuracy (OA). In contrast, the performance of the hierarchical lesion fusion evaluation network is evaluated using accuracy. First, the relevant terms are defined as follows:

[0141] True positives (TP) refer to the number of correctly identified positive samples, indicating that the model accurately predicts positive examples.

[0142] False positives (FP) represent the number of negative samples that are incorrectly classified as positive examples by the model.

[0143] True negatives (TN) correspond to the number of correctly identified negative examples, where the model accurately predicts negative examples.

[0144] False negatives (FN) reflects the number of positive samples that the model incorrectly classifies as negative examples.

[0145] Specifically, IoU is an indicator that measures the degree of overlap between the predicted result and the true label, and is defined as follows:

[0146]

[0147] OA directly reflects the proportion of correct classifications of the model on all test samples and represents the overall performance of the model. OA is defined as follows:

[0148]

[0149] Accuracy is a basic indicator for measuring the performance of a classification model, reflecting the model's ability to correctly classify a given sample set. Accuracy is defined as:

[0150]

[0151] Among them, T a represents the number of samples correctly classified by the model in the test set, T n Represents the total number of samples in the test set.

[0152] Step 5. Test: Input the leaf photos of the test set into the hierarchical lesion feature fusion evaluation network to evaluate the severity. Finally, the effectiveness of the model is verified by comparing with the real labels.

[0153] Experiment 1: Quantitative Comparison: A location-guided lesion representation network was pre-trained on a dataset containing healthy and diseased image pairs. Subsequently, a hierarchical lesion fusion evaluation network was trained and evaluated using a real dataset annotated with disease severity, and its performance was compared with twelve classification models. Two sets of experiments were conducted on the training set, one using the real dataset and the other using the generated dataset. During the testing phase, each experiment was performed on three separate test sets: the real dataset, the generated dataset, and a hybrid dataset consisting of the two. The results are shown in Table 2, revealing the following key advantages:

[0154] Results 1.1: Our method consistently outperforms other models, whether trained on real or generated data. This superior performance highlights our ability to mitigate background noise through location-guided representation learning, focusing on extracting features from lesion regions and capturing rich semantic information, leading to more accurate evaluation results.

[0155] Results 1.2: Models trained on real data and tested on generated or mixed data outperformed other methods. Similarly, when trained on generated data and tested on real or mixed data, our method consistently achieved better results, demonstrating strong generalization capabilities.

[0156] Table 2

[0157]

[0158] Experiment 2, Single Plant Experiment: To further verify the performance of the present invention in assessing the severity of a single plant disease, the aim is to demonstrate the applicability of the method in a specific plant scenario. The single dataset used in this invention takes apples, tomatoes, and potatoes as examples. The experimental design includes two sets of comparative experiments: the first set trains and tests each plant separately, and the second set trains on the total dataset of all plants and tests on a single plant. The experimental results are shown in Table 3. The results of the subset show that:

[0159] Results 2.1: Although the performance differences between various methods are small, the proposed method still shows obvious advantages on the apple and tomato datasets, reflecting its good generalization ability between different data.

[0160] Results 2.2: When trained on the full dataset and tested on individual plant datasets, our proposed method significantly outperformed other models across multiple crop categories, particularly on the tomato and potato datasets. This result demonstrates that our proposed method has strong feature extraction and classification capabilities when processing multi-category plant leaf disease images and can adapt to the characteristics of different plants.

[0161] Table 3

[0162]

[0163] Experiment 3. Position Guidance: To verify the accuracy of the present invention in locating the lesion area, the performance indicators of the pre-trained lesion guidance representation learning network were evaluated and the network output was visualized. The pre-trained lesion guidance representation learning network was tested and the IoU value, OA value, and F1 score of 0.934, 0.961, and 0.9615 were obtained, demonstrating the excellent performance of the method in accurately locating the lesion area. The visualization results of the lesion area positioning of the three plants are shown in Figure 2. Figure 4 As shown in the figure, we can see that:

[0164] Results 3.1: The present invention achieves accurate localization of the lesion area, which is significantly better than the ResNet18 model and is basically consistent with the facts.

[0165] Results 3.2: The generated images of diseased leaves and healthy leaves differ only in the lesion area, forming the healthy-disease image pairs required by the lesion-guided representation learning network, effectively guiding the network to learn disease features.

[0166] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A plant leaf disease severity assessment method based on location-guided lesion representation learning, characterized by: include: Acquire diseased leaf images; Inputting the diseased leaf image into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, the real training set is a leaf image labeled with disease severity, the hierarchical lesion fusion assessment network model includes a feature encoder with prior knowledge, a hierarchical lesion feature fusion module and an assessment module, and the feature encoder with prior knowledge is obtained by inputting an image pair into a position-guided lesion representation learning network model for training; Acquiring feature encoders with prior knowledge includes: Constructing the position-guided lesion representation learning network model, wherein the position-guided lesion representation learning network model is an encoder-decoder based model, the encoder is a dual-branch feature encoder, and the decoder is a differential positioning decoder; The position-guided lesion representation learning network model is trained using the image pairs to obtain the trained position-guided lesion representation learning network model, and the trained dual-branch feature encoder is frozen as the feature encoder with prior knowledge, wherein the training includes: The image pair is input into a dual-branch feature encoder for feature extraction, comprising: Extracting hierarchical fine-grained features from the image pair using a hierarchical information extractor to obtain fine-grained semantic features; extracting high-level semantic features from the image pairs using an information aggregator to obtain high-level semantic features; Inputting the fine-grained semantic features and the high-level semantic features into a cross-layer feature fusion module for feature fusion to obtain fused features, wherein the cross-layer feature fusion module is based on a plurality of cross-attention modules, and the fused features and the hierarchical fine-grained features are features of the image pair; Inputting the fine-grained semantic features and the high-level semantic features into a cross-layer feature fusion module for feature fusion includes: Multiplying the fine-grained semantic features of each layer by the high-level semantic features and dividing the result by the scaling value to obtain the attention weight; The attention weight is multiplied by the fine-grained semantic features of each layer to obtain the attention matrix of each layer and connected along the channel dimension to obtain the fusion feature.

2. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 1, characterized in that: Training the position-guided lesion representation learning network model using the image pairs includes: Inputting a healthy leaf image into a diffusion model to generate a diseased leaf image, and constructing the image pair based on the healthy leaf image and the corresponding diseased leaf image; Inputting the image pair into a dual-branch feature encoder to perform feature extraction to obtain features of the image pair; Inputting the features of the image pair into the differential positioning decoder, locating the lesion area by calculating the difference between the image pair, and obtaining a lesion area prediction result; Based on the lesion area prediction result and the actual lesion area, the position-guided lesion representation learning network model is trained through the constraints of the loss function.

3. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 2, characterized in that: Obtaining lesion area prediction results includes: Calculating the features of the image pair using a difference method and performing fusion and convolution operations to obtain differential features; Aggregating the differential features through cascaded upsampling and addition operations and restoring them to the original resolution; The original resolution feature map is converted into a binary map through an activation function and a preset threshold to generate the lesion area prediction result.

4. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 1, characterized in that: Inputting the diseased leaf image into the hierarchical lesion fusion assessment network model to obtain the leaf disease severity assessment result includes: Inputting the diseased leaf image and the copied diseased leaf image into the feature encoder with prior knowledge to perform feature extraction to obtain diseased leaf features; Inputting the diseased leaf features into the hierarchical lesion feature fusion module for feature fusion and enhancement to obtain fusion-enhanced features; The fused enhanced features are input into the evaluation module to obtain the leaf disease severity evaluation result.

5. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 4, characterized in that: Inputting the diseased leaf features into the hierarchical lesion feature fusion module for feature fusion and enhancement includes: The diseased leaf features are fused by an addition operation, and the fused diseased leaf features are input into a channel attention module to enhance the channel features to obtain a channel feature map; The channel feature map is input into the spatial attention module to enhance the spatial features and obtain a fusion enhancement map.

6. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 4, characterized in that: Inputting the fused enhanced features into an evaluation module to obtain the leaf disease severity evaluation result includes: The fused enhanced features are processed in sequence by convolution layer, batch normalization and ReLu activation function, residual block, adaptive average pooling layer, fully connected layer and softmax function to obtain the probability distribution of disease severity at each level, and the disease severity level corresponding to the highest probability is used as the leaf disease severity assessment result.

Images