Plant leaf disease severity assessment method based on position-guided focus characterization learning

Abstract

The invention relates to the technical field of computer vision, in particular to a plant leaf disease severity assessment method based on position-guided focus characterization learning, which comprises the following steps: acquiring a disease leaf image; the disease leaf image is input into a hierarchical lesion fusion evaluation network model, a leaf disease severity evaluation result is obtained, the hierarchical lesion fusion evaluation network model is obtained based on training of a real training set, the real training set is a leaf image marked with the disease severity, and the leaf disease severity evaluation result is obtained; the hierarchical lesion fusion evaluation network model comprises a feature encoder with priori knowledge, a hierarchical lesion feature fusion module and an evaluation module, the feature encoder with priori knowledge is obtained by inputting an image pair into a position-guided lesion characterization learning network for training, the image pair refers to health and disease states of the same leaf, and the evaluation module refers to the position-guided lesion characterization learning network. The leaf disease state is a lesion leaf generated from a healthy leaf through a diffusion model. According to the invention, more accurate positioning and severity evaluation can be carried out on the focus area.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for evaluating the severity of plant leaf diseases based on position-guided lesion representation learning. Background Art

[0002] Crop yields are often threatened by various plant diseases, with annual yield reductions of approximately 13% to 22%, resulting in significant agricultural losses. Deep learning technology has made significant progress in plant disease identification, with an accuracy rate of over 90% due to its powerful feature extraction and automatic learning capabilities, but it is limited to disease identification and cannot provide precise guidance on pesticide dosage. This limitation has led to problems such as reduced crop yields, environmental pollution, economic losses, and missed optimal treatment periods. Therefore, accurate assessment of the severity of plant diseases is essential to achieve intelligent crop management and increase food production.

[0003] Traditional methods for assessing plant disease severity often rely on the experience and judgment of agricultural experts or laboratory biochemical analysis. Although effective in some cases, these methods are limited by the subjectivity of human judgment and labor intensity and are not suitable for large-scale assessment. With the advancement of computer vision, image processing technology is now widely used in plant disease severity assessment, and machine learning methods can be used to automatically process disease information through image analysis. However, existing machine learning methods have difficulty in accurately assessing disease severity in complex field environments because their performance is highly dependent on high-quality feature engineering.

[0004] In recent years, with the rapid development of computer vision, convolutional neural networks (CNN) and visual transformers (ViTs) have been increasingly used in plant disease severity assessment, showing higher automation and accuracy. These methods extract image features through deep models, overcome the reliance of traditional methods on manual feature extraction, and show greater potential. However, these methods mainly rely on overall image features, and it is easy to mistake non-lesion areas such as background information as lesion areas, resulting in deviations in severity assessment.

[0005] Therefore, developing a method that can more accurately locate and assess the severity of the diseased area is of great significance for improving agricultural intelligent management and crop health monitoring. Summary of the invention

[0006] The purpose of the present invention is to provide a plant leaf disease severity assessment method based on position-guided lesion representation learning, which is suitable for severity assessment of various plant leaf diseases and can be migrated to fruit disease severity assessment.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] The plant leaf disease severity assessment method based on location-guided lesion representation learning includes:

[0009] Acquire diseased leaf images;

[0010] The diseased leaf image is input into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, the real training set is a leaf image with annotated disease severity, and the hierarchical lesion fusion assessment network model is constructed based on a feature encoder with prior knowledge, and the feature encoder with prior knowledge is obtained by inputting image pairs into a position-guided lesion representation learning network for training.

[0011] Optionally, obtaining a feature encoder with prior knowledge includes:

[0012] Constructing a location-guided lesion representation learning network model, wherein the location-guided lesion representation learning network model is an encoder-decoder based model, the encoder is a dual-branch feature encoder, and the decoder is a differential positioning decoder;

[0013] The position-guided lesion representation learning network model is trained using the image to obtain the trained position-guided lesion representation learning network model, and the trained dual-branch feature encoder is frozen as the feature encoder with prior knowledge.

[0014] Optionally, using the image to train the position-guided lesion representation learning network model includes:

[0015] Inputting the healthy leaf image into the diffusion model to generate the diseased leaf image, and constructing the image pair based on the healthy leaf image and the corresponding diseased leaf image;

[0016] Inputting the image pair into a dual-branch feature encoder to perform feature extraction to obtain features of the image pair;

[0017] Inputting the features of the image pair into the differential positioning decoder, locating the lesion area by calculating the difference between the image pair, and obtaining a lesion area prediction result;

[0018] Based on the lesion area prediction result and the actual lesion area, the position-guided lesion representation learning network model is trained through the constraint of the loss function.

[0019] Optionally, inputting the image pair into a dual-branch feature encoder for feature extraction includes:

[0020] Extracting hierarchical fine-grained features from the image pair using a hierarchical information extractor to obtain fine-grained semantic features;

[0021] Extracting high-level semantic features from the image pair using an information aggregator to obtain high-level semantic features;

[0022] The fine-grained semantic features and the high-level semantic features are input into a cross-layer feature fusion module for feature fusion to obtain fused features, wherein the cross-layer feature fusion module is obtained based on a number of cross-attention modules, and the fused features and the hierarchical fine-grained features are features of an image pair.

[0023] Optionally, inputting the fine-grained semantic features and the high-level semantic features into a cross-layer feature fusion module for feature fusion includes:

[0024] Multiply the fine-grained semantic features of each layer by the high-level semantic features and divide the result by the scaling value to obtain an attention weight;

[0025] The attention weight is multiplied by the fine-grained semantic features of each layer to obtain the attention matrix of each layer and connect them along the channel dimension to obtain the fusion feature.

[0026] Optionally, obtaining the lesion area prediction result includes:

[0027] The features of the image pair are calculated by using a difference method and then fused and convolved to obtain a difference feature;

[0028] Aggregating the differential features through cascaded upsampling operations and addition operations, and restoring them to the original resolution;

[0029] The original resolution feature map is converted into a binary map through an activation function and a preset threshold to generate the lesion area prediction result.

[0030] Optionally, inputting the diseased leaf image into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result includes:

[0031] Inputting the diseased leaf image and the copied diseased leaf image into the feature encoder with prior knowledge to extract features, thereby obtaining diseased leaf features;

[0032] Inputting the diseased leaf features into a hierarchical lesion feature fusion module for feature fusion and enhancement to obtain fusion enhanced features;

[0033] The fused enhanced features are input into an evaluation module to obtain the leaf disease severity evaluation result.

[0034] Optionally, inputting the diseased leaf features into a hierarchical lesion feature fusion module for feature fusion and enhancement includes:

[0035] The diseased leaf features are fused by an addition operation, and the fused diseased leaf features are input into a channel attention module to enhance the channel features, thereby obtaining a channel feature map;

[0036] The channel feature map is input into the spatial attention module to enhance the spatial features and obtain a fused enhanced map.

[0037] Optionally, inputting the fused enhanced features into an evaluation module to obtain the leaf disease severity evaluation result includes:

[0038] The fused enhanced features are processed in sequence by convolution layer, batch normalization and ReLu activation function, residual block, adaptive average pooling layer, fully connected layer and softmax function to obtain the probability distribution of each level of disease severity, and the disease severity level corresponding to the highest probability is used as the leaf disease severity assessment result.

[0039] Compared with the prior art, the present invention has the following advantages and technical effects:

[0040] The present invention aims at the problem that traditional deep learning methods rely on global image features and easily mistake background information for lesion areas, resulting in deviations in severity assessment. A position-guided lesion representation learning evaluation strategy is proposed. This strategy focuses on extracting features from the lesion area and further evaluating the severity of the disease based on these features. Specifically, the present invention first uses a diffusion model to generate diseased leaves from healthy leaves to construct image pairs of healthy-diseased leaves. Then, the position-guided disease representation learning network trains the network to identify and locate the lesion area on the leaf by comparing healthy and diseased leaves, thereby obtaining an encoder with prior knowledge. Finally, the hierarchical disease fusion evaluation network shares and freezes the encoder, extracts features focused on the lesion area, and fuses and classifies these features through a hierarchical strategy to achieve a quantitative assessment of the severity of the disease. Through the above method, the present invention effectively eliminates the interference of background information, accurately performs feature learning of the lesion area, and thus significantly improves the accuracy of disease severity assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1This is a general network architecture diagram of a method for evaluating plant leaf disease severity based on location-guided lesion representation learning according to an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of a position-guided representation learning network according to an embodiment of the present invention;

[0044] Figure 3 It is a schematic diagram of the structure of a hierarchical lesion fusion assessment network according to an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of visualization results of positioning three plant lesion regions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] This embodiment provides a method for assessing the severity of plant leaf diseases based on location-guided lesion representation learning, including:

[0049] Acquire diseased leaf images;

[0050] The diseased leaf image is input into the hierarchical lesion fusion assessment network model to obtain the leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, and the real training set is a leaf image with annotated disease severity. The hierarchical lesion fusion assessment network model includes a feature encoder with prior knowledge, a hierarchical lesion feature fusion module and an assessment module. The feature encoder with prior knowledge is obtained by inputting image pairs into the position-guided lesion representation learning network model for image pair training.

[0051] Specifically, the training of this embodiment requires a total of two data sets, one is a plant leaf health or disease data set in a real environment for the final severity assessment, and the other is an image pair constructed based on a generative model.

[0052] Realistic data set construction: The data set is selected from the real world. After the data is selected, the data set is manually labeled with severity. During the training process, the present invention sets the division ratio of the training set to the test set to 8:2 to train the hierarchical lesion fusion evaluation network.

[0053] Construction of image pairs: Use the trained diffusion model to generate plant disease images from healthy leaf images, and make one-to-one correspondence with healthy leaves to form image pairs. The training set and test set are divided into training sets and test sets in a ratio of 8:2, and the training positions are used to guide the lesion representation learning network.

[0054] Furthermore, obtaining a feature encoder with prior knowledge includes:

[0055] Constructing a position-guided lesion representation learning network model, wherein the position-guided lesion representation learning network model is an encoder-decoder based model, the encoder is a dual-branch feature encoder, and the decoder is a differential positioning decoder;

[0056] The image is used to train the position-guided lesion representation learning network model, the trained position-guided lesion representation learning network model is obtained, and the trained dual-branch feature encoder is frozen as a feature encoder with prior knowledge.

[0057] Specifically, the position-guided lesion representation learning network: the constructed image pairs are input into the position-guided lesion representation learning network, the features are extracted using a dual-branch feature encoder, and the lesion area is decoded through a differential positioning decoder. After learning, a feature encoder with prior knowledge can be obtained.

[0058] Furthermore, using images to guide the location of lesion representation learning network models to train includes:

[0059] The healthy leaf image is input into the diffusion model to generate the diseased leaf image, and an image pair is constructed based on the healthy leaf image and the corresponding diseased leaf image;

[0060] Input the image pair into the dual-branch feature encoder for feature extraction to obtain the features of the image pair;

[0061] The features of the image pair are input into the differential positioning decoder, and the lesion area is located by calculating the difference between the image pairs to obtain the lesion area prediction result;

[0062] Based on the lesion area prediction results and the actual lesion area, the lesion representation learning network model is guided by the constrained training position of the loss function.

[0063] Specifically, the location guides the training of the lesion representation learning network: In order to constrain the model, binary cross entropy and Dice loss are used to optimize the localization performance of the lesion area. By training the network with the constraint of the loss function, the localization of the lesion area can be achieved, and a two-branch feature encoder with prior knowledge of the lesion area can be obtained, and its weights are saved as the encoder of the hierarchical lesion fusion evaluation network.

[0064] Furthermore, the image is input into the dual-branch feature encoder for feature extraction including:

[0065] A hierarchical information extractor is used to extract hierarchical fine-grained features from image pairs to obtain fine-grained semantic features;

[0066] Using the information aggregator to extract high-level semantic features from the image pairs, high-level semantic features are obtained;

[0067] The fine-grained semantic features and high-level semantic features are input into the cross-layer feature fusion module for feature fusion to obtain fused features, wherein the cross-layer feature fusion module is obtained based on several cross-attention modules. The final output of the encoder is an image pair feature including fused features and hierarchical fine-grained features.

[0068] Specifically, the extraction of hierarchical fine-grained features: The hierarchical feature extractor is similar in structure to ResNet18, which can retain visual semantic information at each level while maintaining computational efficiency. It adopts a strategy of training from scratch, which is specifically used to extract detail information from the input image and generate features of different resolutions: 256×256, 128×128, and 64×64.

[0069] Extraction of high-level semantic information: The information aggregator uses a ResNet18 architecture pre-trained on 421,133 plant disease images and fine-tuned to adapt to the new task during model training. Its output is a feature of size 32×32.

[0070] Furthermore, the fine-grained semantic features and high-level semantic features are input into the cross-layer feature fusion module for feature fusion, including:

[0071] Multiply the fine-grained semantic features of each layer with the high-level semantic features and divide them by the scaling value to obtain the attention weight;

[0072] Multiply the attention weights with the fine-grained semantic features of each layer to obtain the attention matrix of each layer and connect them along the channel dimension to obtain the fusion features.

[0073] Specifically, the cross-layer feature fusion module consists of three cross-attention blocks, which refine the hierarchical features by using high-level features to guide the enhancement of the focus area. The fine-grained semantic features and high-level semantic features are combined using the attention matrix, that is, the fine-grained information of each layer is multiplied by the high-level semantic information and then divided by the scaling value to obtain the attention weight. The weight is multiplied by the fine-grained information of each layer to obtain the attention matrix of each layer, and then connected along the channel dimension.

[0074] Furthermore, obtaining the lesion area prediction results includes:

[0075] The features of the image pair are calculated using the difference method and then fused and convolved to obtain the difference features;

[0076] The differential features are aggregated through cascaded upsampling and addition operations and restored to the original resolution;

[0077] The original resolution feature map is converted into a binary image through the activation function and the preset threshold to generate the lesion area prediction result.

[0078] Specifically, the features of the extracted image pairs are input into the differential positioning decoder, and the feature differences between healthy leaves and diseased leaves are directly calculated to locate the lesion area. Then, the original resolution is restored through upsampling and fusion, and the lesion area prediction result is obtained after activation.

[0079] Furthermore, the diseased leaf image is input into the hierarchical lesion fusion assessment network model to obtain the leaf disease severity assessment results including:

[0080] The diseased leaf image and the replicated diseased leaf image are input into a feature encoder with prior knowledge to extract features, thereby obtaining the diseased leaf features;

[0081] The diseased leaf features are input into the hierarchical lesion feature fusion module for feature fusion and enhancement to obtain fusion enhanced features;

[0082] The fused enhanced features are input into the evaluation module to obtain the leaf disease severity assessment results.

[0083] Specifically, the diseased leaf image can be input into the hierarchical lesion fusion assessment network to complete the severity assessment of the leaf. It integrates a pre-trained dual-branch feature encoder for feature extraction, uses the hierarchical lesion fusion module to fuse and enhance the features, and finally obtains the corresponding grade through the assessment module.

[0084] Furthermore, the diseased leaf features are input into the hierarchical lesion feature fusion module for feature fusion and enhancement including:

[0085] The diseased leaf features are fused through addition operation, and the fused features of the diseased leaves are input into the channel attention module to enhance the channel features and obtain the channel feature map;

[0086] The channel feature map is input into the spatial attention module to enhance the spatial features and obtain the fused enhanced map.

[0087] Specifically, feature fusion enhancement: the hierarchical lesion information fusion module is used to fuse the two types of extracted features, and the channel attention and spatial attention are cascaded to enhance the focus on key areas.

[0088] Feature fusion: Through adaptive adjustment, the extracted fine-grained hierarchical features and the high-level semantic features extracted and enhanced by the cross-layer feature fusion module are adjusted to the same scale, and then the two types of features are fused through an addition operation.

[0089] Channel attention: The channel attention mechanism assigns weights to feature map channels to highlight important channels. Specifically, the fused feature maps undergo global max and average pooling to capture channel context, which is passed through the fully connected layer. The resulting weights are fused and applied to the fused feature map of the feature fusion step to enhance key channels.

[0090] Spatial Attention: Spatial attention highlights important locations in the feature map. Specifically, the map after channel attention is compressed along the channel dimension through maximum and average pooling to produce spatial feature maps. These maps are added to generate attention weights, which are applied to the spatial dimensions of the feature map obtained by the channel attention step.

[0091] Furthermore, the fused enhanced features are input into the evaluation module to obtain the leaf disease severity evaluation results including:

[0092] The fused enhanced features are processed sequentially through the convolution layer, batch normalization and ReLu activation function, residual block, adaptive average pooling layer, fully connected layer and softmax function to obtain the probability distribution of each level of disease severity, and the disease severity level corresponding to the highest probability is taken as the leaf disease severity assessment result.

[0093] Specifically, level evaluation: the step output features are mapped through 3×3 convolution to capture local spatial information. At the same time, batch normalization and ReLu activation function are used to ensure the stability of training and introduce nonlinearity. Then, the residual block is used to further enhance the capture of deep semantic information to ensure that the model can learn complex feature relationships at a deeper level. Subsequently, adaptive average pooling is applied to downsample the feature map to a size of 1×1 to unify the dimension of the features. Finally, the features are projected to the corresponding hierarchical space through a fully connected layer, and the results are normalized using the softmax function to obtain the probability distribution of each level. The highest probability is the severity level of the assessment.

[0094] In this embodiment, the severity of plant diseases is divided into six levels, corresponding to multi-class classification tasks. In order to optimize the hierarchical lesion fusion evaluation network, cross entropy loss is applied.

[0095] The method of this embodiment is further described in detail below:

[0096] The network model used in the plant leaf disease severity assessment method based on position-guided lesion representation learning in this embodiment is composed of three main parts: an image generation network, a position-guided lesion representation learning network, and a hierarchical lesion fusion assessment network. Figure 1 . The image generation network uses a diffusion model to generate diseased leaves from healthy leaves to construct paired images required by the location-guided lesion representation learning network. The location-guided lesion representation learning network uses an encoder-decoder architecture to identify and locate lesion areas on leaves by comparing healthy-lesion paired photos, and the encoder is trained using prior knowledge of plant lesion areas. The hierarchical lesion fusion assessment network uses an encoder trained by the learning network to extract lesion-specific features, and then uses a hierarchical strategy to fuse and classify these features to provide a quantitative assessment of disease severity.

[0097] The plant leaf disease severity assessment method based on position-guided lesion representation learning specifically includes the following steps:

[0098] Step 1: Construct a data set. The training of this embodiment requires two data sets in total, one is a plant leaf health or disease data set in a real environment for the final severity assessment, and the other is an image pair constructed based on a generative model.

[0099] Step 1.1, construction of real data set. The data set is selected from the real world. After the data is selected, the data set is manually labeled with severity. The classification standards used are shown in Table 1. During the training process, the present invention sets the division ratio of the training set to the test set to 8:2.

[0100] Step 1.2: Construction of image pairs: Use the trained image generation network (diffusion model) to generate plant disease images from healthy leaf images, and make one-to-one correspondence with healthy leaves to form image pairs.

[0101] Table 1

[0102]

[0103] Step 2: Location-guided lesion representation learning, see Figure 2 The image pair is input into the position-guided lesion representation learning network, which is an encoder-decoder based model. Specifically, a dual-branch feature encoder is used to extract features, and the lesion area is decoded by a differential positioning decoder. After learning, a feature encoder with prior knowledge can be obtained.

[0104] Step 2.1, feature extraction. The image pair is input into the dual-branch feature encoder to obtain two types of feature information, namely, fine-grained and high-level semantic features are extracted from the image pair using a parallel hierarchical feature extractor and an information aggregator (also known as an information refiner), and these features are integrated through a cross-layer feature fusion module to enhance the focus on the lesion area.

[0105] Step 2.1.1: Hierarchical fine-grained features The hierarchical feature extractor is similar to ResNet18 in structure and can preserve visual semantic information at all levels while maintaining computational efficiency. Specifically, the hierarchical feature extractor uses the first three layers of ResNet18 to generate features of different resolutions. 256×256, 128×128 and 64×64. The strategy adopted is to train from scratch, which is specifically used to extract detail information from the input image. Step 2.1.2, high-level semantic information F A The information aggregator uses a four-layer network of ResNet18, which uses the ResNet18 architecture pre-trained on 421,133 plant disease images and fine-tuned during model training to adapt to the new task. Its output is a feature F of size 32×32 A .

[0106] Step 2.1.3, feature enhancement. The cross-layer feature fusion module consists of three cross-attention blocks, which refines the hierarchical features by using high-level features to guide the enhancement of the focus area. and F A The output features of each layer Respectively with F A Multiply and divide by the scale value Get the attention weight, multiply the weight by The attention matrices of each layer can be obtained and then connected along the channel dimension. The process is defined as:

[0107] F C =Concat(F 1 ,F 2 ,F 3 )

[0108]

[0109] Among them, Concat(·) represents the connection in the channel dimension, F i Represents three levels of fine-grained features With high-level semantic features F A The result after cross attention, F C It means concatenating the results of the three attention operations along the channel dimension.

[0110] Step 2.2: Decode the lesion area. The features of the image pair are input into the differential positioning decoder, and the feature differences between healthy leaves and diseased leaves are directly calculated to locate the lesion area. The original resolution is then restored through upsampling and fusion, and the lesion area prediction result is obtained after activation.

[0111] Specifically, the output of the dual-branch feature encoder is calculated, that is, the four-layer healthy leaf feature and leaf characteristics of diseased spots The difference between the two layers is obtained by using the difference method, and the original features are connected and fused with the difference features. Then, the fused features are convolved to obtain the difference features of each layer. Then, through cascade upsampling and addition operations, the differential features from deep to shallow layers are gradually aggregated and finally restored to the original resolution H×W. Finally, the original resolution feature map is converted into Convert it into a binary image and generate the prediction result P of the lesion area. The specific process is defined as follows:

[0112]

[0113] Among them, MLP is a three-layer convolutional network, |·| represents the absolute value operation, and US(·) is a convolution with a kernel size of 1×1, which is used to reduce the channel dimension and improve the resolution.

[0114] Step 2.3: Position-guided lesion representation learning network training. In order to constrain the model, binary cross entropy and Dice loss are used to optimize the localization performance of the lesion area. Binary cross entropy loss L bceIt is used to measure the accuracy of the model in the binary classification task so that the model network can better identify the pixel-level lesion area. It is defined as follows:

[0115]

[0116] Among them, P represents the predicted value, Y represents the label value, N represents the total number of pixels, and P i Represents the prediction result of the i-th pixel, Y i Represents the true value of the pixel.

[0117] Dice loss L dice It is used to solve the problem of unbalanced data sets, especially when the ratio of lesion area to background area is unbalanced. It is defined as follows:

[0118]

[0119] Here, ε is a smoothing term used to avoid zero division.

[0120] The objective function is composed of binary cross entropy loss and Dice loss to form a hybrid loss function, which can effectively balance the pixel classification accuracy and region overlap optimization in the segmentation task, and is defined as follows: L LGRL =L bce (P,Y)+L dice (P,Y)

[0121] By training the network with the constraints of the loss function, the lesion area can be located, and a two-branch feature encoder with prior knowledge of the lesion area can be obtained, and its weights are saved as the encoder of the hierarchical lesion fusion evaluation network.

[0122] Step 3: Severity Assessment, see Figure 3 The severity of the leaf can be assessed by inputting the diseased leaf image into the hierarchical lesion fusion assessment network. It integrates a pre-trained dual-branch feature encoder for feature extraction, uses a hierarchical lesion fusion module to fuse and enhance the features, and finally obtains the corresponding grade through the assessment module.

[0123] Step 3.1, feature extraction: directly use the pre-trained two-branch feature encoder for feature extraction, and freeze its parameters to retain the prior knowledge of the lesion area, so as to achieve efficient positioning of the diseased area. The assessment of severity only requires the input of a diseased leaf image. In order to maintain the original input image pair structure of the encoder, the paired input is constructed by copying the diseased leaf image.

[0124] Step 3.2, feature fusion enhancement: Use the hierarchical lesion information fusion module to fuse the two types of features extracted by the encoder, and cascade channel attention and spatial attention to enhance the focus on key areas.

[0125] Step 3.2.1, feature fusion: Through adaptive adjustment, the fine-grained hierarchical features extracted from the hierarchical feature extractor are The high-level semantic features F extracted from the information aggregator and enhanced by the cross-layer feature fusion module C Adjust to the same scale, and then fuse the two types of features through addition operation. The detailed process is as follows:

[0126]

[0127] Where DS(·) represents adaptive downsampling.

[0128] Step 3.2.2, Channel Attention: The channel attention mechanism assigns weights to feature map channels to highlight important channels. Specifically, the fused feature map undergoes global maximum and average pooling to capture channel context, which is passed through the fully connected layer. The resulting weights are fused and applied to the original feature map to enhance key channels:

[0129]

[0130] Among them, MaxPool(·) and AvgPool(·) correspond to the maximum pooling and average pooling operations in the spatial dimension respectively, the symbol σ(·) represents the sigmoid activation function, and FC(·) represents the fully connected layer. Represents the dot product operation.

[0131] Step 3.2.3, spatial attention: Spatial attention highlights important locations in the feature map. Specifically, the map after channel attention is compressed along the channel dimension through maximum and average pooling to produce spatial feature maps. These maps are added to generate attention weights, which are applied to the spatial dimension:

[0132]

[0133] Among them, MaxPool(·) and $AvgPool(·) correspond to the maximum pooling and average pooling operations along the channel dimension respectively.

[0134] Step 3.3, level evaluation: Input feature F SAThrough 3×3 convolution mapping, local spatial information is captured. At the same time, batch normalization and ReLu activation function are used to ensure the stability of training and introduce nonlinearity. Then, the residual block is used to further enhance the capture of deep semantic information to ensure that the model can learn complex feature relationships at a deeper level. Subsequently, adaptive average pooling is applied to downsample the feature map to a size of 1×1 to unify the dimension of the features. Finally, the features are projected to the corresponding hierarchical space through a fully connected layer, and the results are normalized using the softmax function to obtain the probability distribution of each level. The highest probability is the severity level of the assessment, defined as follows:

[0135] level=Softmax(FC(AvgPool(Block(BN(Relu(Conv(F SA )))))))

[0136] Among them, Conv(·) represents the convolution operation, Relu(·) represents the ReLu activation function, BN(·) represents the batch normalization process, Block(·) represents the residual block, AvgPool(·) corresponds to the adaptive average pooling operation, and FC(·) is called the fully connected layer.

[0137] Step 3.4: Training of hierarchical lesion fusion assessment network. The severity of plant diseases is divided into six levels, corresponding to multi-class classification tasks. To optimize the network, cross entropy loss is applied, which is defined as follows:

[0138]

[0139] Where N is the number of categories, y i is the true label, is the probability distribution predicted by the model.

[0140] Step 4. Network evaluation: Given that the two trained networks perform different tasks, targeted evaluation metrics are selected to measure their performance. For the location-guided lesion representation learning network, two commonly used evaluation metrics are used: intersection over union (IoU) and overall accuracy (OA). In contrast, the performance of the hierarchical lesion fusion evaluation network is evaluated using accuracy. First, the relevant terms are defined as follows:

[0141] True positives (TP) refer to the number of correctly identified positive samples, indicating that the model accurately predicts positive examples.

[0142] False Positives (FP) represent the number of negative samples that are incorrectly classified as positive examples by the model.

[0143] True negatives (TN) correspond to the number of correctly identified negative examples, where the model accurately predicts negative examples.

[0144] False negatives (FN) reflect the number of positive samples that the model incorrectly classified as negative examples.

[0145] Specifically, IoU is an indicator to measure the degree of overlap between the predicted result and the true label, which is defined as follows:

[0146]

[0147] OA directly reflects the proportion of the model's correct classification on all test samples and represents the overall performance of the model. OA is defined as follows:

[0148]

[0149] Accuracy is a basic indicator to measure the performance of a classification model, reflecting the model's ability to correctly classify a given sample set. Accuracy is defined as:

[0150]

[0151] Among them, T a represents the number of samples correctly classified by the model in the test set, T n Represents the total number of samples in the test set.

[0152] Step 5, testing: Input the leaf photos of the test set into the hierarchical lesion feature fusion evaluation network for severity assessment, and finally verify the effectiveness of the model by comparing with the real labels.

[0153] Experiment 1: Quantitative comparison: The location-guided lesion representation network was pre-trained on a dataset containing healthy and diseased image pairs. Subsequently, the hierarchical lesion fusion evaluation network was trained and evaluated using a real dataset annotated with disease severity, and its performance was compared with twelve classification models. Two sets of experiments were conducted on the training set, one using the real dataset and the other using the generated dataset. In the testing phase, each experiment was performed on three separate test sets: the real dataset, the generated dataset, and a mixed dataset consisting of the two. The results are shown in Table 2, and the following key advantages can be seen:

[0154] Results 1.1. Whether trained on real data or generated data, the proposed method consistently outperforms other models. This superior performance highlights the ability of the proposed method to mitigate background noise through location-guided representation learning, focusing on extracting features from lesion areas and capturing rich semantic information, thereby obtaining more accurate evaluation results.

[0155] Results 1.2: The model trained on real data and tested on generated or mixed data outperforms other methods. Similarly, when trained on generated data and tested on real or mixed data, the technical method of the present invention consistently shows better results, demonstrating strong generalization capabilities.

[0156] Table 2

[0157]

[0158] Experiment 2, single plant experiment: In order to further verify the performance of the present invention in assessing the severity of a single plant disease, the purpose is to prove the applicability of the method in a specific plant scenario. The single data set used in the present invention takes apples, tomatoes and potatoes as examples. The experimental design includes two groups of comparative experiments: the first group trains and tests each plant separately, and the second group trains on the total data set of all plants and tests on a single plant. The experimental results are shown in Table 3. The results of the subset show that:

[0159] Results 2.1: Although the performance differences between various methods are small, the present invention still shows obvious advantages on the apple and tomato datasets, reflecting its good generalization ability between different data.

[0160] Results 2.2: When trained on the full dataset and tested on a single plant dataset, the proposed method significantly outperformed other models on multiple crop categories, especially on the tomato and potato datasets. This result shows that the proposed method has strong feature extraction and classification capabilities when processing multi-category plant leaf disease images and can adapt to the characteristics of different plants.

[0161] Table 3

[0162]

[0163] Experiment 3. Position guidance: In order to verify the accuracy of the present invention in locating the lesion area, the performance indicators of the pre-trained lesion guidance representation learning network were evaluated and the output of the network was visualized. The pre-trained lesion guidance representation learning network was tested and the IoU was 0.934, the OA value was 0.961, and the F1 score was 0.9615, which proved the excellent performance of the method in accurately locating the lesion area. The visualization results of the lesion area positioning of the three plants are shown in Figure 2. Figure 4 As shown in the figure, it can be seen that:

[0164] Results 3.1: The present invention achieves accurate positioning of the lesion area, which is significantly better than the ResNet18 model and is basically consistent with the facts.

[0165] Results 3.2: The generated diseased leaf and healthy leaf images differed only in the lesion area, forming the healthy-disease image pairs required by the lesion-guided representation learning network, effectively guiding the network to learn disease features.

[0166] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for assessing plant leaf disease severity based on location-guided lesion representation learning, characterized in that: include: Acquire diseased leaf images; The diseased leaf image is input into a hierarchical lesion fusion assessment network model to obtain a leaf disease severity assessment result, wherein the hierarchical lesion fusion assessment network model is trained based on a real training set, and the real training set is a leaf image with annotated disease severity. The hierarchical lesion fusion assessment network model includes a feature encoder with prior knowledge, a hierarchical lesion feature fusion module and an assessment module, and the feature encoder with prior knowledge is obtained by inputting an image pair into a position-guided lesion representation learning network model for training.

2. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 1, characterized in that: The feature encoders that acquire prior knowledge include: Constructing the position-guided lesion representation learning network model, wherein the position-guided lesion representation learning network model is an encoder-decoder based model, the encoder is a dual-branch feature encoder, and the decoder is a differential positioning decoder; The position-guided lesion representation learning network model is trained using the image to obtain the trained position-guided lesion representation learning network model, and the trained dual-branch feature encoder is frozen as the feature encoder with prior knowledge.

3. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 2, characterized in that: Using the image to train the location-guided lesion representation learning network model includes: Inputting the healthy leaf image into the diffusion model to generate the diseased leaf image, and constructing the image pair based on the healthy leaf image and the corresponding diseased leaf image; Inputting the image pair into a dual-branch feature encoder to perform feature extraction to obtain features of the image pair; Inputting the features of the image pair into the differential positioning decoder, locating the lesion area by calculating the difference between the image pair, and obtaining a lesion area prediction result; Based on the lesion area prediction result and the actual lesion area, the position-guided lesion representation learning network model is trained through the constraint of the loss function.

4. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 3, characterized in that: Inputting the image pair into a dual-branch feature encoder to extract features comprises: Extracting hierarchical fine-grained features from the image pair using a hierarchical information extractor to obtain fine-grained semantic features; Extracting high-level semantic features from the image pair using an information aggregator to obtain high-level semantic features; The fine-grained semantic features and the high-level semantic features are input into a cross-layer feature fusion module for feature fusion to obtain fused features, wherein the cross-layer feature fusion module is obtained based on a number of cross-attention modules, and the fused features and the hierarchical fine-grained features are features of an image pair.

5. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 4, characterized in that: Inputting the fine-grained semantic features and the high-level semantic features into a cross-layer feature fusion module for feature fusion includes: Multiply the fine-grained semantic features of each layer by the high-level semantic features and divide the result by the scaling value to obtain an attention weight; The attention weight is multiplied by the fine-grained semantic features of each layer to obtain the attention matrix of each layer and connect them along the channel dimension to obtain the fusion feature.

6. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 3, characterized in that: Obtaining the lesion area prediction results includes: The features of the image pair are calculated by using a difference method and then fused and convolved to obtain a difference feature; Aggregating the differential features through cascaded upsampling operations and addition operations, and restoring them to the original resolution; The original resolution feature map is converted into a binary map through an activation function and a preset threshold to generate the lesion area prediction result.

7. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 1, characterized in that: Inputting the diseased leaf image into the hierarchical lesion fusion assessment network model to obtain the leaf disease severity assessment result includes: Inputting the diseased leaf image and the copied diseased leaf image into the feature encoder with prior knowledge to extract features, thereby obtaining diseased leaf features; Inputting the diseased leaf features into the hierarchical lesion feature fusion module for feature fusion and enhancement to obtain fusion enhanced features; The fused enhanced features are input into the evaluation module to obtain the leaf disease severity evaluation result.

8. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 7, characterized in that: Inputting the diseased leaf features into the hierarchical lesion feature fusion module for feature fusion and enhancement includes: The diseased leaf features are fused by an addition operation, and the fused diseased leaf features are input into a channel attention module to enhance the channel features, thereby obtaining a channel feature map; The channel feature map is input into the spatial attention module to enhance the spatial features and obtain a fused enhanced map.

9. The method for assessing plant leaf disease severity based on position-guided lesion representation learning according to claim 7, characterized in that: Inputting the fused enhanced features into an evaluation module to obtain the leaf disease severity evaluation result includes: The fused enhanced features are processed in sequence by convolution layer, batch normalization and ReLu activation function, residual block, adaptive average pooling layer, fully connected layer and softmax function to obtain the probability distribution of each level of disease severity, and the disease severity level corresponding to the highest probability is used as the leaf disease severity assessment result.

Images