Apple leaf disease segmentation method based on lightweight dual-path network and related device

Abstract

The invention discloses an apple leaf disease segmentation method based on a lightweight dual-path network, an apple leaf disease segmentation device based on the lightweight dual-path network, an apple leaf disease segmentation device and a computer readable storage medium. The problems that an existing disease segmentation method is high in model complexity and insufficient in multi-scale recognition capability are effectively solved. The lightweight encoder adopts a depth separable convolution and channel recombination technology, so that the parameter quantity and the calculation complexity are greatly reduced while the feature extraction capability is maintained, and the model can be deployed on edge equipment such as an unmanned aerial vehicle and a field robot. The enhanced cavity space pyramid pooling module constructs abundant multi-scale receptive fields through multi-branch parallel cavity convolution with different expansion rates, and can capture feature information of initial tiny disease spots and later fused disease spots at the same time.

Description

Technical Field

[0001] This invention belongs to the field of agricultural information technology, and more specifically, relates to a method for segmenting apple leaf diseases based on a lightweight dual-path network, a device for segmenting apple leaf diseases based on a lightweight dual-path network, a computer-readable storage medium. Background Technology

[0002] Apples are one of my country's most important economic fruit trees, ranking first in the world in both planting area and yield. However, leaf diseases are a major factor restricting the development of the apple industry. Common diseases such as brown spot, black spot, anthracnose, rust, and gray spot severely affect the photosynthesis of fruit trees and fruit quality, leading to yield losses of 30%-50%. Traditional disease identification relies on manual inspection and expert judgment, which suffers from low efficiency, strong subjectivity, and inability to monitor large areas in real time. With the development of computer vision and deep learning technologies, intelligent disease identification methods based on image segmentation have gradually become a research hotspot, enabling automatic detection, precise location, and quantitative analysis of diseases.

[0003] Leaf disease image segmentation methods in related technologies mainly employ semantic segmentation networks, such as DeepLabV3+, PSPNet, and U-Net. While these methods have achieved certain results in segmentation accuracy, they still suffer from two major drawbacks: First, the model parameters and computational load are too large, making it difficult to deploy on edge devices with limited computing power, and thus failing to meet the needs of real-time applications such as drone inspections and field robots. Second, the ability to identify multi-scale lesions is insufficient; the scale difference between early-stage small lesions and later-stage fused lesions can exceed 20 times, and convolution with a single receptive field cannot effectively capture lesion features of different scales simultaneously, resulting in a high rate of missed detection for small lesions and blurred boundaries for large lesions.

[0004] Therefore, how to reduce the number of model parameters and computational load while improving the accuracy of multi-scale lesion identification and boundary segmentation, and realize real-time inference on edge computing devices to meet the requirements of high precision, low latency and low power consumption in smart agriculture applications is a key issue of concern to those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a method for segmenting apple leaf diseases based on a lightweight dual-path network, a device for segmenting apple leaf diseases based on a lightweight dual-path network, a computer-readable storage medium for segmenting apple leaf diseases, and a computer-readable storage medium. This method aims to reduce the number of model parameters and computational load while improving the accuracy of multi-scale lesion identification and boundary segmentation, enabling real-time inference on edge computing devices, and meeting the requirements of high precision, low latency, and low power consumption in smart agriculture applications.

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method for apple leaf disease segmentation based on a lightweight dual-path network, comprising: Acquire a leaf image, and perform contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain a preprocessed image; The preprocessed image is input into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are then fused using an enhanced hollow space pyramid pooling module to generate a binary segmentation mask for leaves and background. Based on the binary classification segmentation mask, the leaf region in the preprocessed image is located, and the located leaf region image is input into the lesion fine segmentation network. The multi-scale attention module in the lesion fine segmentation network aggregates lesion features of different scales and performs six-class semantic segmentation, including healthy leaves, brown spot, black spot, anthracnose, rust, and gray spot, to obtain lesion segmentation results. Based on the lesion segmentation results and ground truth labels, a hybrid loss function of weighted Dice loss, boundary loss, and consistency loss is used to calculate the total loss value, and the network parameters are optimized through backpropagation using the total loss value. The lesion segmentation results are output, and the area ratio of various lesions is calculated based on the pixel statistics of the segmented regions for use in disease diagnosis analysis and prevention and control decision output.

[0007] Optionally, the lightweight encoder is an EfficientNet-B3 network, which extracts features through depthwise separable convolutions and moving inverted bottleneck structures; the enhanced dilated spatial pyramid pooling module uses depthwise separable dilated convolutions with dilation rates of 6, 12, and 18 to extract multi-scale contextual information in parallel.

[0008] Optionally, after generating the binary classification mask for leaves and background, the process also includes: The edge region of the binary classification segmentation mask is refined by a boundary refinement unit. The boundary refinement unit is reduced in dimensionality by 1×1 convolution and then concatenated with the low-level feature map. Finally, it is output as a refined leaf boundary by 3×3 convolution.

[0009] Optionally, the multi-scale attention module in the lesion fine segmentation network uses three convolutional branches with kernel sizes of 1×1, 3×3, and 5×5 to extract features from different receptive fields in parallel, and performs weighted fusion of features from each branch through a channel attention mechanism; wherein, the channel attention mechanism extracts channel statistical information through global average pooling and global max pooling and then generates channel weights through a multilayer perceptron.

[0010] Optionally, the hybrid loss function is expressed as: L_total=α·L_Dice+β·L_boundary+γ·L_consistency; Wherein, L_Dice is the weighted Dice loss, which sets weight coefficients for different categories to alleviate class imbalance, L_boundary is the boundary loss used to enhance edge segmentation accuracy, L_consistency is the consistency loss used to constrain the coherence of the two-stage segmentation results, and α, β, and γ are preset coefficients.

[0011] Optional, also includes: During model training, the Adam optimizer is used, with a preset learning rate and preset batch size. When the validation set loss does not decrease for 5 consecutive rounds, the learning rate is reduced to 0.1 times the original learning rate until the validation loss converges to below 0.45 or the maximum number of training rounds of 63 is reached.

[0012] This application also provides an apple leaf disease segmentation device based on a lightweight dual-path network, comprising: The image preprocessing module is used to acquire a leaf image and perform contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain a preprocessed image. The background segmentation module is used to input the preprocessed image into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are fused by an enhanced hollow space pyramid pooling module to generate a binary segmentation mask for leaves and background. The fine segmentation module is used to locate the leaf region in the preprocessed image based on the binary classification segmentation mask, and input the located leaf region image into the lesion fine segmentation network; the multi-scale attention module in the lesion fine segmentation network aggregates lesion features of different scales, and performs six-class semantic segmentation including healthy leaves, brown spot disease, black spot disease, anthracnose, rust disease and gray spot disease to obtain the lesion segmentation result; The network parameter optimization module is used to calculate the total loss value based on the lesion segmentation results and ground truth labels, using a hybrid loss function of weighted Dice loss, boundary loss, and consistency loss, and then optimize the network parameters through backpropagation using the total loss value. The segmentation result output module is used to output the lesion segmentation results, and to calculate the area ratio of various lesions based on the pixel statistics of the segmented region, for disease diagnosis analysis and prevention and control decision output.

[0013] This application also provides an apple leaf disease segmentation device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the apple leaf disease segmentation method as described above.

[0014] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the apple leaf disease segmentation method described above.

[0015] This application provides a method for apple leaf disease segmentation based on a lightweight dual-path network, comprising: acquiring a leaf image; performing contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain a preprocessed image; inputting the preprocessed image into a lightweight encoder for multi-scale feature extraction; fusing the extracted multi-scale features through an enhanced hollow spatial pyramid pooling module to generate a binary classification segmentation mask for leaves and background; locating leaf regions in the preprocessed image based on the binary classification segmentation mask; inputting the located leaf region image into a lesion fine segmentation network; aggregating lesion features of different scales through a multi-scale attention module in the lesion fine segmentation network; performing six-class semantic segmentation including healthy leaves, brown spot disease, black spot disease, anthracnose, rust disease, and gray spot disease to obtain lesion segmentation results; calculating a total loss value based on the lesion segmentation results and ground truth labels using a hybrid loss function of weighted Dice loss, boundary loss, and consistency loss; optimizing network parameters through backpropagation using the total loss value; outputting the lesion segmentation results; and calculating the area ratio of various lesions based on the pixel statistics of the segmented regions for disease diagnosis analysis and control decision output.

[0016] It has the following beneficial effects: By employing a lightweight dual-path network architecture, this solution effectively addresses the issues of high model complexity and insufficient multi-scale recognition capabilities in existing disease segmentation methods. The lightweight encoder utilizes depthwise separable convolution and channel recombination techniques, significantly reducing the number of parameters and computational complexity while maintaining feature extraction capabilities, enabling the model to be deployed on edge devices such as drones and field robots. The enhanced hollow spatial pyramid pooling module constructs rich multi-scale receptive fields through multi-branch parallel hollow convolutions with varying dilation rates, simultaneously capturing feature information from early-stage micro-lesions and later-stage fused lesions. The multi-scale attention module further enhances the feature representation of key lesion regions, suppresses interference from complex backgrounds, and improves the segmentation accuracy of lesion boundaries. The overall solution achieves a balance of high precision, low latency, and low power consumption, meeting the practical needs of real-time monitoring in smart agriculture and providing strong technical support for early warning and precise control of apple leaf diseases. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an apple leaf disease segmentation method based on a lightweight dual-path network, provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of an apple leaf disease segmentation device based on a lightweight dual-path network provided in an embodiment of this application; Figure 3 This is a schematic diagram of the apple leaf disease segmentation device provided in an embodiment of this application. Detailed Implementation

[0019] The purpose of this application is to provide a method for segmenting apple leaf diseases based on a lightweight dual-path network, a device for segmenting apple leaf diseases based on a lightweight dual-path network, a computer-readable storage medium for segmenting apple leaf diseases, and a computer-readable storage medium. This method aims to reduce the number of model parameters and computational load while improving the accuracy of multi-scale lesion identification and boundary segmentation, enabling real-time inference on edge computing devices, and meeting the requirements of high precision, low latency, and low power consumption in smart agriculture applications.

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] The following embodiment illustrates the apple leaf disease segmentation method based on a lightweight dual-path network provided in this application.

[0022] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an apple leaf disease segmentation method based on a lightweight dual-path network, provided as an embodiment of this application.

[0023] In this embodiment, the method may include: S101, acquire the leaf image, perform contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain the preprocessed image; This step aims to acquire a leaf image, perform contrast-limited adaptive histogram equalization enhancement on the leaf image, and obtain a preprocessed image.

[0024] Furthermore, this step may include: Images of leaves were captured in the apple orchard using a digital camera or smartphone. The camera was kept 20-30 cm away from the leaves, ensuring the leaves occupied the center of the image. The original images were captured at a resolution of 1920×1080 pixels, including both healthy leaves and leaves infected with brown spot, black spot, anthracnose, rust, or gray spot diseases.

[0025] Contrast-limited adaptive histogram equalization enhancement processing was performed on the acquired leaf images. Specifically, the images were converted from the RGB color space to the LAB color space, the CLAHE algorithm was applied to the L channel, the clipLimit parameter was set to 2.0, and the tileGridSize was set to 8×8. After processing, the images were converted back to the RGB color space to obtain a contrast-enhanced preprocessed image. This preprocessing step effectively improved the image quality under different lighting conditions and enhanced the contrast between lesions and healthy tissue.

[0026] Furthermore, the preprocessed image is adjusted to a uniform size of 512×512 pixels, and the pixel values ​​are normalized to the [0, 1] range to meet the input requirements of the neural network.

[0027] S102, the preprocessed image is input into the lightweight encoder for multi-scale feature extraction, and the extracted multi-scale features are fused by the enhanced hollow space pyramid pooling module to generate a binary classification segmentation mask for leaves and background. Building upon S101, this step aims to input the preprocessed image into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are then fused using an enhanced hollow spatial pyramid pooling module to generate a binary segmentation mask for leaves and background.

[0028] Furthermore, this step may include: The preprocessed image is input into the lightweight encoder EfficientNet-B3 for multi-scale feature extraction. The EfficientNet-B3 encoder consists of 7 stages, extracting features layer by layer from low-level texture features to high-level semantic features. The encoder is pre-trained on the ImageNet dataset to accelerate convergence and improve feature extraction capabilities.

[0029] The feature map output by the encoder undergoes multi-scale feature fusion via the Enhanced Hollow Spatial Pyramid Pooling (LE-ASPP) module. The LE-ASPP module comprises four parallel branches: Branch 1: 1×1 convolution, outputting 256 channels, used to capture point features; Branch 2: 3×3 depth-separable dilated convolution with an inflation rate of 6 and 256 output channels; Branch 3: 3×3 depth-separable dilated convolution with an inflation rate of 12 and 256 output channels; Branch 4: 3×3 depth-separable dilated convolution with an inflation rate of 18 and 256 output channels.

[0030] The output feature maps of the four branches are concatenated along the channel dimension to obtain a 1024-channel fused feature map, which is then reduced to 256 channels by a 1×1 convolution.

[0031] The fused feature map is upsampled to the original image size by the decoder and then fused with the low-level feature map of the encoder via skip connections. Finally, a 1×1 convolution is used to output 2 channels of logits, which are then used to generate a binary classification segmentation mask for the leaf and the background using a Softmax activation function. In this mask, regions with a pixel value of 1 represent leaves, and regions with a pixel value of 0 represent the background.

[0032] S103: Based on a binary classification segmentation mask, the leaf region in the preprocessed image is located, and the located leaf region image is input into the lesion fine segmentation network. The multi-scale attention module in the lesion fine segmentation network aggregates lesion features of different scales and performs six-class semantic segmentation including healthy leaves, brown spot, black spot, anthracnose, rust, and gray spot to obtain the lesion segmentation result. Building upon S102, this step aims to locate leaf regions in the preprocessed image based on a binary classification segmentation mask. The located leaf region image is then input into the lesion fine segmentation network. The multi-scale attention module in the lesion fine segmentation network aggregates lesion features at different scales and performs six-class semantic segmentation, including healthy leaves, brown spot, black spot, anthracnose, rust, and gray spot, to obtain the lesion segmentation results.

[0033] Furthermore, this step may include: Based on the binary segmentation mask generated in step S2, the leaf region in the preprocessed image is located. Specifically, the binary segmentation mask is multiplied pixel by pixel with the preprocessed image, the pixel values ​​of the background region are set to 0, and only the image information of the leaf region is retained to obtain the leaf region image.

[0034] Leaf region images are input into a fine-grained lesion segmentation network. This network uses the same encoder-decoder structure as the first stage, but embeds a multi-scale attention module (MSAM) in the decoder. The MSAM module contains three parallel convolutional branches: Branch 1: 1×1 convolution, capturing local detailed features; Branch 2: 3×3 convolution, capturing medium-scale features; Branch 3: 5×5 convolution, capturing large-scale contextual features.

[0035] The output feature maps from the three branches are concatenated and then adaptively weighted using a channel attention mechanism. This mechanism first performs global average pooling and global max pooling on the concatenated feature maps, obtaining two one-dimensional channel descriptors. These two descriptors are then input into a shared multilayer perceptron (MLP). The MLP contains one hidden layer with 1 / 16 the number of channels and ReLU activation. The outputs of the two MLPs are summed and then multiplied by a sigmoid activation function to generate channel weights. These weights are then multiplied channel-by-channel with the original feature map to obtain the attention-enhanced feature map.

[0036] The decoder ultimately outputs 6 channels of logits, which are then processed by the Softmax activation function to generate a six-class semantic segmentation result. The six classes are: healthy leaves, brown spot, black spot, anthracnose, rust, and gray spot. Each pixel is assigned to the class with the highest probability, forming the lesion segmentation result.

[0037] S104. Based on the lesion segmentation results and ground truth labels, a hybrid loss function of weighted Dice loss, boundary loss and consistency loss is used to calculate the total loss value, and the network parameters are optimized by backpropagation using the total loss value. Building upon S103, this step aims to calculate the total loss value based on the lesion segmentation results and ground truth labels, using a hybrid loss function of weighted Dice loss, boundary loss, and consistency loss, and then optimize the network parameters through backpropagation based on the total loss value.

[0038] Furthermore, this step may include: Based on the lesion segmentation results and manually labeled ground truth values, a hybrid loss function is used to calculate the total loss value. The hybrid loss function consists of three components: Weighted Dice Loss L_Dice: This loss assigns weights to different categories to address class imbalance. Healthy leaves have a weight of 0.5, while brown spot, black spot, anthracnose, rust, and gray spot have weights of 1.2, 1.5, 1.3, 1.4, and 1.3, respectively. The Dice loss is calculated as: L_Dice = 1 - (2 × Σ(w_i × p_i × g_i) + ε) / (Σ(w_i × p_i) + Σ(w_i × g_i) + ε), where w_i is the class weight, p_i is the predicted probability, g_i is the ground truth label, and ε is a smoothing term of 1 × 10⁻⁻⁶. 5 .

[0039] Boundary loss L_boundary: Enhances edge segmentation accuracy. First, the boundary region of the ground truth label is extracted through morphological operations, with a width of 3 pixels. The cross-entropy loss between the predicted result and the ground truth boundary region is calculated, with the formula: L_boundary=-Σ(b_i×log(p_i)), where b_i is the boundary mask, which is 1 in the boundary region and 0 in other regions.

[0040] Consistency loss (L_consistency): Constrains the consistency of the two-stage segmentation results. The leaf mask predicted in the first stage should be consistent with the non-background region (the union of healthy leaves and various lesions) predicted in the second stage. Calculate the mean squared error between the two: L_consistency = MSE(M_1, M_2), where M_1 is the leaf mask in the first stage and M_2 is the non-background region mask in the second stage.

[0041] The formula for the total loss function is: L_total = 0.5 × L_Dice + 0.3 × L_boundary + 0.2 × L_consistency.

[0042] The gradient is calculated using the backpropagation algorithm, and the network parameters are updated using the Adam optimizer. The first-order moment estimation decay rate β1 of the Adam optimizer is set to 0.9, the second-order moment estimation decay rate β2 is set to 0.999, and the numerical stability term ε is set to 1×10⁻. 8 .

[0043] S105 outputs the lesion segmentation results, calculates the area ratio of various lesions based on the pixel statistics of the segmented region, and is used for disease diagnosis analysis and prevention and control decision output.

[0044] Based on S104, this step aims to output the lesion segmentation results, and calculate the area ratio of various lesions according to the pixel statistics of the segmented area, which is used for disease diagnosis analysis and prevention and control decision output.

[0045] Furthermore, this step may include: After training, the test image is input into the model, and the lesion segmentation results are output. The segmentation results are presented in the form of a color mask, with different colors representing different disease categories: healthy leaves are green, brown spot is brown, black spot is black, anthracnose is dark brown, rust is orange, and gray spot is gray.

[0046] The area ratio of each type of lesion is calculated based on the pixel count of the segmented regions. Specifically, the number of pixels in each category is counted and divided by the total number of pixels on the leaf to obtain the area ratio of each type of lesion. For example, if the total number of pixels on the leaf is 100,000 and the number of pixels for brown spot disease is 15,000, then the area ratio for brown spot disease is 15%.

[0047] Disease diagnosis is based on area proportion: when the area proportion of a certain type of lesion exceeds 5%, it is judged as mild infection; when it exceeds 15%, it is judged as moderate infection; and when it exceeds 30%, it is judged as severe infection. The system outputs a diagnostic report, including disease type, severity, and recommended control measures, to guide targeted pesticide spraying and disease early warning.

[0048] In summary, this embodiment effectively addresses the issues of high model complexity and insufficient multi-scale recognition capability in existing disease segmentation methods through a lightweight dual-path network architecture. The lightweight encoder employs depthwise separable convolution and channel recombination techniques, significantly reducing the number of parameters and computational complexity while maintaining feature extraction capabilities, enabling the model to be deployed on edge devices such as drones and field robots. The enhanced hollow spatial pyramid pooling module constructs a rich multi-scale receptive field through multi-branch parallel hollow convolution with different dilation rates, capable of simultaneously capturing feature information from early-stage micro-lesions and later-stage fused lesions. The multi-scale attention module further enhances the feature representation of key lesion regions, suppresses interference from complex backgrounds, and improves the segmentation accuracy of lesion boundaries. The overall solution achieves a balance of high precision, low latency, and low power consumption, meeting the practical needs of real-time monitoring in smart agriculture and providing strong technical support for early warning and precise control of apple leaf diseases.

[0049] The specific structure of the lightweight encoder and the enhanced void space pyramid pooling module is further illustrated by another specific embodiment.

[0050] The lightweight encoder employs the EfficientNet-B3 network architecture. EfficientNet-B3 achieves balanced optimization across the depth, width, and resolution dimensions through a composite scaling method, enabling high-precision feature extraction while maintaining a low parameter count.

[0051] The core building block of EfficientNet-B3 is the Move Inverted Bottleneck (MBConv) module. The MBConv module consists of three stages: Expansion stage: The number of input channels is expanded to 6 times the original number through 1×1 convolution, enhancing the feature representation capability; Depthwise convolution stage: Spatial feature extraction is performed using 3×3 or 5×5 depthwise separable convolution. Depthwise separable convolution decomposes standard convolution into channel-wise convolution and pointwise convolution, reducing computation by approximately 8-9 times compared to standard convolution; Projection stage: The number of channels is compressed back to the target dimension through 1×1 convolution.

[0052] A Squeeze-and-Excitation (SE) attention module is introduced between the expansion and projection phases. The SE module obtains channel-level global information through global average pooling, and generates channel weights through a two-layer fully connected network, with a channel compression ratio of 0.25.

[0053] EfficientNet-B3 consists of 7 stages, with output feature map sizes of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. In this embodiment, the output feature maps of stage 3 (1 / 8), stage 4 (1 / 16), and stage 7 (1 / 32) are selected for subsequent processing.

[0054] The Enhanced Spatial Pyramid Pooling (LE-ASPP) module employs depthwise separable dilated convolutions with dilation rates of 6, 12, and 18. Dilated convolutions expand the receptive field without increasing the number of parameters by inserting holes into the convolution kernel. The dilation rate r defines the spacing between kernel elements, and the effective receptive field is (k-1)×r+1, where k is the kernel size.

[0055] For a 3×3 convolution kernel: When the expansion rate is 6, the effective receptive field is 13×13; When the expansion rate is 12, the effective receptive field is 25×25. When the expansion rate is 18, the effective receptive field is 37×37.

[0056] Three branches with different dilation rates can capture multi-scale contextual information from local to global. Depth-separable dilated convolution decomposes dilated convolution into channel-wise dilated convolution and 1×1 pointwise convolution, reducing the number of parameters and computation by about 75% compared to standard dilated convolution.

[0057] In its implementation, the input feature map size of the LE-ASPP module is 16×16×256 (feature map after 1 / 32 downsampling). The output feature maps of the four parallel branches are concatenated along the channel dimension to obtain a fused feature map of 16×16×1024, which is then reduced to 16×16×256 through a 1×1 convolution. This design reduces the number of parameters in the LE-ASPP module to only 57% of that in the traditional ASPP module, and reduces the computational cost by approximately 63%.

[0058] Another specific embodiment further illustrates the specific implementation of the boundary refinement unit.

[0059] After generating the binary classification mask, the edge region of the mask is refined by the boundary refinement unit to improve the segmentation accuracy of the blade boundary.

[0060] The input to the boundary refinement unit consists of two parts: The high-level feature map after upsampling by the decoder has a size of 128×128×256; The low-level feature map output by encoder stage 3 has a size of 128×128×48.

[0061] Low-level feature maps contain rich edge and texture information, but weak semantic information. High-level feature maps are rich in semantic information, but lose spatial details. Boundary refinement units combine the advantages of both.

[0062] The specific processing procedure is as follows: Step 1: Apply 1×1 convolution to the high-level feature map to reduce the dimensionality, decreasing the number of channels from 256 to 48, resulting in a dimensionality-reduced high-level feature map with dimensions of 128×128×48. Dimensionality reduction reduces subsequent computation, and simultaneously, through learning, selects the feature channels most important for boundary segmentation.

[0063] Step 2: The high-level feature map and the low-level feature map after dimensionality reduction are concatenated along the channel dimension to obtain a fused feature map with a size of 128×128×96.

[0064] Step 3: Apply a 3×3 convolution to the fused feature map for feature integration. The 3×3 convolution contains 256 kernels, outputs 256 channels, and uses ReLU activation. The 3×3 receptive field can effectively capture contextual information from the edge neighborhood.

[0065] Step 4: Apply Batch Normalization to normalize the model, accelerate convergence, and improve model stability.

[0066] Step 5: Upsample the feature map from 128×128 to 512×512 using 4x bilinear interpolation, making it consistent with the original image size.

[0067] Step 6: Finally, apply a 1×1 convolution to output 2 channels of logits, and generate a refined leaf boundary segmentation mask using Softmax.

[0068] The boundary refinement unit focuses specifically on precise segmentation within a 3-5 pixel range of the blade edge. During training, higher weights are assigned to the loss for edge regions, prompting the network to prioritize learning boundary features.

[0069] Experiments show that after introducing boundary refinement units, the IoU of the leaf boundary increased from 82.3% to 89.7%, and the classification accuracy of edge pixels improved by 6.8 percentage points. Fine edge segmentation provides more accurate leaf regions for subsequent lesion localization and reduces interference from background noise.

[0070] The specific structure and implementation of the Multi-Scale Attention Module (MSAM) are explained in detail through another specific embodiment.

[0071] A multi-scale attention module is embedded in the decoder of the second-stage lesion fine segmentation network to aggregate lesion features at different scales. In diseased leaves, the initial lesion diameter may be only 2-3 mm (corresponding to about 10-15 pixels in the image), while the later merged lesions can reach 30-50 mm (corresponding to 200-300 pixels), a scale difference of more than 20 times. Convolution with a single receptive field is difficult to effectively capture these multi-scale features simultaneously.

[0072] The MSAM module takes as input the feature maps from the intermediate layers of the decoder, with a size of 64×64×256. The module contains three parallel convolutional branches: Branch 1: 1×1 Convolution Branch: This branch uses a 1×1 convolution kernel, disregarding spatial neighborhood and focusing only on features at a single pixel location. This branch effectively captures local details of lesions, such as texture variations and color abrupt changes. The 1×1 convolution contains 64 kernels, outputting 64 channels, and uses the ReLU activation function. This branch is highly effective for detecting small, early-stage lesions (10-20 pixels).

[0073] Branch 2: 3×3 Convolution Branch: Employs a 3×3 convolution kernel, with a receptive field covering a 9-pixel neighborhood. This branch captures medium-scale features, enabling the identification of typical lesion morphology features (such as the circular or oval spots of brown spot disease, 20-50 pixels in diameter). The 3×3 convolution contains 64 kernels, outputting 64 channels, with padding set to 1 to maintain the feature map size, and the activation function is ReLU.

[0074] Branch 3: 5×5 Convolution Branch: Employs a 5×5 convolution kernel, with a receptive field covering a 25-pixel neighborhood. This branch captures large-scale contextual features, enabling the identification of late-stage fused lesions (100-300 pixels in diameter) and the boundary between lesions and surrounding healthy tissue. The 5×5 convolution contains 64 kernels, outputting 64 channels, with padding set to 2 to maintain the feature map size, and ReLU activation function.

[0075] The output feature maps of the three branches are concatenated along the channel dimension to obtain a fused feature map with a size of 64×64×192.

[0076] Furthermore, the stitched fused feature map contains multi-scale information, but different channels have varying importance. The channel attention mechanism adaptively assigns weights to each channel, highlighting the feature channels most critical for lesion segmentation.

[0077] Step 1, Global Average Pooling (GAP): Perform average pooling on each channel of the fused feature map in the spatial dimension (64×64) to obtain a 192-dimensional channel descriptor F_avg. This descriptor encodes the global statistics for each channel.

[0078] Step 2, Global Max Pooling (GMP): Max pooling is performed on each channel of the fused feature map in the spatial dimension to obtain a 192-dimensional channel descriptor F_max. This descriptor captures the most salient response of each channel.

[0079] Step 3: Shared Multilayer Perceptron (MLP): F_avg and F_max are input to the same MLP. The MLP consists of two fully connected layers: Layer 1: 192-dimensional input, 12-dimensional output (192 / 16=12, compression ratio is 16), activation function is ReLU; Layer 2: 12-dimensional input, 192-dimensional output, no activation function.

[0080] Step 4, Weight Generation: The outputs of the two MLPs are added together and a 192-dimensional channel weight vector W_c is generated by the Sigmoid activation function. The value of each element is in the range of [0, 1].

[0081] Step 5, Feature Recalibration: Multiply the channel weight vector W_c with the fused feature map channel by channel to obtain an attention-enhanced feature map, still with a size of 64×64×192. Channels with larger weights are enhanced, while channels with smaller weights are suppressed.

[0082] Step 6: Reduce the number of channels from 192 to 256 using 1×1 convolution, and output the final feature map of the MSAM module with a size of 64×64×256, which will be used for subsequent decoder processing.

[0083] Experiments show that the MSAM module improves the recall rate for detecting small lesions (area < 100 pixels) from 68.5% to 84.2%, and the IoU for large lesions (area > 1000 pixels) from 91.3% to 94.6%, effectively solving the problem of multi-scale lesion recognition. The module's parameter count increases by only 0.15M, and the computational cost increases by 0.42 GFLOPs, making the overhead manageable.

[0084] The following describes an apple leaf disease segmentation device based on a lightweight dual-path network provided in the embodiments of this application. The apple leaf disease segmentation device based on a lightweight dual-path network and the apple leaf disease segmentation method based on a lightweight dual-path network described below can be referred to in correspondence with each other.

[0085] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of an apple leaf disease segmentation device based on a lightweight dual-path network, provided in an embodiment of this application.

[0086] In this embodiment, the device may include: Image preprocessing module 100 is used to acquire leaf images and perform contrast-limited adaptive histogram equalization enhancement processing on the leaf images to obtain preprocessed images; Background segmentation module 200 is used to input the preprocessed image into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are fused by an enhanced hollow space pyramid pooling module to generate a binary segmentation mask for leaves and background. The fine segmentation module 300 is used to locate the leaf region in the preprocessed image based on the binary classification segmentation mask, and input the located leaf region image into the lesion fine segmentation network; the multi-scale attention module in the lesion fine segmentation network aggregates lesion features of different scales, and performs six-class semantic segmentation including healthy leaves, brown spot disease, black spot disease, anthracnose, rust disease and gray spot disease to obtain the lesion segmentation result; The network parameter optimization module 400 is used to calculate the total loss value based on the lesion segmentation results and ground truth labels, using a hybrid loss function of weighted Dice loss, boundary loss and consistency loss, and then optimize the network parameters through backpropagation based on the total loss value. The segmentation result output module 500 is used to output the lesion segmentation results. It calculates the area ratio of various lesions based on the pixel statistics of the segmented region, and is used for disease diagnosis analysis and prevention and control decision output.

[0087] This application also provides a device for separating diseases from apple leaves; please refer to it. Figure 3 , Figure 3 This is a schematic diagram of the apple leaf disease segmentation device provided in an embodiment of this application. The apple leaf disease segmentation device may include: Memory, used to store computer programs; The processor, when executing a computer program, can implement the steps of any of the apple leaf disease segmentation methods based on a lightweight dual-path network as described above.

[0088] like Figure 3 The diagram shows the structural composition of an apple leaf disease segmentation device. The device may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other via the communication bus 13.

[0089] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0090] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the abnormal IP identification method.

[0091] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions: Obtain the leaf image, and perform contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain the preprocessed image; The preprocessed image is input into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are fused by an enhanced hollow space pyramid pooling module to generate a binary segmentation mask for leaves and background. The leaf regions in the preprocessed image are located based on a binary classification segmentation mask, and the located leaf region image is input into the lesion fine segmentation network. By aggregating lesion features at different scales through the multi-scale attention module in the lesion fine segmentation network, a six-class semantic segmentation including healthy leaves, brown spot, black spot, anthracnose, rust, and gray spot is performed to obtain lesion segmentation results; Based on the lesion segmentation results and ground truth labels, a hybrid loss function of weighted Dice loss, boundary loss and consistency loss is used to calculate the total loss value, and the network parameters are optimized by backpropagation using the total loss value. Output lesion segmentation results, and calculate the area ratio of various lesions based on the pixel statistics of the segmented regions for use in disease diagnosis analysis and prevention and control decision output.

[0092] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0093] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0094] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0095] Of course, it should be noted that, Figure 3 The structure shown does not constitute a limitation on the apple leaf disease segmentation device in the embodiments of this application. In practical applications, the apple leaf disease segmentation device may include devices that are more advanced than those described above. Figure 3 More or fewer components as shown, or combinations of certain components.

[0096] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of any of the above-described apple leaf disease segmentation methods based on a lightweight dual-path network.

[0097] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0099] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0100] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0102] The foregoing has provided a detailed description of the apple leaf disease segmentation method, the apple leaf disease segmentation device, and the computer-readable storage medium based on a lightweight dual-path network provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for segmenting apple leaf diseases based on a lightweight dual-path network, characterized in that, include: Acquire a leaf image, and perform contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain a preprocessed image; The preprocessed image is input into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are then fused using an enhanced hollow space pyramid pooling module to generate a binary segmentation mask for leaves and background. Based on the binary classification segmentation mask, the leaf region in the preprocessed image is located, and the located leaf region image is input into the lesion fine segmentation network. The multi-scale attention module in the lesion fine segmentation network aggregates lesion features of different scales and performs six-class semantic segmentation, including healthy leaves, brown spot, black spot, anthracnose, rust, and gray spot, to obtain lesion segmentation results. Based on the lesion segmentation results and ground truth labels, a hybrid loss function of weighted Dice loss, boundary loss, and consistency loss is used to calculate the total loss value, and the network parameters are optimized through backpropagation using the total loss value. The lesion segmentation results are output, and the area ratio of various lesions is calculated based on the pixel statistics of the segmented regions for use in disease diagnosis analysis and prevention and control decision output.

2. The method for segmenting apple leaf diseases according to claim 1, characterized in that, The lightweight encoder is an EfficientNet-B3 network, which extracts features through depthwise separable convolutions and a moving inverted bottleneck structure; the enhanced dilated spatial pyramid pooling module uses depthwise separable dilated convolutions with dilation rates of 6, 12, and 18 to extract multi-scale contextual information in parallel.

3. The method for segmenting apple leaf diseases according to claim 2, characterized in that, After generating the binary classification mask for leaves and background, the following steps are also included: The edge region of the binary classification segmentation mask is refined by a boundary refinement unit. The boundary refinement unit is reduced in dimensionality by 1×1 convolution and then concatenated with the low-level feature map. Finally, it is output as a refined leaf boundary by 3×3 convolution.

4. The method for segmenting apple leaf diseases according to claim 3, characterized in that, The multi-scale attention module in the lesion fine segmentation network uses three convolutional branches with kernel sizes of 1×1, 3×3, and 5×5 to extract features from different receptive fields in parallel. The features of each branch are weighted and fused through a channel attention mechanism. The channel attention mechanism extracts channel statistics through global average pooling and global max pooling, and then generates channel weights through a multilayer perceptron.

5. The method for segmenting apple leaf diseases according to claim 4, characterized in that, The hybrid loss function is expressed as: L_total=α·L_Dice+β·L_boundary+γ·L_consistency; Wherein, L_Dice is the weighted Dice loss, which sets weight coefficients for different categories to alleviate class imbalance, L_boundary is the boundary loss used to enhance edge segmentation accuracy, L_consistency is the consistency loss used to constrain the coherence of the two-stage segmentation results, and α, β, and γ are preset coefficients.

6. The method for segmenting apple leaf diseases according to claim 5, characterized in that, Also includes: During model training, the Adam optimizer is used, with a preset learning rate and preset batch size. When the validation set loss does not decrease for 5 consecutive rounds, the learning rate is reduced to 0.1 times the original learning rate until the validation loss converges to below 0.45 or the maximum number of training rounds of 63 is reached.

7. A disease segmentation device for apple leaves based on a lightweight dual-path network, characterized in that, include: The image preprocessing module is used to acquire a leaf image and perform contrast-limited adaptive histogram equalization enhancement processing on the leaf image to obtain a preprocessed image. The background segmentation module is used to input the preprocessed image into a lightweight encoder for multi-scale feature extraction. The extracted multi-scale features are fused by an enhanced hollow space pyramid pooling module to generate a binary segmentation mask for leaves and background. The fine segmentation module is used to locate the leaf region in the preprocessed image based on the binary classification segmentation mask, and input the located leaf region image into the lesion fine segmentation network; the multi-scale attention module in the lesion fine segmentation network aggregates lesion features of different scales, and performs six-class semantic segmentation including healthy leaves, brown spot disease, black spot disease, anthracnose, rust disease and gray spot disease to obtain the lesion segmentation result; The network parameter optimization module is used to calculate the total loss value based on the lesion segmentation results and ground truth labels, using a hybrid loss function of weighted Dice loss, boundary loss, and consistency loss, and then optimize the network parameters through backpropagation using the total loss value. The segmentation result output module is used to output the lesion segmentation results, and to calculate the area ratio of various lesions based on the pixel statistics of the segmented region, for disease diagnosis analysis and prevention and control decision output.

8. A device for segmenting diseases in apple leaves, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the apple leaf disease segmentation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the apple leaf disease segmentation method as described in any one of claims 1 to 6.

Images