A grape leaf spot disease recognition method and device based on an AB-Net
By using an AB-Net-based method for identifying grape leaf spot disease, multi-scale feature extraction and fusion are performed through dilated convolution and improved depthwise separable convolution, which solves the problem of low recognition rate in existing technologies and achieves high-precision identification of grape leaf spot disease.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAIYIN INSTITUTE OF TECHNOLOGY
- Filing Date
- 2023-04-14
- Publication Date
- 2026-06-05
AI Technical Summary
Existing methods for identifying grape leaf spot disease suffer from problems such as high subjectivity, low recognition rate, need for a large amount of historical sample data, and difficulty in feature extraction, making it difficult to effectively distinguish the shape and color changes of different parts of grape leaves.
A grape leaf spot disease identification method based on AB-Net is adopted, including a feature extraction module and a feature fusion module CNeck that is repeatedly stacked three times. By utilizing dilated convolution and improved depthwise separable convolution, a grape leaf spot disease identification model is constructed through multi-scale feature extraction and feature fusion.
It improved the accuracy and universality of grape leaf spot disease identification, achieving high-precision disease identification with an identification rate of 99.96%.
Smart Images
Figure CN116452979B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer technology and image recognition technology, specifically relating to a method and device for identifying grape leaf spot disease based on AB-Net. Background Technology
[0002] Grapes are high in water content and rich in sugars (glucose, fructose, pentose, etc.), as well as various minerals, organic acids, and vitamins, offering excellent health benefits and making them popular. However, grape leaf spot diseases cause a significant drop in grape yields each year, resulting in substantial economic losses for farmers. These diseases include black rot, black spot, and leaf blight, which exhibit different physical characteristics on grape leaves, such as variations in shape, color, and morphology in different parts of the leaf, making them difficult to distinguish. Therefore, the detection and control of grape leaf diseases is a highly challenging and significant task.
[0003] The identification, prevention, and control of crop diseases and pests are important disciplines in crop protection and crop medicine. With the continuous development of science and technology and a deeper understanding of crop diseases and pests, crop disease and pest control techniques have been greatly improved. From early manual identification methods, instrument-based methods, and mathematical statistics-based methods to pattern recognition and machine learning-based methods, these methods all have significant limitations, including strong subjectivity, low recognition rates, the need for large amounts of historical sample data, and difficulties in feature extraction. In recent years, deep learning methods have achieved great success in disease identification, yield prediction, and weed identification. Compared to traditional image recognition methods, deep learning technology does not require manual feature design; the system can learn and summarize features on its own. It also boasts high recognition accuracy, and the error rate of deep learning in image recognition is already lower than the human average. Summary of the Invention
[0004] Purpose of the invention: This invention provides a grape leaf spot disease identification method and device based on AB-Net that has strong adaptability, high accuracy, and can effectively improve the identification rate of leaf spot diseases.
[0005] Technical solution: This invention proposes a method for identifying grape leaf spot diseases based on AB-Net, comprising the following steps:
[0006] (1) Collect and save image data of grape leaf spot disease;
[0007] (2) The image data of grape leaf spot disease were manually labeled according to the disease category and a dataset was created;
[0008] (3) Divide the dataset into training and testing sets according to a certain ratio and preprocess the grape leaf spot disease images;
[0009] (4) Construct a grape leaf spot disease identification model based on AB-Net, including a feature extraction module and a feature fusion module CNeck that is repeatedly stacked three times; the CNeck module includes a BottleNeck dual residual path module and an ANeck module;
[0010] (5) Obtain the best pre-trained model weights through multiple iterations, and then verify and classify the disease samples in the test set.
[0011] Furthermore, the disease categories mentioned in step (2) are four types, including grape black scab, grape black rot, grape leaf blight, and grape health.
[0012] Furthermore, the preprocessing procedure for the grape leaf spot disease image described in step (3) is as follows:
[0013] Median filtering was used to denoise grape leaf spot disease images.
[0014] Data augmentation is performed on the noise-processed sample data, including contrast transformation, cropping, image transposition, rotation, and horizontal mirroring. A portion of the region is cropped but not filled with 0 pixels; instead, it is randomly filled with pixel values from other regions in the training set.
[0015] Furthermore, the feature extraction module described in step (4) has the following structure:
[0016] A multi-scale feature extraction module is constructed using three branches, each employing a 3x3 dilated convolution. Adaptive average pooling is applied to the outputs of the three branches after dilation and convolution to ensure that the feature maps output by the three branches have the same height and width. The outputs from the three branches are then concatenated and fused to obtain a fused feature vector. This fused feature vector is then subjected to a 1x1 convolution operation to reduce the number of channels. Finally, a max pooling operation is performed to reduce the image size.
[0017] Furthermore, the ANeck module structure described in step (4) is as follows:
[0018] A multi-scale feature fusion module is constructed using four branches. The first branch uses a 1*1 convolution; the second branch uses a 1*1 convolution and downsampling; the third branch uses a 1*1 convolution and two m*m convolutions; and the fourth branch consists of a 1*1 convolution and two asymmetric convolutions with an asymmetric convolution kernel size of (1, n). The feature vectors output from the four branches are upsampled to ensure that the feature maps output from the four branches have the same height and width.
[0019] Furthermore, the BottleNeck dual residual path module structure described in step (4) is as follows:
[0020] This module consists of a residual network and an improved depthwise separable convolution. The backbone network of this module is composed of two sub-modules. The first sub-module consists of a 1*1 PW convolution, a 3*3 DW convolution, and a 1*1 PW convolution, which first increases the dimensionality and then decreases it to reduce the number of parameters. The second module decomposes a 3*3 DW convolution into two 3*3 DW convolutions, and then concatenates them with a 1*1 PW convolution. The sub-modules use residual connections to map low-level features to high-level spaces, thereby enhancing feature propagation and gradient propagation.
[0021] Furthermore, the dilation coefficients of the dilated convolutions are 6, 8, and 10, respectively.
[0022] Based on the same inventive concept, the present invention also proposes a grape leaf spot disease identification device based on AB-Net, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is loaded onto the processor, it implements the above-mentioned grape leaf spot disease identification method based on AB-Net.
[0023] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows:
[0024] 1. This invention employs a multi-scale feature extraction module based on dilated convolution, which increases the receptive field of the convolution kernel while keeping the number of parameters constant. This not only ensures that each convolution output contains a large range of information, but also guarantees that the size of the output feature map remains unchanged.
[0025] 2. This invention is based on an improved depthwise separable convolution. Considering the high correlation between rows of the parameter matrix used in the convolutional layer, a 3*3 DW convolution is decomposed into two 3*3 DW convolutions. By decomposing the convolution kernel matrix and adding orthogonal constraints, the correlation between parameters is reduced, thereby enhancing the feature fusion capability of the network model.
[0026] 3. This invention achieves feature fusion through the stacking of multiple modules, thereby improving the universality and accuracy of leaf spot disease identification. Attached Figure Description
[0027] Figure 1 The flowchart shows a method for identifying grape leaf spot disease based on AB-Net.
[0028] Figure 2 This is a schematic diagram of the grape leaf spot disease identification network model constructed in this invention;
[0029] Figure 3 This is a schematic diagram of the feature extraction module.
[0030] Figure 4 This is a schematic diagram of the BottleNeck module structure;
[0031] Figure 5 A schematic diagram of the improved depthwise separable convolutional module structure;
[0032] Figure 6 This is a schematic diagram of the ANeck module structure. Detailed Implementation
[0033] The present invention will now be described in further detail with reference to the accompanying drawings.
[0034] This invention proposes a method for identifying grape leaf spot diseases based on AB-Net, such as... Figure 1 As shown, the specific steps include:
[0035] Step 1: Collect and save grape leaf spot image data, and manually label the grape leaf spot image data according to the disease category and create a dataset.
[0036] The image data on grape leaf spot diseases are divided into four categories based on the type of leaf spot disease: grape black scab, grape black rot, grape leaf blight, and healthy grapes. These images include leaves lacking protein, leaves in shade, leaves under different light conditions, leaves wet from rain, leaves damaged by pests, leaves contaminated with animal feces, and diseased leaves.
[0037] Step 2: Divide the dataset into training and testing sets according to a certain ratio and preprocess the grape leaf spot disease images.
[0038] The sample images of grape leaves were divided into a training set and a test set in an 8:2 ratio. The test set was used to detect the accuracy of the neural network model trained on the training set.
[0039] Preprocessing of sample data from the grape leaf spot disease dataset includes:
[0040] (i) The leaf spot disease dataset is denoised using median filtering (which removes noise while preserving image edge details).
[0041] (ii) Perform data augmentation on the noise-processed sample data. The data augmentation includes contrast transformation, cropping, image transposition, rotation, horizontal mirroring, cropping a portion of the sample data without filling it with 0 pixels, and randomly filling it with the pixel values of other data in the training set. The preprocessed dataset contains a total of 9027 images, of which 7222 images are in the training set and 1805 images are in the test set.
[0042] Step 3: Construct a grape leaf spot disease identification model based on AB-Net, such as... Figure 2 As shown, the system includes one feature extraction module and three feature fusion modules (CNeck). First, the input feature vector is processed by the feature extraction module, which is also a multi-path parallel branch module. This module consists of dilated convolutions with different dilation coefficients. Then, the feature vectors output from the three branches undergo adaptive average pooling to ensure that the feature vectors have the same height and width for the fusion operation. Next, a 1x1 convolution is cascaded for dimensionality reduction, decreasing the number of channels in the feature vector. Then, a max pooling operation is cascaded to reduce the image size. Next, the CNeck module is constructed. The CNeck module consists of a dual residual path module (BottleNeck) and an Aneck module.
[0043] BottleNeck module: This module consists of residual connections and improved depthwise separable convolutions, such as... Figure 5 As shown, the backbone network consists of two sub-modules. The first sub-module is composed of a 1*1 PW convolution, a 3*3 DW convolution, and a 1*1 PW convolution, performing dimensionality increase followed by dimensionality reduction to extract features while maintaining the same number of channels and reducing the number of parameters. The second module first decomposes a 3*3 DW convolution into two 3*3 DW convolutions, and then concatenates them with a 1*1 PW convolution. Residual connections are used within each sub-module to map low-level features to a high-level space, enhancing feature propagation and gradient propagation.
[0044] The ANeck module employs four branches to construct a multi-scale feature extraction module. The first branch uses a 1x1 convolution; the second branch uses a 1x1 convolution and downsampling; the third branch uses a 1x1 convolution and two mxm convolutions; and the fourth branch consists of a 1x1 convolution and two asymmetric convolutions with a kernel size of (1, n). Upsampling is applied to the feature vectors output from each branch to ensure that the feature maps from all four branches have the same height and width. Each time the CNeck module is stacked, the m and n parameters of the ANeck module are adjusted accordingly to further extract features and improve the feature extraction capability of the network model.
[0045] The CNeck module is stacked three times to achieve image feature fusion, resulting in the final grape leaf spot disease recognition network model. The specific steps include:
[0046] First, such as Figure 3As shown, multi-scale feature extraction is performed on a preprocessed image of size 224*224*3. In the first branch, the input feature vector is first passed through a dilated convolution with a kernel size of 3*3 and a dilation factor of 6, resulting in an output feature vector of (212, 212, 64). This is then passed through a BatchNorm activation function and a LeakyReLU activation function, and finally through an adaptive average pooling process to obtain a feature vector of (112*112*64). In the second branch, the input feature vector is passed through a dilated convolution with a kernel size of 3*3 and a dilation factor of 8, resulting in an output feature vector of (112*112*64). The first feature vector (216, 216, 64) is processed by a BatchNorm and a LeakyReLU activation function, followed by adaptive average pooling to obtain a feature vector (112*112*64). In the third branch, the input feature vector is dilated by a 3x3 kernel with a dilation factor of 10, resulting in an output feature vector (220, 220, 64). This is then processed by another BatchNorm and a LeakyReLU activation function, followed by adaptive average pooling to obtain another feature vector (112*112*64). The output feature vectors from the three branches are concatenated and fused to obtain an output feature vector (112*112*192). Then, the output feature vector is dimensionality-reduced by a 1x1 convolution, resulting in an output vector (112*112*64). Finally, max pooling is performed on the feature vector to halve its size, yielding the input feature vector (56, 56, 64) for the feature fusion module.
[0047] like Figure 4 As shown, after passing through the CNeck_1 module: first, the input feature vector is passed through a BottleNeck1_1 module consisting of a 1*1 PW convolution, a 3*3 DW convolution, and a 1*1 PW convolution. Then, the output vector is residually concatenated with the initial input feature vector, and finally, upsampling is performed to obtain the input feature vector of BottleNeck1_2 (56, 56, 128). Then, it passes through a BottleNeck1_2 module consisting of a 3*3 DW convolution, a 3*3 DW convolution, and a 1*1 PW convolution. The output vector is then residually concatenated with the input feature vector of this module, and finally, upsampling is performed to obtain the input feature vector of Aneck_1 (56, 56, 256). Then, the output feature vector of the BottleNeck1 module passes through the four branches of the Aneck1 module, as follows... Figure 6As shown, the first branch first undergoes a 1*1 convolution and a max pooling operation to obtain the output feature vector (28, 28, 64); the second branch first undergoes a 1*1 convolution and then two asymmetric convolutions with kernel_size (1, 5) to obtain the output feature vector (28, 28, 20); the third branch first undergoes a 1*1 convolution and then two 3*3 convolutions to obtain the output feature vector (28, 28, 20); the fourth branch undergoes a 1*1 convolution operation to obtain the feature vector (28, 28, 24); finally, the four branches are connected and merged to obtain the input feature vector (28, 28, 128) of the CNeck_2 module.
[0048] After passing through the CNeck_2 module: First, the input feature vector is passed through a BottleNeck2_1 module consisting of a 1*1 PW convolution, a 3*3 DW convolution, and a 1*1 PW convolution. Then, the output vector is residually concatenated with the initial input feature vector. Finally, upsampling is performed to obtain the input feature vector of BottleNeck2_2 (28, 28, 160). Then, it is passed through a BottleNeck2_2 module consisting of a 3*3 DW convolution, a 3*3 DW convolution, and a 1*1 PW convolution. The output vector is residually concatenated with the input feature vector of this module. Finally, upsampling is performed to obtain the input feature vector of Aneck_2 (28, 28, 192). The output feature vector of the BottleNeck2 module is then passed through four branches of the Aneck2 module. The first branch performs a 1x1 convolution and a max pooling operation to obtain the output feature vector (14, 14, 64). The second branch performs a 1x1 convolution followed by two asymmetric convolutions with kernel size (1, 7) to obtain the output feature vector (14, 14, 64). The third branch performs a 1x1 convolution followed by two 5x5 convolutions to obtain the output feature vector (14, 14, 64). The fourth branch performs a 1x1 convolution operation to obtain the feature vector (14, 14, 64). Finally, the four branches are concatenated and merged to obtain the input feature vector (14, 14, 256) of the CNeck_3 module.
[0049] After passing through the CNeck_3 module: First, the input feature vector is passed through a BottleNeck3_1 module consisting of a 1*1 PW convolution, a 3*3 DW convolution, and a 1*1 PW convolution. Then, the output vector is residually concatenated with the initial input feature vector. Finally, upsampling is performed to obtain the input feature vector (14, 14, 288) of BottleNeck3_2. Then, it is passed through a BottleNeck3_2 module consisting of a 3*3 DW convolution, a 3*3 DW convolution, and a 1*1 PW convolution. The output vector is residually concatenated with the input feature vector of this module. Finally, upsampling is performed to obtain the input feature vector (14, 14, 384) of Aneck_3. The output feature vector from the BottleNeck3 module is then processed through four branches of the Aneck3 module. The first branch performs a 1x1 convolution followed by a max pooling operation to obtain the output feature vector (7, 7, 64). The second branch performs a 1x1 convolution followed by two asymmetric convolutions with kernel size (1, 3) to obtain the output feature vector (7, 7, 64). The third branch performs a 1x1 convolution followed by two 5x5 convolutions to obtain the output feature vector (7, 7, 64). The fourth branch performs a 1x1 convolution operation to obtain the feature vector (7, 7, 64). Finally, the four branches are concatenated and merged to obtain the output feature vector (7, 7, 256).
[0050] Finally, a 7*7 average pooling layer is used instead of a fully connected layer to obtain an output feature vector of (1, 1, 256). Then, a Softmax classifier is constructed to convert the individual probabilities of each prediction result into the overall probability in the prediction.
[0051] Step 4: Obtain the best pre-trained model weights through multiple iterations, and then perform validation classification on the disease samples in the test set.
[0052] The target loss function is the cross-entropy loss function, which allows the model's predictions to continuously approximate the true labels. Adam is used as the optimizer to reduce the difference loss calculated by the loss function, making the convergence of the loss function more stable. The formula for the cross-entropy loss function is:
[0053]
[0054] The larger the prediction result p(x) of x, the smaller the cross-entropy f(x).
[0055] Based on the same inventive concept, this invention proposes a grape leaf spot disease identification device based on AB-Net, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the above-mentioned grape leaf spot disease identification method based on AB-Net when loaded onto the processor.
[0056] The performance evaluation metrics of this invention include accuracy, precision, recall, and the F-Measure. The formulas for each evaluation metric are as follows:
[0057]
[0058] This indicator is mainly used to indicate the number of samples that are correctly predicted out of the total number of samples.
[0059]
[0060] Precision rate reflects the proportion of samples that were predicted to be positive but were actually positive.
[0061]
[0062] This indicator reflects the proportion of samples predicted as positive out of the total number of positive samples; where TP represents the number of positive samples predicted as positive; TN represents the number of negative samples predicted as negative; FP represents the number of negative samples predicted as positive; and FN represents the number of positive samples predicted as negative.
[0063] To address the occasional issue with precision and recall (i.e., high precision often leads to low recall, and vice versa, limiting its application to simple classification tasks), the F1-Measure metric was introduced. This metric is the weighted harmonic mean of precision and recall. Its calculation formula is as follows:
[0064]
[0065] Where β is a variable, P is precision, and R is recall. When β = 1, it is the most common F1-Measure.
[0066]
[0067] Finally, a grape leaf spot disease identification model was used to classify grape leaf spot diseases.
[0068] Table 1 Comparison of experimental results of this invention with GoogleNet and MobileNet-V2
[0069] Modelname Accuracy GoogleNet 97.63％ MobileNet-V2 98.72％ AB-Net 99.96％
[0070] As shown in Table 1, the average accuracy of grape leaf spot disease identification using the present invention is approximately 99.96%. Compared to some existing models, the present invention has higher accuracy, stronger robustness, and a higher identification rate.
Claims
1. A method for identifying grape leaf spot diseases based on AB-Net, characterized in that, Includes the following steps: (1) Collect and save image data of grape leaf spot disease; (2) Manually label the grape leaf spot image data according to the disease category and create a dataset; (3) Divide the dataset into training and test sets according to a certain ratio and preprocess the grape leaf spot disease images; (4) Construct a grape leaf spot disease identification model based on AB-Net, including a feature extraction module and a feature fusion module CNeck that is repeatedly stacked three times; the CNeck module includes a BottleNeck dual residual path module and an ANeck module; (5) Obtain the best pre-trained model weights through multiple iterations, and then verify and classify the disease samples in the test set; The ANeck module structure described in step (4) is as follows: A multi-scale feature fusion module is constructed using four branches. The first branch uses a 1*1 convolution; the second branch uses a 1*1 convolution and downsampling; the third branch uses a 1*1 convolution and two m*m convolutions; and the fourth branch consists of a 1*1 convolution and two asymmetric convolutions with a kernel size of (1, n). The feature vectors output from the four branches are upsampled to ensure that the feature maps output from the four branches have the same height and width. The BottleNeck dual residual path module structure described in step (4) is as follows: This module consists of a residual network and an improved depthwise separable convolution. The backbone network of this module is composed of two sub-modules. The first sub-module consists of a 1*1 PW convolution, a 3*3 DW convolution, and a 1*1 PW convolution, which first increases the dimensionality and then decreases it to reduce the number of parameters. The second module first decomposes a 3*3 DW convolution into two 3*3 DW convolutions, and then concatenates them with a 1*1 PW convolution. The sub-modules use residual connections to map low-level features to high-level spaces, thereby enhancing feature propagation and gradient propagation.
2. The method for identifying grape leaf spot disease based on AB-Net according to claim 1, characterized in that, The disease categories mentioned in step (2) are four types, including grape black scab, grape black rot, grape leaf blight, and grape health.
3. The method for identifying grape leaf spot disease based on AB-Net according to claim 1, characterized in that, The preprocessing procedure for grape leaf spot disease images described in step (3) is as follows: Median filtering was used to denoise grape leaf spot disease images. Data augmentation is performed on the noise-processed sample data, including contrast transformation, cropping, image transposition, rotation, and horizontal mirroring. A portion of the region is cropped but not filled with 0 pixels; instead, it is randomly filled with pixel values from other regions in the training set.
4. The method for identifying grape leaf spot disease based on AB-Net according to claim 1, characterized in that, The feature extraction module in step (4) has the following structure: A multi-scale feature extraction module is constructed using three branches, each employing a 3x3 dilated convolution. Adaptive average pooling is applied to the outputs of the three branches after dilation and convolution to ensure that the feature maps output by the three branches have the same height and width. The outputs from the three branches are then concatenated and fused to obtain a fused feature vector. This fused feature vector is then subjected to a 1x1 convolution operation to reduce the number of channels. Finally, a max pooling operation is performed to reduce the image size.
5. The method for identifying grape leaf spot disease based on AB-Net according to claim 4, characterized in that, The dilation coefficients of the dilated convolutions are 6, 8, and 10, respectively.
6. A grape leaf spot disease identification device based on AB-Net, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the AB-Net-based method for identifying grape leaf spot disease according to any one of claims 1-5.