Method for identifying potato scab

Abstract

The application belongs to the field of potato scab disease recognition, and specifically relates to a potato scab disease recognition system based on semantic segmentation. The application aims to accurately segment potato scab spots and accurately estimate scab coverage, and proposes a potato scab disease segmentation model based on multi-scale feature fusion, named MSFF-UNet. The model uses VGG16 as the backbone network on the basis of the U-Net model to enhance the feature extraction and semantic understanding ability of the model. Secondly, a deep feature enhancement module is added to the process of upward feature fusion in the decoder, and a series of convolution and normalization operations are stacked to improve the discrimination of potato scab spots and their distribution at different sizes. Finally, a multi-scale feature fusion module is added to the decoder to strengthen feature extraction of high-level features and feature fusion of up-sampled low-level features, and then gradually fuse layer by layer to capture semantic information of potato scab spots at different scales and realize accurate segmentation of potatoes and scab spots.

Description

TECHNICAL FIELD

[0001] The present application relates to a disease identification method for potato cultivation, in particular to a method for identifying potato scab. BACKGROUND

[0002] Potato is the fourth largest food crop in the world and is widely planted. As the world's largest potato producer, potato industry is crucial to ensure food security and transformation of agricultural development mode. Potato scab is a soil-borne disease caused by multiple Streptomyces. In recent years, it has been widely prevalent in potato planting areas. With the expansion of potato planting scale, the pathogen species of potato scab gradually increases, and the incidence of potato scab is becoming more and more serious, which has become a major problem restricting the development of potato industry and economic value. At present, the calculation of potato scab lesion coverage and the assessment of disease severity mainly rely on manual observation and estimation, which has low accuracy and affects the progress of potato scab prevention and control research. In addition, the non-destructive detection of potato appearance in enterprise production also relies on manual observation, which has low detection efficiency and high cost, affecting the economic benefits. Therefore, it is of great significance to accurately segment potato scab lesion and accurately estimate its coverage for accurate evaluation of potato scab disease, screening of disease-resistant germplasm, sorting of potato scab, and promoting the development of potato industry.

[0003] Some domestic and foreign scholars have carried out research on the detection of potato tuber diseases. Zhang Jing applied genetic frog jumping algorithm to the parameter adjustment of PCNN neural network and proposed an image segmentation model based on genetic frog jumping algorithm to realize the segmentation of potato sclerotinia, gray mold, and root rot nematode disease. However, the segmentation result of this method is a binary image, which will lead to difficulty in threshold selection when the light condition and potato lesion color change greatly, thereby affecting the accuracy of segmentation. Oppenheim D collected potato disease data such as scab, black skin disease, and psoriasis, and used a pre-trained convolutional neural network for classification to realize the detection of potato tuber diseases. It can distinguish different diseases, but cannot evaluate the disease severity. Qi Yannan proposed a potato lesion positioning method and a potato scab regression identification method. The parameters of scab lesions were obtained through normalization, and a BP neural network identification model for potato scab was established. Although this method can detect potato scab defects, it is limited to the detection of lesion targets and cannot realize the quantification of potato scab lesion coverage and the evaluation of potato scab disease severity.

[0004] In view of the above problems, the present technical scheme is proposed. SUMMARY

[0005] The application uses VGG16 as a backbone network on the basis of the U-Net model to realize recognition of potato scab lesions. And on the basis, a deep feature enhancement module and a multi-scale feature fusion module are further proposed to perform another layer-by-layer upward fusion on multi-level features through stacking a series of convolution and batch normalization operations in the process of upward fusion of decoder features.

[0006] The technical solution of the application is as follows:

[0007] S1: potato scab data collection: collect images of each scab;

[0008] S2: data processing and dataset construction: pre-process the collected potato scab image set, label the potato scab image by using a Labelme labeling tool, label two target objects of potato and scab lesions, convert the labeled image from a.json format to a.png format for saving, and finally divide the labeled data into a training set, a validation set and a test set according to a 7:2:1 ratio;

[0009] S3: constructing a potato scab segmentation model;

[0010] S3-1: replacing the backbone network of the U-Net encoder part with VGG16;

[0011] S4: training the potato scab recognition model through the training set;

[0012] S4-1: in order to verify the performance of potato scab recognition, MPA, MIoU, IoU, Precision and other indicators are used for evaluation in the training experiment. The calculation formula is as follows:

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] Wherein TN represents that the model correctly classifies negative class samples as negative class, TP represents that the model correctly classifies positive class samples as positive class, FN represents that the model incorrectly classifies positive class samples as negative class, FP represents that the model incorrectly classifies negative class samples as positive class, and k is the number of classes;

[0018] S4-2: training;

[0019] S5: deploy the trained potato scab disease recognition model in S4 to the mobile APP, recognize potato scab disease through mobile APP shooting, and estimate the scab coverage.

[0020] Further, at least 200 images of each type of scab disease.

[0021] Further, the preprocessing includes data enhancement by random rotation, mirror flip, brightness adjustment, contrast adjustment and saturation adjustment, and uniform resolution of input feature images by adjusting the image size.

[0022] Further, the scab disease in step S1 is set to three categories, namely convex, concave and flat scab lesions.

[0023] Further, feature extraction is performed by superimposed convolution and maximum pooling. The calculation formula of the output feature size of the maximum pooling layer is as follows:

[0024] ;

[0025] In the formula, In size represents the input feature map size, Pool size represents the pooling window size, and Stride represents the step size.

[0026] The calculation formula of the output size of each layer of convolution after maximum pooling is:

[0027] ;

[0028] In the formula, In size represents the input feature map size, F size represents the convolution kernel size, P represents the padding number, and Stride represents the step size.

[0029] A deep feature extraction module is added to the feature extraction part of the decoder to generate more rich and stable feature expression by introducing superimposed operation of different convolution kernel sizes and adding batch normalization layer. The calculation method of batch normalization can be represented as:

[0030] ;

[0031] In the formula, x i is the input original feature data, mu and sigma 2 are the mean and variance of the samples in the batch, epsilon is a positive number tending to 0, gamma is a scaling factor, and beta is a translation factor.

[0032] Further, a multi-scale feature fusion module is added to the feature upward fusion part of the decoder. Through transpose convolution operation on the low-size feature map, superposition convolution and batch normalization operation on the high-size feature map, etc., the two are finally fused. The fused feature map is fused again with the feature map upward fused in the decoder, the receptive field range is expanded, the feature level is enriched, and the semantic information of potato scab lesion under different scales is better captured, wherein the calculation formula of the transpose convolution is as follows:

[0033] ;

[0034] wherein In size represents the input feature map size, stride represents the step, P represents the padding number, kernel size represents the convolution kernel size, and the feature map after transpose convolution can be represented by the following formula:

[0035] ;

[0036] wherein TransposeConv represents the transpose convolution operation, x1 represents the input feature map, and the feature map output after multiple convolution and normalization operations can be represented as:

[0037] ;

[0038] wherein ReLU represents the activation function, BatchNorm represents the batch normalization operation, Conv represents the convolution operation, and X2 represents the input feature map.

[0039] The technical effects of the technical scheme of the application are as follows:

[0040] Based on the MSFF-UNet semantic segmentation network, VGG16 is used as the backbone network on the basis of the U-Net model, the deep and rich feature expression ability of VGG16 is used to enhance the feature extraction and semantic understanding ability of the model, and all aspects of the model are improved.

[0041] Secondly, a deep feature enhancement module is proposed and added to the feature upward fusion process of the decoder, and a series of convolution and normalization operations are superimposed to improve the discrimination of potato scab lesions and their distribution under different scales.

[0042] Finally, a multi-scale feature fusion module is added to the decoder, and high-level features are enhanced for feature extraction, low-level features are upsampled and then fused, and then layer-by-layer upward fusion is performed to capture semantic information of potato scab lesions under different scales.

[0043] The potato scab area and healthy area are separated by the semantic segmentation technology, so that the quantitative evaluation of the disease degree is realized, which has important significance for the field resistance evaluation of potato, the screening of disease-resistant germplasm, the disease prevention, the sorting of scab potato and the promotion of potato industry development.

[0044] The technical scheme is to explore the improvement of U-Net model brought by VGG16, deep feature enhancement module and multi-scale feature fusion module, and the following ablation test is carried out, the feasibility of the improved semantic segmentation model is verified by evaluating MPA, MIoU, IoU, Precision and other evaluation indexes, and the test results are shown in Table 1.

[0045] The ablation test uses U-Net model as scheme 1 for comparison, scheme 2 uses VGG16 as the backbone network, and the results show that all aspects of the model are improved, especially the class average pixel accuracy from 82.98% to 88.65%, an increase of 5.67%, which means that the classification accuracy at the pixel level has made obvious progress, fully proving the feature extraction ability of VGG16 and its superior performance in the task of potato scab lesion segmentation. Therefore, VGG16 is introduced in the following experiments to enhance the segmentation effect. After adding the deep feature enhancement module in scheme 3, compared with VGG16-UNet, the class average pixel accuracy, average intersection over union and precision of the model reached 90.31%, 87.39% and 93.42%, because the deep feature enhancement module superimposes multiple convolution and batch normalization operations, which enhances the expression ability and classification ability of the features. In scheme 4, the multi-scale feature fusion module is added, which further deepens the network level by connecting multiple feature maps, and the class average pixel accuracy, average intersection over union and precision reach 93.29%, 87.58% and 93.79. In scheme 5, MSFF-UNet combines the advantages of the above methods, and the class average pixel accuracy, average intersection over union, precision, and the intersection over union and precision of scab lesion and potato are all the highest, which are 93.51%, 87.72%, 93.90%, 75.28%, 89.64%, 89.98% and 93.74% respectively, which are obviously improved compared with the initial U-Net model, and the percentage points are increased by 10.53, 5.41, 1.49, 11.56, 4.39, 0.86 and 5.46 respectively, which shows that using VGG16 as the backbone network and using deep feature enhancement and multi-scale feature fusion modules in the decoder part can obtain more rich semantic information, so MSFF-UNet is selected as the optimal model for testing.

[0046]

[0047] Note: mPA is the class average pixel accuracy; mIoU is the average intersection over union; Precision is the precision; Scab IoU is the scab lesion intersection over union; Potato IoU is the potato intersection over union; Scab Precision is the precision of segmenting scab lesion; Potato Precision is the precision of segmenting potato. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The step operation diagram of the present application;

[0049] Figure 2 is a legend of three kinds of potato scab;

[0050] Figure 3 is a schematic diagram of image standard effect;

[0051] Figure 4 is the result of segmenting potato and scab lesion by different models. DETAILED DESCRIPTION

[0052] Example 1

[0053] The technical solution of the present application comprises the following steps:

[0054] S1: Potato scab data collection: collect images of each kind of scab;

[0055] S2: Data processing and dataset construction: preprocess the collected potato scab image set, use Labelme annotation tool to annotate potato scab images, annotate two kinds of targets of potato and scab lesion, convert the annotated images from.json format to.png format for saving, and finally divide the annotated data into training set, validation set and test set according to the ratio of 7:2:1;

[0056] S3: Constructing a potato scab segmentation model: replace the backbone network of the U-Net encoder part with VGG16;

[0057] S4: Training the potato scab recognition model through the training set;

[0058] S4-1: In order to verify the performance of potato scab recognition, MPA, MIoU, IoU, Precision and other indicators are used for evaluation in the training experiment. The calculation formula is as follows:

[0059] ;

[0060] ;

[0061] ;

[0062] ;

[0063] wherein TN represents that the model correctly classifies the negative class sample as the negative class, TP represents that the model correctly classifies the positive class sample as the positive class, FN represents that the model incorrectly classifies the positive class sample as the negative class, FP represents that the model incorrectly classifies the negative class sample as the positive class, and k is the number of classes;

[0064] S4-2: training is performed;

[0065] S5: the potato scab disease recognition model trained in the step S4 is deployed to a mobile terminal APP, the potato scab disease is recognized through the mobile terminal APP, and the scab coverage rate is estimated.

[0066] Embodiment 2

[0067] The difference between this embodiment and embodiment 1 is that the number of images of each scab disease in this embodiment is at least 200. The pre-processing includes data enhancement by random rotation, mirror flip, adjustment of brightness, contrast and saturation, and the resolution of the input feature image is unified by adjusting the image size. The scab disease in step S1 is set to three categories, which are convex, concave and flat scab lesions. A deep feature extraction module is added to the feature extraction part of the decoder to generate more rich and stable feature expressions by introducing superposition operation of different convolution kernel sizes and adding a batch normalization layer. The calculation method of the batch normalization can be expressed as:

[0068] ;

[0069] In the formula, x i is the input original feature data, μ and σ 2 are the mean and variance of the samples in the batch, ε is a positive number tending to 0, γ is a scaling factor, and β is a translation factor.

[0070] Embodiment 3

[0071] The difference between this embodiment and embodiment 2 is that this embodiment further includes a multi-scale feature fusion module added to the feature upward fusion part of the decoder. The low-size feature map is subjected to a transpose convolution operation, the high-size feature map is subjected to a convolution and batch normalization operation, and finally the two are fused. The fused feature map is fused again with the feature map upward fused in the decoder, the receptive field range is expanded, the feature level is enriched, and the semantic information of the potato scab lesion under different scales is better captured. The calculation formula of the transpose convolution is as follows:

[0072] ;

[0073] In the formula, In sizeIn represents the input feature map size, Pool size represents the convolution kernel size, and the feature map after transposed convolution can be represented by the following formula:

[0074] ;

[0075] where TransposeConv represents the transposed convolution operation, and x1 represents the input feature map. The feature map output after multiple convolution and normalization operations can be represented as:

[0076] ;

[0077] where ReLU represents the activation function, BatchNorm represents the batch normalization operation, Conv represents the convolution operation, and X2 represents the input feature map.

[0078] Embodiment 4

[0079] The difference between this embodiment and Embodiment 2 is that there are at least 300 images of each type of scab disease in this embodiment. Step S3 further includes feature extraction by stacking convolution and max pooling. The calculation formula of the output feature size of the max pooling layer is as follows:

[0080] ;

[0081] In size represents the input feature map size, Pool size represents the pooling window size, and Stride represents the stride;

[0082] The calculation formula of the output size of each layer of convolution layer after max pooling is as follows:

[0083] ;

[0084] In size represents the input feature map size, F size represents the convolution kernel size, and P represents the padding quantity, and Stride represents the stride.

Claims

1. A method for identifying potato scab disease, characterized in that, It includes the following steps: S1: Potato scab data collection: Acquire images of each type of scab; S2: Data Processing and Dataset Construction: The collected potato scab image set was preprocessed, and the potato scab images were labeled using the Labelme annotation tool, labeling two types of targets: potatoes and scab spots. The labeled images were then converted from .json format to .png format for saving. Finally, the labeled data was divided into training set, validation set and test set according to a 7:2:1 ratio. S3: Constructing a segmentation model for potato scab: The backbone network of the U-Net encoder is set to VGG16, and a deep feature extraction module is added to the feature extraction part of the decoder. By introducing the stacking operation of different convolutional kernel sizes and adding batch normalization layers, richer and more stable feature representations are generated. The batch normalization calculation method can be expressed as: ； In the formula, x i Given the original feature data as input, μ and σ 2 ε represents the mean and variance of the samples within the batch, ε is a positive number approaching 0, γ is a scaling factor, and β is a translation factor. A multi-scale feature fusion module is added to the feature up-fusion part of the decoder. This module performs transposed convolution on the low-size feature map, stacked convolution on the high-size feature map, and batch normalization. Finally, the two are fused together, and the fused feature map is then fused again with the feature map fused up-fusion in the decoder. This expands the receptive field and enriches the feature hierarchy, enabling better capture of the semantic information of potato scab lesions at different scales. The formula for calculating the transposed convolution is as follows: ； In size The input feature map size is represented by Stride, P represents the padding amount, and kernel is the kernel size. size Representing the kernel size, the feature map after transpose convolution can be represented by the following formula: ； Where TransposeConv represents the transpose convolution operation, x1 represents the input feature map, and the output feature map after multiple convolutions and normalization operations can be represented as: ； Where ReLU represents the activation function, BatchNorm represents batch normalization, Conv represents convolution, and X2 represents the input feature map. S4: Train a potato scab identification model using the training set; S4-1: To verify the performance of potato scab identification, the training experiment used MPA, MIoU, IoU, and Precision metrics for evaluation. The calculation formulas are as follows: ； ； ； ； Where TN indicates that the model correctly classifies negative class samples as negative, TP indicates that the model correctly classifies positive class samples as positive, FN indicates that the model incorrectly classifies positive class samples as negative, FP indicates that the model incorrectly classifies negative class samples as positive, and k is the number of classes; S4-2: Conduct training; S5: Deploy the potato scab identification model trained in step S4 to a mobile app, and use the mobile app to take pictures to identify potato scab and estimate the scab coverage rate.

2. The method for identifying potato scab disease according to claim 1, characterized in that: At least 200 images for each type of scabies.

3. The method for identifying potato scab disease according to claim 1, characterized in that: Preprocessing includes data augmentation through random rotation, mirror flipping, and adjustment of brightness, contrast, and saturation, and unifying the resolution of the input feature image by adjusting the image size.

4. The method for identifying potato scab disease according to claim 1, characterized in that: In step S1, scabies are classified into three categories: raised, depressed, and flat scabies lesions.

5. The method for identifying potato scab disease according to claim 1, characterized in that: Step S3 also includes: Feature extraction is performed by stacking convolutions and max pooling. The formula for calculating the output feature size of the max pooling layer is as follows: ； In the formula size Pool represents the size of the input feature map. size The `stride` parameter represents the pooling window size; The formula for calculating the output size of each convolutional layer after max pooling is: ； In the formula size F represents the size of the input feature map. size P represents the kernel size, P represents the padding amount, and Stride represents the stride.

Images