Pepper disease and pest recognition method and system based on hierarchical detection double task model

Abstract

The application belongs to the technical field of pest and disease identification, and discloses a pepper pest and disease identification method and system based on a disease grading detection double-task model. 2 The sample image is subjected to enhancement processing by using a saliency target detection model U 2 The sample image is subjected to foreground and background segmentation by using U 2 The main body of the leaf in the image is retained, the authenticity and integrity of the disease characteristics are ensured, and the diversity of the data is increased to achieve the purpose of data enhancement.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pest and disease identification, and particularly relates to a pepper pest and disease identification system based on a disease grading detection double-task model. BACKGROUND

[0002] At present, pepper is one of the vegetable types commonly planted by farmers in China. With the continuous expansion of the pepper planting scale, the pepper may be affected by the bad environment or infected by harmful organisms such as pathogenic microorganisms during the growth and development process, resulting in a series of changes in the tissue structure or morphology of the pepper. The harm of pests and diseases is very serious, which can cause great loss to pepper production. If the pests and diseases can be found as early as possible, the farmers can use targeted medicine and take reasonable prevention and control measures to avoid large-scale economic losses and environmental pollution problems caused by the abuse of medicine. This is a key link to ensure the yield and quality safety of crops and directly affects the development of agricultural planting economy.

[0003] Using sensors, communication technology and computers and other modern information technology to solve the problems that may be encountered in the process of agricultural production, and realizing automatic and intelligent management are the key to the development of smart agriculture. Smart agriculture not only solves the problem of the lack of agricultural labor force, but also effectively improves the agricultural ecological environment and realizes the fine, efficient and green development of agriculture. In order to solve the problems of multiple types of crop pests and diseases, fast transmission speed and delayed best prevention period caused by manual identification of pests and diseases, a series of classification methods have appeared, but there are also certain defects. For the traditional manual identification and machine learning method of crop pest and disease identification, the data preprocessing process is complex and requires a lot of manpower and time. For example, existing technology 1 uses hyperspectral imaging technology for wheat disease images, and uses support vector machine and principal component analysis method for dimension reduction modeling, and finally the recognition accuracy of the classification task of wheat powdery mildew and stripe rust reaches 92%. Existing technology 2 identifies three grape diseases through a support vector machine algorithm, and finally the accuracy reaches 93.41%. They use a cascaded fusion method to process the image RGB color moment, HSV color histogram feature, GLCM texture feature and HOG feature to obtain multi-dimensional grape disease features. Existing technology 3 uses gray histogram features combined with SVM to identify rice leaf diseases, achieving an accuracy of 98.4%.

[0004] With the development of the times and the progress of science and technology, more and more new technologies are applied to agricultural production. Deep learning, with its ability to learn effective features autonomously, is widely used in crop disease and pest identification and can achieve good recognition results. For example, prior art 4 proposes an SSD with an initial module and a rainbow connection (INAR-SSD), which achieves an mAP of 78.8% compared to Faster R-CNN and SSD networks (mAP scores of 73.78% and 75.82%, respectively). Prior art 5 proposes a three-channel convolutional neural network (TCCNN), which implements three fully convolutional feature extraction networks and a fused dense network, with each fully convolutional network receiving only one of the three color channels of RGB. Prior art 6 proposes a method for detecting rice diseases and pests based on a convolutional neural network, which improves detection efficiency by reducing the size of the model. However, it only targets a single crop and a single disease, and its ability to identify multiple diseases is poor. Prior art 7 uses generative adversarial networks (GANs) to generate new leaf images. Several GAN structures (DCGAN, ProGAN, and StyleGAN) were tested, with StyleGAN being the most successful in generating leaf images within a 256x256 pixel range. However, these GAN networks have not been successful in training on images with complex backgrounds. Prior art 8 proposes a convolutional neural network model with 19 convolutional layers for identifying two diseases on apple leaves. It is compared with standard machine learning classifiers such as support vector machines, k-nearest neighbors, random forests, and logistic regression models. The experimental results show that the model achieves an accuracy of 99.2%, which is better than other CNN-based models and machine learning models. Prior art 9 uses an algorithm to segment the stripe rust spots in wheat spectral images and assess the disease severity of the leaf by calculating the ratio of the disease spot area to the total leaf area. This method achieves an accuracy of 98.15%. Prior art 10 proposes a neural network model called PD2SE-Net, which uses residual structures and shuffle units and classifies disease severity into three levels: healthy, general, and severe. This model can simultaneously diagnose plant diseases and estimate severity, achieving an average grading accuracy of 91% on the PlantVillage dataset.

[0005] Through the above analysis, the problems and defects of the prior art are:

[0006] (1) The data sets used in current technical research are mostly images taken in a single background in the laboratory. However, images taken in the field environment are affected by factors such as occlusion and lighting, and have complex background noise, making it difficult to extract effective features. Therefore, models trained in a single background perform poorly when applied to actual field environments with complex backgrounds.

[0007] (2) Deep learning relies on a large amount of experimental data, while taking a large number of crop images in the field is a huge project that is time-consuming, labor-intensive, and inefficient. In addition, the occurrence of diseases is random, making it difficult to collect the corresponding sample data and ensure a certain amount of data.

[0008] (3) There are few existing deep learning-based methods for simultaneously identifying crop diseases and classifying the severity of damage, and there are no publicly available classification datasets. The ability to determine the severity of diseases is crucial, as it allows agricultural personnel to take more accurate treatment measures based on different levels of damage.

[0009] (4) Compared with the classification of different crop diseases, the assessment of the severity of crop diseases at a fine-grained scale is more challenging because there is greater intra-class similarity and smaller inter-class differences. Classification methods that are good at identifying disease types are not sufficient to complete the task of accurately classifying the degree of damage.

[0010] The difficulty in solving the above problems and defects lies in:

[0011] (1) There is a lack of publicly available datasets for research and there is a problem of uniformity.

[0012] (2) The problem of high intra-class similarity and low inter-class difference in the estimation of disease severity makes it difficult for the network to extract effective fine-grained features. Summary of the Invention

[0013] To address the problems existing in the prior art, this invention provides a chili pepper disease and pest identification system based on a dual-task model for disease classification and detection.

[0014] This invention is implemented as follows: a method for identifying pepper diseases and pests based on a dual-task model for disease classification and detection, wherein the method for identifying pepper diseases and pests based on the dual-task model for disease classification and detection includes:

[0015] Using salient target detection model U 2 -Net enhances the sample images to enrich the dataset while balancing the differences in data volume between categories. It constructs a dual-task model for disease classification and severity grading using ResNet50 as the backbone network and two parallel fully connected layers with SE attention modules in the output layer. Transfer learning is introduced to accelerate the convergence speed of the model. The model is trained using a dataset with added synthetic images. The dual-task model for disease classification and severity grading is used to identify the types of pepper diseases and grade the severity of damage.

[0016] Furthermore, the method for identifying pepper diseases and pests based on a dual-task model for disease grading and detection includes the following steps:

[0017] Step one, obtain images of healthy and different diseases of pepper as sample images; construct a data set based on the sample images; and use a salient object detection model U 2 -Net to perform background segmentation and enhancement processing on the sample images in the data set to obtain synthetic images;

[0018] Step two, put the synthetic images processed by splicing and fusion of a plurality of real complex backgrounds back into the original training set, and mark them as corresponding disease categories and damage levels, and balance the data quantity difference between categories by adding different numbers of synthetic images;

[0019] Step three, construct a disease grading detection double-task model, and train and test the disease grading detection double-task model using the images in the data set, use the joint cross-entropy loss value of the two tasks to provide feedback to the network, and optimize the parameters of the neural network through the process of back propagation;

[0020] Step four, obtain a pepper image to be identified, and use the trained disease grading detection double-task model to identify the disease and grade the severity of the pepper image to be identified.

[0021] Further, the construction of the data set based on the sample images comprises:

[0022] The pepper leaf dataset is collected by manually shooting in the pepper planting shed, the PlantVillage public dataset and internet resources, the sample images are classified by disease and graded by severity, and the sample images are divided into a training set, a validation set and a test set in a ratio of 7:2:1;

[0023] Among them, the collected samples include four categories of images of pepper bacterial spot disease, powdery mildew, viral disease and healthy leaves, and the severity is divided into four levels: 0 level for healthy, 1 level for mild, 2 level for moderate, and 3 level for severe.

[0024] Further, the salient object detection model U 2 -Net performs background segmentation and enhancement processing on the sample images to obtain synthetic images, which comprises:

[0025] The pre-trained salient object detection model U 2 -Net performs foreground and background segmentation on part of the collected sample images with clear targets, rotates, moves and processes the segmented leaf images, and replaces the leaf images with different field complex backgrounds to obtain synthetic images, which are mathematically expressed as:

[0026]

[0027] wherein: x i denotes a data set original image; x bg denotes a background image; u(·) denotes U 2 a Net model processing process; M + (·) denotes conversion to a binary mask; M - (·) denotes an inverse mask value; λ denotes a set gray threshold; and & denotes a bit AND operation.

[0028] Further, the salient object detection model U 2 -Net is composed of six levels of encoders, five levels of decoders, and a saliency mapping fusion module connected with the decoder level and the last encoder level;

[0029] The salient object detection model U 2 -Net is based on a RSU structure that mixes different sizes of receptive fields, which is used to extract intra-level multi-scale features and ensure that the resolution of the feature map is unchanged.

[0030] Further, the disease grading detection double-task model comprises a backbone network, an SE attention module, and two parallel fully connected layers;

[0031] The backbone network is an improved ResNet50 model;

[0032] The SE attention module is connected after the last convolution layer in the Block of the improved ResNet50 model, and is used for locating information of interest;

[0033] The two parallel fully connected layers are connected in parallel at the last fully connected layer of the improved ResNet50 model; the two parallel fully connected layers are respectively set as the output result numbers of the disease category identification task and the severity grading task;

[0034] The disease grading detection double-task model weight is trained to jointly minimize two task loss functions.

[0035] Another purpose of the present application is to provide a pepper disease and pest identification system based on the disease grading detection double-task model for implementing the pepper disease and pest identification method based on the disease grading detection double-task model, and the pepper disease and pest identification system based on the disease grading detection double-task model comprises:

[0036] A data set construction module is used to obtain images of healthy peppers and different diseases as sample images, and construct a data set based on the sample images;

[0037] A synthetic image generation module is used to generate a synthetic image by using the salient object detection model U 2The Net performs background segmentation and enhancement processing on sample images in the data set to obtain synthetic images, thereby enriching the data set and balancing the data amount of each category;

[0038] The double-task model construction and training module is configured to construct a disease grading detection double-task model, and train the disease grading detection double-task model using images in the data set, update model parameters using a momentum-based stochastic gradient descent optimizer, and obtain an optimal model.

[0039] The disease grading detection module is configured to obtain a pepper leaf image to be identified, and perform disease identification and severity grading on the pepper leaf image to be identified using the trained disease grading detection double-task model.

[0040] Another object of the present application is to provide a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to enable the processor to perform the steps of the pepper disease and pest identification method based on the disease grading detection double-task model.

[0041] Another object of the present application is to provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to enable the processor to perform the steps of the pepper disease and pest identification method based on the disease grading detection double-task model.

[0042] Another object of the present application is to provide an information data processing terminal for implementing the pepper disease and pest identification system based on the disease grading detection double-task model.

[0043] In combination with the above technical solutions and solved technical problems, the technical solution to be protected by the present application has the following advantages and positive effects:

[0044] Firstly, the present application has the following advantages:

[0045] (1) Strong landing performance: The data set used in the present application collects a part of sample images of healthy peppers and bacterial spot disease from the PlantVillage website, and manually takes a part of pepper leaf images at the pepper planting base of Chengdu Academy of Agricultural and Forestry Sciences. These images are directly taken in the complex background of the pepper shed. By using single background and complex background image data, more representative features can be extracted, and the trained model is less affected by the complex environment, and is more suitable for making rapid and accurate judgments on disease conditions in crop planting bases.

[0046] (2) Improve data utilization: in view of the difficulty and large amount of engineering of collecting a large amount of data on the spot, the present application replaces the different complex backgrounds of the existing images by means of the salient object detection network to synthesize new image data, fully utilizes the existing data, and effectively realizes the diversity of the data set, avoiding the inefficient and laborious engineering of manually switching different scenes for shooting in the actual pepper planting base.

[0047] (3) Can complete the double task detection of disease type and damage degree: the present application adopts the feature extraction shared backbone network mode, extracts the joint features of disease type classification and damage degree classification, and finally completes the result output of the two tasks through two classifiers. While identifying the disease type, the severity evaluation is given, which can help agricultural personnel to take prevention and control measures more timely and accurately, because the estimation of disease severity is the basis for determining the type and dose of medicine when taking prevention and control measures.

[0048] (4) High recognition accuracy: the present application adds a channel attention module in the shared backbone network ResNet50 of fine-grained feature extraction, models the importance of each feature channel, and then selects to enhance or suppress the corresponding feature channel according to different tasks, so as to locate the information interested by the network, improve the extraction ability of the network to key features, and use the Stochastic Gradient Descent with Momentum (SGDM) optimizer to minimize the joint cross-entropy loss value of the two tasks to optimize the model, so that the accuracy of the final model for disease identification and degree classification of pepper leaves under complex background can reach more than 90%.

[0049] Secondly, the present application uses a salient object detection model U 2 -Net to perform foreground-background segmentation on sample images, retains the main body of the leaves in the images, can ensure the authenticity and integrity of the disease characteristics, and can also increase the diversity of the data to achieve the purpose of data enhancement.

[0050] The present application extracts the joint features of the two tasks by sharing the network, which can reduce the complexity of the model. This shared network mode improves the data efficiency, provides a faster learning speed for related tasks, and helps to alleviate the problem of large-scale computing demand of deep learning. The shared model weights are trained to jointly minimize the loss functions of the two tasks.

[0051] The present application improves the attention degree of important feature information of the model by adding the SE attention mechanism in the Block of the ResNet50 model, introduces the transfer learning to speed up the convergence speed of the model, combines the use of the data set after adding the synthetic image to complete the training of the model, and finally the recognition effect of the model is obviously improved.

[0052] The application can judge and identify multiple types of diseases and multiple severities of diseases at the same time, has high accuracy, can perform high-precision identification on the basis of a small amount of sample data, can be applied to images in real complex scenes, has strong generalizability and high practical application value.

[0053] Thirdly, the creativity of the application as the claim is also reflected in the following two important aspects:

[0054] (1) The expected income and commercial value of the technical scheme of the application after transformation are:

[0055] The method of artificially detecting crop diseases is easily affected by the subjective factors of the observers, and since there are differences in experience and ability of different observers, the identification results of the diseases may be biased, and the method is low in efficiency and time-consuming and laborious, and in addition, the general farmers do not have professional knowledge to correctly diagnose the diseases, so it is urgent to provide support for them by using modern information technology. The technical scheme of the application can be applied to the rapid and accurate automatic detection of pepper diseases in a complex field environment, brings convenience to pepper growers, and is more helpful to improve the prevention and treatment effect of pepper diseases, reduce the yield loss and economic loss of peppers, etc. At the same time, targeted medication and reasonable control measures can avoid environmental pollution caused by drug abuse. Designing a mature intelligent detection system can be commercially popularized and used.

[0056] (2) Whether the technical scheme of the application overcomes the technical bias:

[0057] In the field of image classification, the researchers have always simplified the images of the data set, as much as possible to crop or segment the target area to feed the neural network to learn, reduce the interference of irrelevant information to the network, and promote the network to learn effective features, but it does not fundamentally improve the filtering ability of the network for complex information. The application adopts the U 2 -Net segmentation algorithm to segment the pepper leaves, and splices and fuses the collected background images to generate new synthetic images under different complex backgrounds to increase the diversity of the data set, simulates the effect of switching different scene shooting samples, and this way of enriching the data set is more effective than the traditional data enhancement method, and the image classification effect of the finally trained model for complex background will be better. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is the method principle diagram of the U 2 -Net based synthetic image generation method provided by the embodiment of the application;

[0059] Figure 2This is a schematic diagram of the steps of the pepper disease and pest identification method based on a dual-task model for disease classification and detection provided in an embodiment of the present invention;

[0060] Figure 3 This is a flowchart of the chili pepper disease and pest identification method based on a dual-task model for disease classification and detection provided in this embodiment of the invention;

[0061] Figure 4 This is a structural block diagram of the chili pepper disease and pest identification system based on the dual-task model for disease classification and detection provided in this embodiment of the invention.

[0062] Figure 5 This is a diagram of a dual-task shared network architecture for disease classification and detection provided in an embodiment of the present invention;

[0063] Figure 6 This is the overall network layer architecture diagram of the dual-task model for disease classification and detection provided in this embodiment of the invention;

[0064] Figure 7 This is a schematic diagram of a traditional image enhancement method provided in an embodiment of the present invention;

[0065] Figure 8 This is a schematic diagram of the accuracy and loss curves of the model as iterates over two tasks, provided in this embodiment of the invention.

[0066] Figure 9 This is a schematic diagram of the recognition rate confusion matrix for two tasks on the test set: classification of pepper disease types and grading of damage severity, provided by an embodiment of the present invention.

[0067] Figure 10 This is a schematic diagram of a smart terminal APP for identifying and classifying the severity of chili diseases, provided in an embodiment of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0069] like Figures 1-3 As shown, the pepper disease and pest identification method based on a dual-task model for disease grading and detection provided in this embodiment of the invention includes:

[0070] S101, acquire images of healthy chili peppers and different diseases as sample images; construct a dataset based on the sample images; and utilize the salient object detection model U. 2 -Net performs background segmentation and enhancement processing on the sample images in the dataset to obtain synthetic images;

[0071] S102, the synthesized images processed by respectively using a plurality of real complex background splicing and fusion are put back into the original training set, and are marked as corresponding disease categories and damage degrees, and the data amount difference between categories is balanced by adding different numbers of synthesized images;

[0072] S103, a disease grading detection double-task model is constructed, and the disease grading detection double-task model is trained and tested using the images in the data set, a joint cross-entropy loss value of two tasks is used to provide feedback to the network, and the parameters of the neural network are optimized through the process of back propagation;

[0073] S104, a pepper image to be recognized is acquired, and the trained disease grading detection double-task model is used for disease recognition and severity grading of the pepper image to be recognized.

[0074] As shown in Figure 4 the pepper disease and pest recognition system based on the disease grading detection double-task model provided by the embodiment of the application comprises:

[0075] A data set construction module is configured to acquire images of healthy peppers and different diseases as sample images, and construct a data set based on the sample images.

[0076] A synthesized image generation module is configured to use a salient object detection model U 2 -Net to perform background segmentation and enhancement processing on the sample images in the data set, so as to obtain synthesized images, enrich the data set, and balance the data amount of each category.

[0077] A double-task model construction and training module is configured to construct a disease grading detection double-task model, and train the disease grading detection double-task model using the images in the data set, update the model parameters using a momentum-based stochastic gradient descent method optimizer, and obtain an optimal model.

[0078] A disease grading detection module is configured to acquire a pepper leaf image to be recognized, and use the trained disease grading detection double-task model to perform disease recognition and severity grading on the pepper leaf image to be recognized.

[0079] The pepper disease and pest recognition method based on the disease grading detection double-task model provided by the embodiment of the application comprises the following steps:

[0080] (1) Synthesized image

[0081] The salient object detection model U 2-Net performs foreground-background segmentation on the collected part of the target-specific sample images, retains the main body of the leaf in the image, and the purpose is to ensure the authenticity and integrity of the disease characteristics. Then the segmented leaf is processed by traditional rotation, movement, etc., and different field complex backgrounds are replaced to obtain synthetic images. Although from the human's priori, the synthetic image and the original image belong to the same distribution, and the label has not changed, but for a convolutional neural network, it is a different image sample.

[0082] U 2 -Net is composed of two layers of nested U-shaped structures, and does not use any pre-trained backbone network from image classification. It mainly consists of six-level encoders, five-level decoders, and a saliency mapping fusion module connected to the decoder level and the last encoder level. The bottom layer is a new ReSidual U-blocks (RSU) structure, which mixes different sizes of receptive fields, can extract intra-stage multi-scale features and ensure the resolution of feature maps unchanged. This nested U-shaped structure can more effectively extract intra-stage multi-scale features and aggregate inter-stage multi-level features.

[0083] In existing image classification tasks, most of the time, U 2 -Net performs foreground-background segmentation on the dataset, the purpose is to simplify the dataset image, cut off the useless background information, and only send the target object part with a lot of information into the neural network training and learning. But this way also has limitations, for the complex background, the target is fuzzy or the number is more, the foreground and background of this kind of image often cannot be correctly segmented. And input the simplified image into the network, although it can promote the model to learn the correct weight parameters, but it does not fundamentally improve the network's ability to extract effective features, which is a temporary solution. Therefore, the present application uses U 2 -Net to complete the background replacement of the dataset image, does not simplify the image, only to increase the diversity of data to achieve the purpose of data enhancement. The implementation process of the method is as shown in Figure 1 .

[0084] The mathematical expression of the synthetic image operation is as follows:

[0085]

[0086] In the formula: x i is the original image of the dataset; x bg is the background image; u(·) is the U 2 -Net model processing process; M + (·) represents conversion to a binary mask; M - (·) represents the inverse mask value; λ represents the set gray threshold value; & represents the bit AND operation.

[0087] (2) Dual-task network

[0088] There are two categories of existing methods for estimating disease severity using image processing techniques: quantitative evaluation and hierarchical qualitative evaluation. Quantitative evaluation requires image segmentation techniques to segment the leaf and lesion areas. In order to reduce the complexity of the network, the present application adopts a deep learning method to perform hierarchical evaluation of disease severity, and different levels of damage are considered as different categories to feed the neural network model for training and learning.

[0089] Since the present application is dedicated to solving the two tasks of disease type identification and severity classification, these two tasks have a strong correlation. In order to simplify the network model, the two tasks share a backbone network, which allows the network to learn joint features useful for both tasks. Finally, two parallel fully connected layers are used to complete the identification of pepper disease types and the classification of damage severity. This shared network approach improves data efficiency, provides faster learning speed for related tasks, and helps to alleviate the problem of large-scale deep learning computation. The shared model weights are trained to jointly minimize the loss functions of the two tasks. Loss refers to the difference between the model's predicted results and the actual results. The smaller the loss function value, the closer the model's predicted results are to the actual results, and the better the model performance. The present application uses a cross-entropy loss function to measure the difference between the distribution learned by the multi-classification model and the true distribution. The cross-entropy loss calculation formula is as follows:

[0090]

[0091] In the formula: N represents the number of samples; M represents the number of categories; y ij is a symbolic function, indicating that the i-th sample belongs to the j-th category of the true label, if the true class of i is equal to j, then y ij takes the value of 1, otherwise takes the value of 0; p ij represents the probability that the model predicts that the i-th sample belongs to the j-th category; log represents the natural logarithm with base e.

[0092] It should be noted that since the present application is a dual-task network model, the cross-entropy loss of the two classification tasks is first calculated, and then the average of the two loss values is taken as the final loss of the model described in the present application. The loss value is used to provide feedback to the network to guide the next training and optimization direction. The loss calculation formula is as follows:

[0093]

[0094] In the formula: L dis represents the cross-entropy loss value for classifying pepper disease types, and L sevCross-entropy loss value representing the severity grading of the pepper damage.

[0095] The specific method of the present application is to use ResNet50 as the backbone network for extracting the joint features of the two tasks, add a fully connected layer in parallel at the last fully connected layer of the original network, and set it as the output of the corresponding task, such as defining 4 categories of output in the disease type identification task and the severity grading task, which means that the network can make judgments on 4 disease types and 4 severities. The double-task network structure is as shown in Figure 5

[0096] (3) Improve ResNet50

[0097] Since the present application is to solve the problem of pepper disease identification under complex background, the feature extraction ability of the network is required to be higher. The attention mechanism in deep learning is to focus on the key information more meaningful to the current task target from a large amount of information, which is similar to the selective visual attention mechanism of human beings. It filters out a large amount of irrelevant useless information through a top-down information selection mechanism, which is generally in the form of probability graph or probability feature vector, which can effectively improve the network's ability to extract key features.

[0098] The specific method of the present application is to add an SE attention module in the Block of ResNet50 to help the network locate the information of interest and improve the network's ability to extract key features, so as to effectively improve the recognition accuracy of the final model. The purpose of the SE module is to give different weights to different positions of the image from the channel domain through a weight matrix, and then rescale the feature map to enhance the feature representation of the input data. The key operation mainly includes two steps of squeezing (Squeeze) and excitation (Excitation). Since the present application is aimed at the disease problem of pepper leaves, the position of the disease occurrence is random and unpredictable, but the same disease has a certain correlation. Obviously, the channel-based attention mechanism is more suitable for the feature extraction network of the present application than the space-based attention mechanism.

[0099] ​ResNet50 has two block structures of Conv Block and Identity Block, wherein the input and output dimensions of the Conv Block are different, so that the channel adjustment of the residual edge is required by using the 1*1 convolution, and then the matrix addition of the main branch and the shortcut branch can be performed; and the input and output dimensions of the Identity Block are the same, so that the network can be deepened by being continuously connected in series. In order to facilitate the description, the two blocks added with the SE attention module are respectively named as SE_ConvBlock and SE_Identity Block, and the whole network layer architecture of the disease grading detection double-task model of the application is as shown in Figure 6 The shape of the input pepper leaf image is (224, 224, 3), that is, the image channel number is 3 and the size is 224*224. The model is finally connected by two FC full connection layers to connect the channel 2048 and the num_class (the output num_class of the two tasks in the embodiments described in the application is 4) to output.

[0100] It should be noted that the embodiments described in the application are only used to describe the technical solutions and are not limited, and appropriate changes can be made in data and details.

[0101] comprising the following steps:

[0102] Step 1: data set acquisition:

[0103] The data set used in the application is obtained by two ways. First, a part of the sample images of the healthy pepper and bacterial spot disease of the pepper are collected from the PlantVillage website, and these images are all taken in a single background in the laboratory, and the size is uniform, that is, 256 pixels*256 pixels. In addition, a part of the pepper leaf images are manually taken in the pepper planting base of Chengdu Academy of Agricultural and Forestry Sciences, and these images are directly taken in the complex background of the pepper shed, and have different size.

[0104] Step 2: data preprocessing:

[0105] The collected sample images are then classified and graded by agricultural experts according to the severity of the disease, which is divided into four levels: 0 level for healthy (Healthy), 1 level for mild (Mild), 2 level for moderate (Moderate), and 3 level for severe (Serious). The grading standard is divided according to the proportion of disease spots in the whole leaf area: 0 level without disease spots; 1 level disease spots ≤ 20%; 20% < 2 level disease spots ≤ 50%; 3 level disease spots > 50%. In this embodiment, samples of four disease types of pepper leaf, i.e. Healthy, Bacterial, Powdery Mildew and virus, are finally selected. Because the present application is to solve the problem of disease recognition in complex field background, the samples in the test set should all be leaf images in complex background, but due to the limited number of samples taken in the field, the overall data set is divided into training set, validation set and test set in the ratio of 7:2:1, and the specific number is shown in Table 1. Among them, the validation set is used for model parameter optimization, and the test set is used to finally evaluate the generalization ability of the model of the present application. Before sending the image data into the feature extraction network, the size is unified to 224x224 pixels. And traditional image enhancement methods are used: horizontal and vertical mirroring, rotation, translation, scaling and contrast adjustment, Figure 7 Some traditional image processing operations are shown.

[0106] Table 1: Number of Pepper Leaf Disease Sample Information

[0107]

[0108]

[0109] Step 3: Generate synthetic images:

[0110] U 2 -Net is a new deep network architecture based on U-Net, which is simple but powerful, and is often used for salient object detection, which can segment the most visually attractive objects in the image. The present application will directly use the model parameters pre-trained by the designer on DUTS-TR, and will not retrain the U 2 -Net, because the pre-trained U 2 -Net has achieved very good results for salient object segmentation. First, several real field background images for replacing the background in the future are collected on the Internet, and then some leaf images are randomly selected in each category of the existing training data set to segment the whole leaf by U 2 -Net, and then the background image is spliced to generate a synthetic image.

[0111] The specific operation includes the following four steps:

[0112] 1) Randomly selected 556 images from the existing data set to input the pre-trained U-Net model 2 - The U-Net model generates a mask image of the leaf, and it is noted that the selected image is preferably easy to segment;

[0113] 2) Then, the gray mask image is converted into a binary mask according to a given threshold, and then combined with the original image to obtain a leaf segmentation image;

[0114] 3) After adjusting the background image to the size of the original data set, the processed inverse mask value and the background image are used for AND operation;

[0115] 4) Finally, the leaf segmentation image and the background image processed by the inverse mask are spliced to obtain a new synthetic image with a changed background.

[0116] Step 4: Constructing a double-task network model:

[0117] The improved ResNet50 model is used as the backbone network for feature extraction. First, an SE attention module is added after the last convolution layer in the Block of ResNet50, and the size of the feature map will not change. Then, a fully connected layer is added in parallel at the last fully connected layer of ResNet50, and the number of output results of the two tasks is set respectively, for example, in the disease type recognition task and the severity classification task in this embodiment, both are defined as 4-class output, which means that the network can judge 4 types of diseases and 4 degrees of severity.

[0118] In order to improve the training efficiency of the model, the transfer learning technology is introduced. First, the improved double-task network model based on ResNet50 is built on the local experimental platform. Then, the pre-trained model weight file of ResNet50 on the ImageNet data set is loaded. In this application, the model is trained end-to-end without freezing any layer.

[0119] Step 5: Hyperparameter setting:

[0120] Before training, some hyperparameters need to be set, such as learning rate, generally set to 0.01, 0.001, 0.0001, momentum, generally set to 0.3, 0.6, 0.9, a total of three momentum gradients. After many experiments in this embodiment, the batch size is set to 24, and the training epoch is 100. In addition, the initial learning rate is set to 0.01, and the learning rate decay strategy is adopted, and the weight decay value is 0.0005, so that the model can converge more stably. In order to achieve better training effect, the momentum stochastic gradient descent optimizer is used to update the model parameters, and the momentum factor is set to 0.9.

[0121] Step 6: training the optimal model

[0122] 1) Configuration: the present application uses the SGDM optimizer and the commonly used cross-entropy loss function, and the network is fed back and optimized by the joint cross-entropy loss value of the two tasks, the activation function uses ReLU, and the evaluation index of the network model recognition effect is accuracy and loss.

[0123] 2) Training: set the training round epoch to 100, and perform integer division operation on the total number of samples and the batch size to obtain the result as each training batch. After all the batches of each epoch are trained, the identification accuracy of the verification set is tested, the network provides feedback according to the accuracy of the verification set, and the loss value after training a batch is adjusted to optimize the model. The model with the two task accuracy change curve and the loss curve of the iteration period are as shown in Figure 8 It can be seen that the training accuracy of the two tasks gradually approaches 100%, and the convergence speed of the model is also very fast. The model basically reaches the convergence state at the 45th epoch.

[0124] 3) Save:

[0125] After the first round of epoch training is completed, the model parameters are saved in.pth format to the local, and the F1 Score is recorded and set as the best F1 Score. Then compare the F1 Score of each subsequent epoch with the best F1 Score, if the current F1 Score is higher, save the new parameters to overwrite the historical.pth file, and update the best F1 Score to the current value. Repeat the above process until all epoch training is completed, that is, the optimal model is obtained.

[0126] The optimal model obtained by training is used on the test set of the embodiment to perform inference prediction, and then compared with the real class label to prove the final recognition effect of the model. It should be noted that the present application is to solve the problem of pepper diseases in complex background, so the images in the test set are all taken in real complex background. The recognition rate confusion matrix of the two tasks of pepper disease species classification and damage degree classification on the test set is as shown in Figure 9As shown, the accuracy of the pepper disease type classification task is 92.15%, and the recognition accuracy of the other three types is more than 95% except for pepper powdery mildew. Because the data set of pepper powdery mildew is completed by multiple shooting, the scene and leaf growth period are quite different, and it is also affected by light and shooting angle, so the final recognition effect is not good. The accuracy of the damage severity classification task is 92.67%, although 28 samples are misclassified in the severity classification, most of the errors are in the adjacent level of the true value, because sometimes the classification of the level itself is uncertain, such as the samples at the critical point of moderate and severe levels, it is difficult to determine which level it is, so this kind of discrimination error is considered to be slight.

[0127] It should be noted that the embodiments of the present application can be realized by hardware, software or a combination of software and hardware. The hardware part can be realized by special logic; the software part can be stored in the memory and executed by the appropriate instruction execution system, such as microprocessor or special designed hardware. Those skilled in the art can understand that the above-mentioned devices and methods can be realized by computer executable instructions and / or included in processor control code, such as carrier medium, such as magnetic disk, CD or DVD-ROM, programmable memory, such as read-only memory (firmware), or data carrier, such as optical or electronic signal carrier. The device of the present application and its modules can be realized by hardware circuit, such as ultra large scale integrated circuit or gate array, semiconductor, such as logic chip, transistor, etc., or programmable hardware device, such as field programmable gate array, programmable logic device, etc., can also be realized by software executed by various types of processors, and can also be realized by the combination of the above hardware circuit and software, such as firmware.

[0128] In order to overcome the defects existing in the prior art, the present application proposes a pepper disease degree grading detection double task model used in real complex environment. The structure, purpose and technical effects of the present application will be further described in detail through specific embodiments.

[0129] The pepper disease and pest recognition system based on the disease grading detection double task model is based on the model with the best training effect after adding the synthetic image expansion data set, and an intelligent terminal system for pepper disease recognition and severity grading detection is designed and developed. Users can obtain pepper disease information through simple operation of mobile phone APP.

[0130] The system backend application is developed using the Spring Boot framework, which uses a convention-over-configuration approach to reduce the workload of developers and provides a series of predefined configurations and default values, so that developers can quickly build and deploy the application. First, the trained disease classification and detection double-task model needs to be converted into a pb format file, and the user-uploaded pepper leaf image needs to be resized and dimensionally changed before being input into the model for disease detection. Then the detection results are returned to the mobile APP end.

[0131] To verify the actual application effect of the embodiments of the present application, a client APP with simple interface structure and easy operation is designed on the Android platform, the control layout is clear, and the result display is clear, so that the user can quickly get started. The main function of the APP end is to perform online recognition on the picture selected by the user, upload the picture selected by the user to the server, and return the result after model recognition. The APP end is developed with the help of HBuilderX development tool and using the uni-app front-end framework.

[0132] The user can select the pepper leaf image that needs to be identified by uploading the picture from the local gallery or directly taking a picture by calling the camera. After selecting the picture, the user can upload the picture to the server end by clicking the "online recognition" button, and the server end will call the model code to predict the disease result and return the result to the mobile APP. In order to interact with the server end, the front end needs to set the IP connection configuration first, and the user transmission image data interaction process is realized by Java language. After the server end returns the recognition result, the pepper disease type and disease degree will be displayed below the picture. If it is a healthy leaf, only the disease type is displayed as healthy, and there is no result for the disease degree, as shown in Figure 10 .

[0133] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any modification, equivalent replacement and improvement made by those skilled in the art within the technical scope disclosed by the present application, within the spirit and principles of the present application, should be covered within the protection scope of the present application.

Claims

1. A method for detecting a pepper disease and pest recognition method based on a disease grading detection double-task model, characterized in that, The application relates to a pepper disease and pest recognition method based on a disease grading detection double-task model. Utilizing salient object detection models The sample image is enhanced, the data set is enriched, the data amount difference between various categories is balanced, a main network with ResNet50 as an extraction disease category classification and a joint feature of severity grading two tasks is constructed, a disease grading detection double task model with two parallel full connection layers and an SE attention module is adopted in an output layer, a transfer learning is introduced to accelerate the convergence speed of the model, the model is trained by combining the data set after adding the synthetic image, and the disease grading detection double task model is used for identifying the pepper disease category and grading the damage severity. The pepper disease and pest recognition method based on the disease grading detection double-task model comprises the following steps: Step one, obtain the image of healthy and different diseases of pepper as sample image; based on the sample image, construct a data set; and use a salient object detection model The sample images in the data set are subjected to background segmentation and enhancement processing to obtain a synthetic image; Step 2: the synthesized images processed through splicing and fusion of multiple real complex backgrounds are put back into the original training set and marked as corresponding disease categories and damage degrees, and the data amount difference between different categories is balanced by adding different numbers of synthesized images; Step 3: a disease grading detection double-task model is constructed, and the disease grading detection double-task model is trained and tested by using the images in the data set, the joint cross-entropy loss value of the two tasks is used to provide feedback to the network, and the parameters of the neural network are optimized through the process of back propagation; Step 4: a to-be-recognized pepper image is acquired, and the trained disease grading detection double-task model is used to recognize the diseases and grade the severity of the to-be-recognized pepper image. The utilizing salient object detection model The method for background segmentation and enhancement processing of sample images to obtain a composite image comprises: Utilizing a pre-trained salient object detection model The part of the target clear sample image collected is subjected to foreground and background segmentation, the leaf image obtained by the segmentation is subjected to rotation and movement processing, and different field complex backgrounds are replaced for the leaf image to obtain a synthetic image, and the mathematical formula is as follows: In the formula, indicates a data set original drawing; indicates a background image; indicates a model processing process; indicates conversion into a binary mask; indicates an inverse mask value; indicates a set gray threshold value; indicates a bit AND operation.

2. The pepper disease and pest recognition method based on the disease classification detection double-task model according to claim 1, wherein, The data set is constructed based on sample images, and comprises the following steps: The pepper leaf data set is collected through manual shooting in a pepper planting shed, a PlantVillage public data set and internet resources, the sample images are classified according to diseases and graded according to severity, and the sample images are divided into a training set, a verification set and a test set according to a 7:2:1 ratio. The sample images include four categories of images of pepper bacterial spot, powdery mildew, viral disease and healthy leaves, and the severity is divided into four levels: 0 level for healthy, 1 level for mild, 2 level for moderate and 3 level for severe.

3. The method of claim 1, wherein the method is characterized by, The saliency object detection model consists of a six-level encoder, a five-level decoder, and a saliency mapping fusion module connected with the decoder level and the last encoder level. The saliency object detection model The bottom layer is an RSU structure that mixes different sizes of receptive fields, used to extract multi-scale features within a stage and ensure that the resolution of the feature map remains unchanged.

4. The pepper disease and pest recognition method based on the disease classification detection double-task model according to claim 1, wherein, The disease grading detection double-task model comprises a backbone network, an SE attention module and two parallel full connection layers. The backbone network is an improved ResNet50 model. The SE attention module is connected to the last convolution layer of the improved ResNet50 model and is used for locating information of interest. The two parallel full connection layers are connected to the last full connection layer of the improved ResNet50 model, and the two parallel full connection layers are respectively set as the output result numbers of a disease category recognition task and a severity grading task. The disease grading detection double-task model weight is trained to jointly minimize the loss functions of the two tasks.

5. A pepper disease and pest recognition system based on a disease classification detection double task model implementing the method of claim 4, characterized in that, The pepper disease and pest recognition system based on the disease grading detection double-task model comprises: A data set construction module is used for acquiring images of healthy peppers and different diseases as sample images and constructing a data set based on the sample images. The synthetic image generation module is configured to utilize a saliency object detection model The sample images in the data set are subjected to background segmentation and enhancement processing to obtain synthetic images, thereby enriching the data set and balancing the data amount of each category. A double-task model construction and training module is used for constructing a disease grading detection double-task model, training the disease grading detection double-task model by using the images in the data set, updating the model parameters by using a momentum stochastic gradient descent method optimizer, and obtaining an optimal model. A disease grading detection module is used for acquiring a to-be-recognized pepper leaf image, recognizing diseases and grading the severity of the to-be-recognized pepper leaf image by using the trained disease grading detection double-task model.

6. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the pepper disease and pest identification method based on the disease classification detection double-task model according to any one of claims 1-4. 7.A computer readable storage medium, storing a computer program, the computer program is executed by a processor to make the processor execute the steps of the pepper disease and pest identification method based on the disease classification detection double-task model according to any one of claims 1-4.

8. An information data processing terminal, characterized by The information data processing terminal is used to implement the pepper disease and pest identification system based on the disease classification detection double-task model according to claim 5.

Images