A method for identifying highbush blueberry rot disease based on deep multi-task learning

Abstract

The present invention relates to a method for identifying highbush blueberry stunted fruit disease based on deep multi-task learning, and belongs to the field of plant disease identification. The method comprises: S1: collecting a highbush blueberry stunted fruit disease image dataset, annotating the image dataset to generate an infection site identification dataset, an infection stage division dataset, and an infection severity estimation dataset; S2: performing data amplification on the annotated image dataset; S3: building a deep multi-task blueberry stunted fruit disease identification model based on the EfficientNet model; S4: using the image dataset amplified in step S2 to train the deep multi-task blueberry stunted fruit disease identification model constructed in step S3; S5: using the trained model to identify the infection site of blueberry stunted fruit disease, divide the infection stage, and estimate the infection severity. The present invention can realize efficient and accurate identification of blueberry stunted fruit disease, and has great practical significance and value for agricultural disease monitoring.

Description

Technical Field

[0001] The present invention belongs to the field of plant disease identification and relates to a method for identifying highbush blueberry nectar disease based on deep multi-task learning. Background Art

[0002] At present, the automatic identification of crop diseases has been effectively developed, and the typical representatives are the emerging machine learning and deep learning technologies. The application of deep learning in images is becoming more and more extensive, and it is possible to realize the automatic identification and detection of highbush blueberry rot disease through computers. Plant disease identification includes a series of problems such as identifying the types of plant diseases, dividing the stages of disease infection, and estimating the severity of infection. At present, many scholars have studied many classic convolutional neural network models and applied them to many problems of plant disease identification. However, the current research ideas mainly adopt the idea of ​​single-task learning. For different problems of plant disease identification, convolutional neural network models are constructed to solve different tasks. This method ignores the correlation information between different problems of plant disease identification, resulting in low accuracy of disease identification and limited practical application.

[0003] Therefore, a new plant disease identification method is urgently needed to identify highbush blueberry rot disease. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method for identifying highbush blueberry berry blight disease based on deep multi-task learning. By constructing a deep multi-task blueberry berry blight disease identification model, the three problems of infection site identification, infection stage classification, and infection severity estimation of highbush blueberry berry blight disease identification are solved at the same time, thereby improving the accuracy and efficiency of berry blight disease identification. Correctly identifying the early infection stage of blueberry berry blight and reasonably estimating the severity of disease infection are important for disease prevention and control. Modern agricultural technical measures for crop disease prevention and control, such as determining the time and dosage of pesticide spraying and introducing commercial bees for pollination, all rely on accurate information on the infection time, infection site, and how the blueberry plant is infected. Specific information about identifying infection points on leaves, flowers, or fruits; classification of infection stages; and estimation of disease severity will help manage blueberry berry blight.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for identifying highbush blueberry rot disease based on deep multi-task learning specifically includes the following steps:

[0007] S1: Collect highbush blueberry rot disease image dataset, annotate the image dataset to generate infection site identification dataset, infection stage classification dataset and infection severity estimation dataset;

[0008] S2: Data augmentation is performed on the annotated image dataset. Since deep learning requires strong training data support, the three annotated disease datasets are augmented to increase the total amount of data samples.

[0009] S3: Build a deep multi-task blueberry nectar disease recognition model based on the EfficientNet model, and solve three problems of blueberry nectar disease recognition: infection site identification, infection stage classification, and infection severity estimation;

[0010] S4: using the image dataset amplified in step S2 to train the deep multi-task blueberry rot disease recognition model constructed in step S3;

[0011] S5: Use the trained model to identify blueberry berry blight infection sites, classify infection stages, and estimate infection severity.

[0012] Furthermore, in step S1, the collected highbush blueberry necrosis disease images include image data of blueberry flowers, leaves and fruits; the data collection method is to take pictures on the spot at the wild blueberry planting base in Northeast China and collect data through a web crawler.

[0013] The data annotation principles are as follows: 1) Infection site identification dataset: determine whether the main infected tissue in the image data is flower, leaf or fruit, and perform data annotation; 2) Infection stage classification dataset: according to the pathological characteristics of blueberry berry blight, the data is labeled as healthy stage, primary infection stage or secondary infection stage; 3) Infection severity classification dataset: estimate the area of ​​the infected area in the image and perform data annotation.

[0014] Furthermore, in step S2, the image data set is augmented, specifically by using an image operation data enhancement method based on geometric transformation, color transformation, and pixel transformation, and using random probability and appropriate combination strategies to perform offline data enhancement on the data set to enhance the richness of the generated image.

[0015] Furthermore, in step S3, the deep multi-task blueberry rot disease recognition model constructed mainly includes a parameter sharing module and three specific task modules; the improved EfficientNet model is selected as the parameter sharing module; the original EfficientNet model consists of 9 stages of operations, and the operations of the last stage are removed to build a parameter sharing module; the specific task module is built on the basis of the residual structure, integrating the coordinated attention mechanism and the grouped convolution operation.

[0016] Furthermore, in step S3, the specific task module is built by integrating the coordinated attention mechanism and the grouped convolution operation on the basis of the residual module, which reduces the model parameters while enhancing the feature expression; specifically, the coordinated attention mechanism is added after the second convolution layer of the residual structure, and the grouped convolution operation is added after the third convolution operation of the residual structure.

[0017] Furthermore, in step S4, the model is trained using transfer learning technology, including setting the maximum number of iterations, selecting the stochastic gradient descent optimization algorithm, and setting the learning rate; the data of the three tasks pass through the parameter sharing module in turn to obtain three feature vectors, and then the three feature vectors pass through their respective specific task module branches to complete the identification of the infection site of the blueberry rot disease, the division of the infection stage, and the estimation of the severity of the infection.

[0018] Furthermore, in step 5, the trained model is used to identify the infection site of blueberry berry blight, divide the infection stage, and estimate the infection severity. The model input is three pictures, corresponding to the three problems of three berry blight disease identification, and the output is the recognition results of the three problems.

[0019] The beneficial effects of the present invention are as follows: the blueberry rotten fruit disease identification method of the present invention has higher practical application value, and is constructed by using a lightweight convolutional neural network module, which can simultaneously detect the infection site and infection stage of the blueberry rotten fruit disease, and evaluate the severity of infection, thereby reducing the deployment cost of the model, improving the recognition efficiency, and providing theoretical guidance and technical support for the identification of crop diseases. In view of the problem of few training samples for blueberry diseases, the image data set is expanded and enhanced through various methods such as geometric changes, effectively avoiding the risk of overfitting of the model and optimizing the generalization ability of the diagnostic model; at the same time, a transfer learning strategy is used for model training to accelerate the model training process and improve the recognition accuracy.

[0020] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a dataset for identifying highbush blueberry rot disease, including: Figure 1 (a) is an example of an image in the infection site recognition dataset. Figure 1 (b) Divide the dataset image instances into infection stages, Figure 1 (c) An example of an infection severity estimation dataset image;

[0023] Figure 2 is an example of image enhancement, where Figure 2 The image in (a) is processed by geometric transformation. Figure 2 The image in (b) has been processed with color change. Figure 2 The image in (c) has been processed for pixel changes;

[0024] Figure 3 This is the structure diagram of the deep multi-task blueberry necrosis disease recognition model;

[0025] Figure 4 is a structure diagram of a specific task module, where: Figure 4 (a) is the residual module structure, Figure 4 (b) is a specific task module structure;

[0026] Figure 5 The changes in recognition accuracy and cross entropy of the three data sets on the training and validation sets during the training process;

[0027] Figure 6 The present invention is a flow chart of the method for identifying highbush blueberry rot disease based on deep multi-task learning. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] See also Figure 1 to Figure 6 The present invention provides a blueberry rot disease recognition model method based on deep multi-task learning, such as Figure 6 As shown, it mainly includes the following steps:

[0030] Step 1: Data collection and annotation.

[0031] Step 1.1: Data collection, plant disease identification mainly includes judging the type of plant infection disease, identifying the specific parts of plant infection (flowers, leaves, fruits, trunks), dividing the stages of plant disease infection, estimating the severity of plant infection, etc. However, the current domestic research on plant disease identification mainly adopts the idea of ​​single-task learning. For different problems of plant disease identification, models are established to achieve the goal of disease identification. There is no systematic study on plant leaf, fruit, flower, and trunk diseases by establishing a model to solve many problems of plant disease identification at one time. It is of great significance to effectively control the development of the disease and reduce economic losses by judging the infection parts (flowers, leaves, fruits) of high-bush blueberry rotten fruit disease, as well as the stage and severity of the disease development. Therefore, the present invention collects 1630 images of blueberry rotten fruit disease by field shooting and web crawler.

[0032] Step 1.2: Data annotation. With the assistance of plant disease identification experts, the collected blueberry necrosis disease image data were annotated to generate three data sets, namely, the infection site identification data set (such as Figure 1 (a)), infection stage division dataset (as shown in Figure 1 (b)), infection severity estimation dataset (as shown in Figure 1 (c)).

[0033] Data annotation criteria: 1) Infection site dataset: According to the images, the main infected blueberry disease tissues are flowers, leaves, and fruits, and a total of 6 categories of data are labeled, namely healthy and infected flowers, leaves, and fruits; 2) Infection stage classification dataset: The development stage of blueberry rotten fruit disease identification mainly includes the healthy stage, primary infection stage, and secondary infection stage. Therefore, this dataset contains 3 categories of image data. The healthy category contains healthy flowers, leaves, and fruit images. The primary infection stage contains images of flower buds and leaves that are initially infected by rotten fruit disease. Its main features are the withering of flower buds and the curling of leaves. The secondary infection stage contains images of flowers, leaves, and fruits that are further infected by rotten fruit disease. Its main features are the further withering and browning of flowers, the drying of leaves, and the shrunken fruits; 3) The infection severity estimation dataset consists of 5 categories. The severity is divided by evaluating the area of ​​diseased tissue in the image, namely healthy, very low, low, high, and very high.

[0034] Step 2: Data expansion and division.

[0035] Step 2.1: Data expansion. Since the collected blueberry rot disease recognition image samples are too few to support the development and verification of deep learning models, the trained model does not have generalization. In order to improve the classification and recognition accuracy and make the model more robust, the traditional data enhancement method is used to amplify the three labeled data sets from three dimensions. Figure 2 As shown, Figure 2 The image in (a) is processed by geometric transformation. Figure 2 The image in (b) has been processed with color change. Figure 2 The image in (c) has been processed for pixel changes. In order to ensure the balance of the total number of samples in the three datasets, the data expansion method is used to make the data of each sample reach 2400 images.

[0036] Step 2.2: Data division: For the three expanded data sets, randomly divide the training set and the test set in a ratio of 7:3.

[0037] Step 3: Construction of a deep multi-task blueberry necrosis disease recognition model.

[0038] The deep multi-task blueberry necrosis disease recognition model mainly consists of a parameter sharing module and three specific task modules, such as Figure 3 shown.

[0039] Step 3.1: Parameter sharing module, which is improved on the basis of the EfficientNet network structure. As shown in Table 1, the original EfficientNet network consists of 9 stages of operations. The present invention removes the operations of the convolutional layer and the fully connected layer in the 9th stage, and compresses the model output feature map to a size of 7×7 with 320 channels and a size of 7×7.

[0040] Table 1 EfficientNet network structure

[0041]

[0042] Step 3.2: Task-specific modules. Task-specific modules are Figure 4 (a) The residual module is built by integrating the coordinated attention mechanism and the grouped convolution operation, which can enhance the feature expression while reducing the model parameters. Specifically, Figure 4 As shown in (b), the coordinated attention mechanism is added after the second convolution layer of the residual structure, and the grouped convolution operation is added after the third convolution operation of the residual structure.

[0043] Step 4: Use the training data set divided in step 2 to train the model and the test set to verify the model.

[0044] Step 4.1: Use batch processing to divide the training data and test data into multiple batches. The batch size of both training data and test data is set to 32, that is, 32 images are trained in each batch. Traversing all the images in the training set once is called an iteration.

[0045] Step 4.2: Transfer learning technology is used during training. The model converges very quickly, so the number of model iterations is set to 50. The stochastic gradient descent optimization algorithm is used during model training. After each epoch, the model will verify the validation set, record the accuracy of the model verification and output it. The initial learning rate is 0.005. After each training of 10 epochs, the learning rate is reduced to 0.1 times the current learning rate. Figure 5 The changes in recognition accuracy and cross entropy of the three datasets on the training and validation sets during the training process are shown.

[0046] Step 4.3: The deep multi-task blueberry rot disease recognition model can identify the infected parts of blueberry rot, divide the infection stages, and estimate the severity of infection at one time. Therefore, the model will have three output results in one verification. The model with the largest sum of the verification set recognition accuracy among the three data sets is selected as the best recognition model, and the network weight of the optimal model is saved.

[0047] Step 5: Use the model trained in step 4 to identify blueberry rot disease.

[0048] The hardware devices used in the method of the present invention include: a computer configured with a Windows operating system, an NVIDIA GeForce GTX 3080 (8G) graphics card that supports model acceleration training, and an open source deep learning model training framework Pytorch, with the programming language being Python.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for identifying highbush blueberry rot disease based on deep multi-task learning, characterized in that: The method comprises the following steps: S1: Collect highbush blueberry rot disease image dataset, annotate the image dataset to generate infection site identification dataset, infection stage classification dataset and infection severity estimation dataset; S2: Perform data augmentation on the annotated image dataset; S3: A deep multi-task blueberry necrosis disease recognition model was built based on the EfficientNet model. The disease recognition model includes a parameter sharing module and three specific task modules. The improved EfficientNet model was selected as the parameter sharing module. The original EfficientNet model consists of 9 stages of operations. The last stage of operations was removed to build a parameter sharing module. The specific task module was built on the basis of the residual module by integrating the coordinated attention mechanism and the grouped convolution operation. Specifically, the coordinated attention mechanism was added after the second convolution layer of the residual structure, and the grouped convolution operation was added after the third convolution operation of the residual structure. S4: using the image dataset amplified in step S2 to train the deep multi-task blueberry rot disease recognition model constructed in step S3; S5: Use the trained model to identify blueberry berry blight infection sites, classify infection stages, and estimate infection severity.

2. The method for identifying highbush blueberry rot disease based on deep multi-task learning according to claim 1, characterized in that: In step S1, the collected highbush blueberry rot disease images include image data of blueberry flowers, leaves and fruits; The data annotation principles are as follows: 1) Infection site identification dataset: determine whether the main infected tissue in the image data is flower, leaf or fruit, and perform data annotation; 2) Infection stage classification dataset: according to the pathological characteristics of blueberry rot disease, the data is labeled as healthy stage, primary infection stage or secondary infection stage; 3) Infection severity classification dataset: estimate the area of ​​the infected area in the image and perform data annotation.

3. The method for identifying highbush blueberry rot disease based on deep multi-task learning according to claim 1, characterized in that: In step S2, the image data set is augmented, specifically by using an image operation data enhancement method based on geometric transformation, color transformation, and pixel transformation, and using random probability and appropriate combination strategies to perform offline data enhancement on the data set to enhance the richness of the generated image.

4. The method for identifying highbush blueberry rot disease based on deep multi-task learning according to claim 1, characterized in that: In step S4, the model is trained using transfer learning technology, including setting the maximum number of iterations, selecting the stochastic gradient descent optimization algorithm, and setting the learning rate; the data of the three tasks pass through the parameter sharing module in turn to obtain three feature vectors, and then the three feature vectors pass through their respective specific task module branches to complete the identification of the infection site of the blueberry rot disease, the division of the infection stage, and the estimation of the severity of the infection.

Images