Plant disease identification method, device and system based on deep learning

Abstract

This application provides a plant disease identification method, device, and system based on deep learning. The method includes: obtaining a target image to be identified, where the target image is a hyperspectral image obtained by photographing a plant; inputting the target image into a trained plant disease identification model to obtain a recognition result for the target image, wherein the recognition result includes a result characterizing the plant species and whether the plant has the target disease. The plant disease identification model includes a ResNet18 network model as a backbone network, as well as a first DeepLabV3+ network model and a second DeepLabV3+ network model. The first DeepLabV3+ network model is used to identify the species in the target image, and the second DeepLabV3+ network model is used to identify whether the target disease is present in the target image. This helps improve the accuracy of plant disease identification.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a plant disease identification method, device and system based on deep learning. Background Art

[0002] In the agricultural sector, outbreaks of economic tree or crop diseases can easily cause significant economic losses. Traditional plant disease detection relies primarily on manual surveys to detect the occurrence of economic tree or crop diseases. While accurate, this method is time-consuming, labor-intensive, inefficient, and requires a high level of professional experience from the inspectors. Furthermore, the large number of sampling points required makes it unsuitable for large-scale application. In recent years, the use of remote sensing technology combined with deep learning for automated crop disease monitoring has become a trend. However, currently, plant disease identification typically relies on a single convolutional neural network, which still suffers from the potential for misidentification and requires further improvement in accuracy. Summary of the Invention

[0003] In view of this, the purpose of the embodiments of the present application is to provide a plant disease identification method, device and system based on deep learning, which can improve the problem of low plant disease identification accuracy due to the single structure of the network model.

[0004] To achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0005] In a first aspect, an embodiment of the present application provides a plant disease identification method based on deep learning, the method comprising:

[0006] Acquire a target image to be identified, where the target image is a hyperspectral image obtained by photographing plants;

[0007] The target image is input into a trained plant disease recognition model to obtain a recognition result of the plant disease recognition model for the target image, wherein the recognition result includes a result characterizing the type of plant and whether the plant has the target disease. The plant disease recognition model includes a ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model. The first DeepLabV3+ network model is used to identify the type in the target image, and the second DeepLabV3+ network model is used to identify whether the target disease exists in the target image.

[0008] In conjunction with the first aspect, in some optional embodiments, inputting the target image into a trained plant disease recognition model to obtain a recognition result of the plant disease recognition model for the target image includes:

[0009] Inputting the target image into the ResNet18 network model in the plant disease recognition model for feature extraction, and inputting the extracted data into the first DeepLabV3+ network model and the second DeepLabV3+ network model for recognition;

[0010] When the second DeepLabV3+ network model identifies the area in the target image as suffering from the target disease, and there is an overlapping area with the area in the target image identified by the first DeepLabV3+ network model as the target plant, the overlapping area is confirmed as the diseased area where the target plant suffers from the target disease and is used as the identification result.

[0011] In conjunction with the first aspect, in some optional implementations, the method further includes:

[0012] Obtaining real-time location data of a first drone loaded with pesticide for treating a target disease;

[0013] When the real-time location data of the first UAV is within the area corresponding to the disease map area, controlling the first UAV to spray pesticides;

[0014] When the real-time location data of the first UAV is not within the area corresponding to the disease map area, the first UAV is controlled to stop spraying pesticides.

[0015] In conjunction with the first aspect, in some optional implementations, obtaining a target image to be identified includes:

[0016] photographing plants in a designated area by a second unmanned aerial vehicle during flight, thereby obtaining a plurality of hyperspectral images arranged in a time sequence, wherein any two hyperspectral images that are adjacent in time sequence among the plurality of hyperspectral images have overlapping image areas;

[0017] The multiple hyperspectral images are stitched into one hyperspectral image using a preset image stitching algorithm to serve as the target image.

[0018] In conjunction with the first aspect, in some optional implementations, before acquiring the target image to be identified, the method further includes:

[0019] Based on the ResNet18 network model and the DeepLabV3+ network model, creating a hybrid model based on deep learning, wherein the hybrid model includes the ResNet18 network model as the backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model;

[0020] The ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model are trained using a training data set, wherein the training data set includes multiple hyperspectral images of target plants, each hyperspectral image is annotated with a first image area representing the target plant and a second image area representing the non-target plant, and each hyperspectral image is also annotated with a first mark representing the presence of a target disease in the target plant, or a second mark representing the absence of the target disease; the first image area and the second image area in the hyperspectral image are used to train the ResNet18 network model and the first DeepLabV3+ network model, and the first mark or the second mark in the hyperspectral image is used to train the second DeepLabV3+ network model;

[0021] The ResNet18 network model, the first DeepLabV3+ network model and the second DeepLabV3+ network model in the trained hybrid model are tested using a test data set corresponding to the training data set, and when the hybrid model meets the preset convergence condition, the tested hybrid model is determined as a plant disease recognition model, which is used to detect whether the target plant in the hyperspectral image has a target disease.

[0022] In combination with the first aspect, in some optional embodiments, before training the ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model using a training data set, the method further includes:

[0023] Acquire an initial dataset, the initial dataset comprising a plurality of hyperspectral images of a target plant;

[0024] By using a preset data enhancement algorithm, some or all of the multiple hyperspectral images in the initial data set are horizontally rotated, and / or mirror-flipped, and / or the reflectances of all channels in the hyperspectral images are adjusted to specified values to obtain multiple hyperspectral images after data enhancement;

[0025] All hyperspectral images in the initial data set and all hyperspectral images after data enhancement are used as sample data sets;

[0026] Each hyperspectral image in the sample data set is annotated with a first image region representing a target plant and a second image region representing a non-target plant, and each hyperspectral image is annotated with a first marker representing the presence of a target disease in the target plant or a second marker representing the absence of the target disease, so as to obtain the training data set.

[0027] In combination with the first aspect, in some optional embodiments, the ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model are trained using a training data set, including:

[0028] Using the hyperspectral image having the first image region and the second image region in the training data set to train the ResNet18 network model and the first DeepLabV3+ network model, so that the trained first DeepLabV3+ network model has the function of detecting whether the target plant exists in the hyperspectral image;

[0029] The ResNet18 network model and the second DeepLabV3+ network model are trained using the hyperspectral images with the first label and the second label in the training data set, so that the trained second DeepLabV3+ network model has the function of detecting whether the target disease exists in the hyperspectral image.

[0030] In combination with the first aspect, in some optional embodiments, the plant includes a Zanthoxylum bungeanum tree, and the target disease includes Zanthoxylum bungeanum rust.

[0031] In a second aspect, an embodiment of the present application further provides a plant disease identification device based on deep learning, the device comprising:

[0032] an acquisition unit, configured to acquire a target image to be identified, wherein the target image is a hyperspectral image obtained by photographing plants;

[0033] An identification unit is used to input the target image into a trained plant disease identification model to obtain a recognition result of the plant disease identification model for the target image, wherein the recognition result includes a result characterizing the type of plant and whether the plant has a target disease. The plant disease identification model includes a ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model. The first DeepLabV3+ network model is used to identify the type in the target image, and the second DeepLabV3+ network model is used to identify whether the target disease exists in the target image.

[0034] In a third aspect, an embodiment of the present application further provides a plant disease identification system, which includes a processor and a memory coupled to each other, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the plant disease identification system performs the above-mentioned method.

[0035] The invention adopting the above technical solution has the following advantages:

[0036] In the technical solution provided in the present application, the target image is input into a trained plant disease recognition model, and the target image is recognized and processed by the plant disease recognition model. Among them, the ResNet18 network model in the plant disease recognition model serves as the backbone network for extracting image features of the target image. The first DeepLabV3+ network model and the second DeepLabV3+ network model serve as two branch networks, which are conducive to introducing multi-scale information to improve the accuracy of recognition. That is, the first DeepLabV3+ network model can be used to identify plant species in hyperspectral images, and the second DeepLabV3+ network model can identify whether the target plant has the target disease. After the network models in the hybrid model are combined, it is conducive to improving the accuracy of plant disease recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present application may be further illustrated by the non-limiting embodiments provided in the accompanying drawings. It should be understood that the following drawings illustrate only certain embodiments of the present application and are therefore not to be construed as limiting the scope of the present application. It is understood that a person skilled in the art can derive other relevant drawings from these drawings without inventive effort.

[0038] Figure 1 A schematic flow chart of a plant disease identification method based on deep learning provided in an embodiment of the present application.

[0039] Figure 2 Schematic diagram of the hyperspectral image before and after recognition provided in an embodiment of the present application.

[0040] Figure 3 A block diagram of the identification device provided in an embodiment of the present application.

[0041] Icon: 200 - identification device; 210 - first acquisition unit; 220 - identification unit. DETAILED DESCRIPTION

[0042] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts in the drawings or descriptions are numbered the same. Implementations not shown or described in the drawings are known to those of ordinary skill in the art. In the description of this application, the terms "first," "second," etc. are used solely to distinguish descriptions and are not to be construed as indicating or implying relative importance.

[0043] Embodiments of the present application provide a plant disease identification system. The system may include a processing module and a storage module. The storage module stores a computer program that, when executed by the processing module, enables the plant disease identification system to perform the corresponding steps of the following deep learning-based plant disease identification method.

[0044] In this embodiment, the plant disease identification system may be a hardware system, for example, a hardware system of a personal computer, a server, or the like.

[0045] The plant disease recognition system can identify hyperspectral plant images to detect the presence of target plants and whether the target plants are infected with target diseases. Target plants can include, but are not limited to, pepper trees, wheat, and rice, and target diseases can include, but are not limited to, pepper rust, wheat rust, and rice rust. For example, the target plant is a pepper tree, and the target disease is pepper rust.

[0046] Please refer to Figure 1 This application also provides a plant disease identification method based on deep learning, which can be applied to the above-mentioned plant disease identification system, and the plant disease identification system performs or implements each step of the method. Among them, the plant disease identification method can include the following steps:

[0047] Step 110: Acquire a target image to be identified, wherein the target image is a hyperspectral image obtained by photographing plants;

[0048] Step 120: input the target image into a trained plant disease recognition model to obtain a recognition result of the plant disease recognition model for the target image, wherein the recognition result includes a result characterizing the type of plant and whether the plant has a target disease. The plant disease recognition model includes a ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model. The first DeepLabV3+ network model is used to identify the type in the target image, and the second DeepLabV3+ network model is used to identify whether the target disease exists in the target image.

[0049] The following is a detailed description of the various steps of the deep learning-based plant disease identification method:

[0050] In step 110, the method for acquiring the target image can be flexibly determined based on the actual situation. For example, the target image can be a single hyperspectral image captured in advance. Alternatively, the target image can be a hyperspectral image formed by piecing together multiple hyperspectral images.

[0051] In this embodiment, step 110 may include:

[0052] photographing plants in a designated area by a second unmanned aerial vehicle during flight, thereby obtaining a plurality of hyperspectral images arranged in a time sequence, wherein any two hyperspectral images that are adjacent in time sequence among the plurality of hyperspectral images have overlapping image areas;

[0053] The multiple hyperspectral images are stitched into one hyperspectral image using a preset image stitching algorithm to serve as the target image.

[0054] The second drone is a drone for acquiring target images, and the first drone described below is a drone for spraying pesticides. The first drone and the second drone may be different drones.

[0055] Exemplarily, the target image can be acquired by using a second drone equipped with an airborne frame-type hyperspectral imager (or hyperspectral camera) to collect data, and then stitching the collected images. After the hyperspectral imager is properly connected to the drone, it can be debugged on the ground. For example, first perform whiteboard radiation correction. Place the whiteboard under the hyperspectral camera (under direct sunlight or the same conditions as the shooting conditions), open the lens cap, click the Cuber software interface preview, click white, and wait for 15 seconds. Then, install the lens cap, click black, perform dark radiation correction, and wait for the white and dark display values in the box to be calibrated. During the drone flight, set the lateral overlap rate to 80%, the heading overlap rate to 75%, the flight altitude to 80m, and the flight speed to 3m / s. In this way, multiple hyperspectral images arranged in chronological order can be obtained, and any two hyperspectral images that are adjacent in time sequence among the multiple hyperspectral images have overlapping image areas.

[0056] During the hyperspectral imagery capture process, you can delete photos of takeoff, landing, and turns to reduce unnecessary data processing. Then, convert the raw data to BIL (Band Interleaved by Line) format. Add band information to the header file of the converted BIL data to complete the data preprocessing step. BIL is a data format used in remote sensing images. The BIL format conversion method is conventional and will not be detailed here.

[0057] In this embodiment, the preset image stitching algorithm can be flexibly selected according to actual conditions. The preset image stitching algorithm can calculate the overlapping areas of the temporally adjacent images in the hyperspectral image, and then merge the overlapping areas. In this way, multiple hyperspectral images can be stitched into one hyperspectral image, such as Figure 2 As shown in (a).

[0058] Figure 2 The image shown in (a) is an image obtained by stitching multiple hyperspectral images and can be used as a target image. Figure 2 The image shown in (a) was obtained by taking a drone photo of the pepper tree planting area.

[0059] In step 120, after acquiring the target image, the plant disease recognition system can input the target image into the plant disease recognition model that has been trained and tested. The first DeepLabV3+ network model in the plant disease recognition model can detect whether the target plant exists in the target image. For example, the pepper tree (target plant) and the background image are classified. The second DeepLabV3+ network model can classify the healthy plants and those suffering from pepper rust (target disease) in the pepper tree. The plant disease recognition model can further analyze the preliminary recognition results of the target image based on the first DeepLabV3+ network model and the second DeepLabV3+ network model to obtain the final recognition result. The recognition result can be as follows Figure 2 As shown in (b), Figure 2 (b) is a schematic diagram of retaining only the area of the Zanthoxylum bungeanum tree in the target image, and the light color represents the healthy Zanthoxylum bungeanum tree, and the dark color represents the Zanthoxylum bungeanum tree suffering from Zanthoxylum bungeanum rust.

[0060] Of course, in other embodiments, healthy and diseased Zanthoxylum bungeanum trees can be rendered and displayed differently using different colors. For example, healthy Zanthoxylum bungeanum trees can be rendered green, and diseased Zanthoxylum bungeanum trees can be rendered red. The plant disease recognition model can calculate the severity of the disease based on the area ratio of diseased Zanthoxylum bungeanum trees to all Zanthoxylum bungeanum trees. The calculation method of the severity of the disease can be flexibly determined according to the actual situation and will not be repeated here.

[0061] In this embodiment, step 120 may include:

[0062] Inputting the target image into the ResNet18 network model in the plant disease recognition model for feature extraction, and inputting the extracted data into the first DeepLabV3+ network model and the second DeepLabV3+ network model for recognition;

[0063] When the second DeepLabV3+ network model identifies the area in the target image as suffering from the target disease, and there is an overlapping area with the area in the target image identified by the first DeepLabV3+ network model as the target plant, the overlapping area is confirmed as the diseased area where the target plant suffers from the target disease and is used as the identification result.

[0064] Understandably, when the second DeepLabV3+ network model identifies the target image as an area suffering from pepper rust (target disease) and the first DeepLabV3+ network model identifies the target image as a pepper tree (target plant) when there is a first overlapping area, the first overlapping area is confirmed as the diseased area of the pepper tree suffering from pepper rust. Similarly, when the second DeepLabV3+ network model identifies the target image as a healthy pepper tree when there is a second overlapping area with the first DeepLabV3+ network model identifies the target image as a pepper tree area, the second overlapping area is confirmed as the area of the healthy pepper tree.

[0065] It should be noted that, because the color characteristics of the exposed soil in the hyperspectral image are similar to the color characteristics of the Zanthoxylum bungeanum tree suffering from Zanthoxylum bungeanum rust, the image area of the exposed soil is easily misidentified as the area suffering from Zanthoxylum bungeanum rust by conventional deep learning models. And in the present embodiment, by combining the first DeepLabV3+ network model and the second DeepLabV3+ network model, it is conducive to improving the accuracy and reliability of Zanthoxylum bungeanum rust identification, and it is possible to avoid the situation of misidentification due to the single model structure. In addition, by utilizing unmanned aerial vehicle in combination with the above-mentioned plant disease identification method, it is possible to achieve early detection of Zanthoxylum bungeanum rust, promote the scientific management of Zanthoxylum bungeanum rust, improve yield and quality, and reduce the use of chemical fertilizers and pesticides. Based on the identified disease area, it is possible to generate an operation diagram for the precise spraying of the unmanned aerial vehicle, and the operation diagram can be loaded into the unmanned aerial vehicle system and then accurately sprayed.

[0066] In this embodiment, the first drone is equipped with a positioning chip that can collect its location data in real time. Furthermore, the first drone is equipped with a container for holding pesticides, an electrically controlled valve, and a spray nozzle. The spray nozzle is connected to the container via a pipe, and the electrically controlled valve is mounted on the pipe. The controller of the first drone is electrically connected to the electrically controlled valve. The controller can control the opening and closing of the electrically controlled valve to turn the pesticide spraying on and off. The spray nozzle can be mounted at the bottom of the container. When the electrically controlled valve is open, the pesticide can be sprayed using gravity.

[0067] Of course, in other embodiments, the first drone may also be provided with a small liquid pump motor electrically connected to the controller of the first drone. When the liquid pump motor is started and the electronically controlled valve is opened, the liquid pump motor can pump the pesticide in the container out and spray it through the nozzle.

[0068] As an optional implementation, the method may further include:

[0069] Obtaining real-time location data of a first drone loaded with pesticide for treating a target disease;

[0070] When the real-time location data of the first UAV is within the area corresponding to the disease map area, controlling the first UAV to spray pesticides;

[0071] When the real-time location data of the first UAV is not within the area corresponding to the disease map area, the first UAV is controlled to stop spraying pesticides.

[0072] Understandably, the pesticides loaded on the drone can be flexibly selected based on the type of target pest. The positioning chip on the first drone can collect the first drone's location information in real time. This real-time location data can be understood as the first drone's projected position on the ground, represented by a two-dimensional coordinate system. Furthermore, the first drone pre-stores a pre-detected pest map, which is a location area converted onto a map.

[0073] When the real-time location data of the first UAV is within the area corresponding to the disease map area, the controller on the first UAV can automatically control the electric control valve to open so that the pesticide in the container is sprayed out through the nozzle.

[0074] If the real-time location data of the first drone is not within the area corresponding to the disease map, the controller on the first drone can automatically control the electric control valve to close, so that the nozzle stops spraying pesticides. This allows drones to precisely spray the diseased area, which helps improve the efficiency and accuracy of spraying.

[0075] Before step 110, the method may further include the step of training and testing the plant disease recognition model. For example, before step 110, the method may further include:

[0076] Step 101: Create a hybrid model based on deep learning based on the ResNet18 network model and the DeepLabV3+ network model, wherein the hybrid model includes the ResNet18 network model as the backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model;

[0077] Step 102: Training the ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model using a training data set, wherein the training data set includes multiple hyperspectral images of target plants, each hyperspectral image is annotated with a first image region representing the target plant and a second image region representing the non-target plant, and each hyperspectral image is further annotated with a first mark representing the presence of a target disease in the target plant, or a second mark representing the absence of the target disease; the first image region and the second image region in the hyperspectral image are used to train the ResNet18 network model and the first DeepLabV3+ network model, and the first mark or the second mark in the hyperspectral image is used to train the second DeepLabV3+ network model;

[0078] Step 103, testing the ResNet18 network model, the first DeepLabV3+ network model and the second DeepLabV3+ network model in the trained hybrid model through a test data set corresponding to the training data set, and when the hybrid model meets the preset convergence condition, determining the tested hybrid model as a plant disease recognition model, and the plant disease recognition model is used to detect whether the target plant in the hyperspectral image has a target disease.

[0079] In step 101, the ResNet18 network model and the DeepLabV3+ network model are conventional deep learning network models. The first DeepLabV3+ network model and the second DeepLabV3+ network model have the same network structure, except that the first DeepLabV3+ network model is used to detect the presence of target plants in hyperspectral images, while the second DeepLabV3+ network model is used to detect the presence of target diseases in plants in hyperspectral images.

[0080] In this embodiment, a hybrid model can be constructed using a ResNet18 network model and two DeepLabV3+ network models. In the hybrid model, the input data of the ResNet18 network model can be a hyperspectral image in the training dataset. The output of the ResNet18 network model is connected to the input of the first DeepLabV3+ network model and the second DeepLabV3+ network model.

[0081] The ResNet18 network model is composed of residual blocks, which add skip mapping to directly add the input and output, which helps to replenish the feature information lost by the ResNet18 network model during the convolution process. The ResNet18 network model is used to extract features from hyperspectral images, which can extract richer feature information. During model training, the extracted feature information is used as input data for the first DeepLabV3+ network model and the second DeepLabV3+ network model, so that these two DeepLabV3+ network models can be learned and trained separately.

[0082] The DeepLabv3+ network model consists of an encoder and decoder. The decoder network is primarily based on a DCNN with dilated convolutions, and can employ commonly used classification networks such as ResNet. The decoder network uses the Atrous Spatial Pyramid Pooling (ASPP) module with dilated convolutions to incorporate multi-scale information. Compared to standard convolutions, dilated convolutions increase the receptive field without increasing parameters, thereby improving the recognition accuracy of the trained model.

[0083] In step 102, the number of hyperspectral images included in the training dataset can be flexibly determined based on actual conditions. Generally speaking, the greater the number of hyperspectral images in the training dataset, the more conducive it is to improving the recognition accuracy of the trained model. For example, the number of hyperspectral images can exceed 1,000.

[0084] In this embodiment, step 102 may include:

[0085] Using the hyperspectral image having the first image region and the second image region in the training data set to train the ResNet18 network model and the first DeepLabV3+ network model, so that the trained first DeepLabV3+ network model has the function of detecting whether the target plant exists in the hyperspectral image;

[0086] The ResNet18 network model and the second DeepLabV3+ network model are trained using the hyperspectral images with the first label and the second label in the training data set, so that the trained second DeepLabV3+ network model has the function of detecting whether the target disease exists in the hyperspectral image.

[0087] In this embodiment, engineers can input the hyperspectral image with the first image area and the second image area into the ResNet18 network model and the first DeepLabV3+ network model to train and learn the ResNet18 network model and the first DeepLabV3+ network model. The trained ResNet18 network model can extract feature information from the hyperspectral image and input it into the trained first DeepLabV3+ network model, so that the trained first DeepLabV3+ network model can initially identify whether the target plant exists in the hyperspectral image.

[0088] Similarly, engineers can input the hyperspectral image with the first label and the second label into the ResNet18 network model and the second DeepLabV3+ network model to train and learn the ResNet18 network model and the second DeepLabV3+ network model. The trained ResNet18 network model can extract feature information from the hyperspectral image and input it into the trained second DeepLabV3+ network model, so that the trained second DeepLabV3+ network model can initially identify whether the target disease exists in the hyperspectral image.

[0089] During training, the hybrid model can use a loss function that combines Dice loss and Cross entropy loss. Adam is used as the optimizer, with an initial learning rate of 0.005 and exponential decay, with each epoch decaying to 0.97 times the previous one. Training is done for 50 epochs. An epoch is a hyperparameter that defines the number of times the learning algorithm works on the entire training dataset.

[0090] During training, engineers can evaluate hybrid models based on metrics such as precision, recall, intersection-over-union (IoU), and F1-score (harmonic mean of precision and recall). The best model can be visually inspected by applying it to the entire image that was not used during model training. The hybrid model can be evaluated using the following formula:

[0091]

[0092]

[0093]

[0094]

[0095] In the above formulas (1)-(4), P refers to the recognition accuracy after the hybrid model training;

[0096] R refers to the recall rate Recall;

[0097] F1 refers to the F1 score;

[0098] I oU Refers to intersection and comparison;

[0099] T P refers to true positive, which means the correctly predicted positive value;

[0100] F P refers to false positive, which means the positive value predicted incorrectly;

[0101] F N Refers to false negative, which means a negative value that is incorrectly predicted;

[0102] A O Area of Overlap refers to the intersection of the actual value and the predicted value of plant diseases;

[0103] A U Area of Union refers to the union of the true value and predicted value of plant diseases.

[0104] It is understandable that engineers can evaluate the reliability of the trained hybrid model based on the above formulas (1)-(4) in a conventional way. In addition, the parameter T P 、F P 、F N The accuracy is based on pixel statistics.

[0105] In step 103, the hyperspectral images in the test dataset have a format similar to that of the hyperspectral images in the training dataset. Specifically, in the test dataset, the hyperspectral images are annotated with a first region representing target plants and a second region representing non-target plants. The hyperspectral images are also annotated with a first marker indicating the presence of a target disease on the target plant, or a second marker indicating the absence of the target disease.

[0106] In this embodiment, step 103 may include:

[0107] Inputting each hyperspectral image in the test data set into the hybrid model to obtain a test result corresponding to each hyperspectral image output by the hybrid model;

[0108] When the test result is different from the first image area and the second image area of the corresponding hyperspectral image, or the first mark or the second mark of the corresponding hyperspectral image, the trained hybrid model is corrected using the first image area and the second image area of the hyperspectral image, or the first mark or the second mark of the hyperspectral image until the hybrid model meets the preset convergence condition.

[0109] Understandably, during the testing process, engineers can input hyperspectral images from the test dataset into the hybrid model. The ResNet18 network model extracts feature information from the hyperspectral images. The first and second DeepLabV3+ network models then perform dilated convolution on the extracted feature information and output test results. The test results can include markers indicating the presence and absence of target plants in the hyperspectral images, as well as the presence of target diseases on the target plants. The test results are then compared with the pre-set first and second regions, and the first or second markers, of the hyperspectral images. If any discrepancies are found, the hybrid model is calibrated based on the pre-set first and second regions, and the first or second markers. If no discrepancies are found, the hybrid model is tested on additional hyperspectral images until all hyperspectral images in the test dataset are tested, or until the hybrid model meets the preset convergence criteria. The preset convergence criteria can be flexibly determined based on actual conditions. For example, the preset convergence criteria could be that the hybrid model's test accuracy exceeds 90%, indicating convergence.

[0110] Understandably, using the test data set to test the trained hybrid model can improve the accuracy and reliability of the trained hybrid model in hyperspectral image recognition.

[0111] As an optional implementation, before training the ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model using a training data set, the method further includes:

[0112] Acquire an initial dataset, the initial dataset comprising a plurality of hyperspectral images of a target plant;

[0113] By using a preset data enhancement algorithm, some or all of the multiple hyperspectral images in the initial data set are horizontally rotated, and / or mirror-flipped, and / or the reflectances of all channels in the hyperspectral images are adjusted to specified values to obtain multiple hyperspectral images after data enhancement;

[0114] All hyperspectral images in the initial data set and all hyperspectral images after data enhancement are used as sample data sets;

[0115] Each hyperspectral image in the sample data set is annotated with a first image region representing a target plant and a second image region representing a non-target plant, and each hyperspectral image is annotated with a first marker representing the presence of a target disease in the target plant or a second marker representing the absence of the target disease, so as to obtain the training data set.

[0116] In this embodiment, the hyperspectral images in the training dataset or initial dataset are image data converted into the BIL format. That is, if the hyperspectral image data format is not in the BIL format, it is necessary to convert the hyperspectral image data format into the BIL format. The target image is also an image converted into the BIL format.

[0117] The preset data enhancement algorithm can be flexibly determined according to actual conditions, and can have one or more of the following functions: horizontally rotating the hyperspectral image, mirror flipping, and adjusting the reflectance of all channels in the hyperspectral image to a specified value.

[0118] The horizontal rotation angle can be flexibly set based on the actual situation, for example, ±30°. The specified reflectance value can also be flexibly determined based on the actual situation. For example, for all initial hyperspectral images, the reflectance of all channels can be adjusted to any non-1 multiple in the range [0.5, 1.5] with a probability of 0.5. After data augmentation, multiple new hyperspectral images can be obtained. All hyperspectral images in the initial dataset and all new hyperspectral images serve as the sample dataset.

[0119] After obtaining a sample dataset through data augmentation, engineers can use the vector image drawing function of ArcGIS tools to annotate the hyperspectral images in the sample dataset. Specifically, each hyperspectral image in the sample dataset is annotated with a first region representing the target plant and a second region representing the non-target plant. Furthermore, each hyperspectral image is annotated with a first marker indicating the presence of the target disease on the target plant or a second marker indicating the absence of the target disease.

[0120] Furthermore, to enhance the refinement of model recognition, the first marker includes multiple level identifiers representing the severity of the target disease. For example, the first marker may include three level identifiers: a first level identifier representing severe diseases, a second level identifier representing common diseases, and a third level identifier representing mild diseases. Hyperspectral images with different level identifiers can be used to train the second DeepLabV3+ network model to identify the severity of the target disease.

[0121] Please refer to Figure 3This application also provides a deep learning-based plant disease identification device, referred to as identification device 200. Identification device 200 includes at least one software functional module that can be stored in a storage module in the form of software or firmware or embedded in an operating system (OS). A processing module is configured to execute executable modules stored in the storage module, such as the software functional modules and computer programs included in identification device 200.

[0122] The recognition device 200 may include a first acquisition unit 210 and a recognition unit 220. The functions of each unit may be as follows:

[0123] A first acquisition unit 210 is configured to acquire a target image to be identified, wherein the target image is a hyperspectral image obtained by photographing plants;

[0124] The recognition unit 220 is used to input the target image into a trained plant disease recognition model to obtain a recognition result of the plant disease recognition model for the target image, wherein the recognition result includes a result characterizing the type of plant and whether the plant has a target disease. The plant disease recognition model includes a ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model. The first DeepLabV3+ network model is used to identify the type in the target image, and the second DeepLabV3+ network model is used to identify whether the target disease exists in the target image.

[0125] Optionally, the first acquiring unit 210 may be configured to:

[0126] photographing plants in a designated area by a second unmanned aerial vehicle during flight, thereby obtaining a plurality of hyperspectral images arranged in a time sequence, wherein any two hyperspectral images that are adjacent in time sequence among the plurality of hyperspectral images have overlapping image areas;

[0127] The multiple hyperspectral images are stitched into one hyperspectral image using a preset image stitching algorithm to serve as the target image.

[0128] Optionally, the identification device 200 may further include a positioning unit and a control unit. The positioning unit may be configured to obtain real-time location data of a first drone carrying pesticide for use in treating a target disease. The control unit may be configured to control the first drone to spray the pesticide when the real-time location data of the first drone is within the area corresponding to the disease map; and to control the first drone to stop spraying the pesticide when the real-time location data of the first drone is not within the area corresponding to the disease map.

[0129] Optionally, the identification unit 220 may be configured to:

[0130] Inputting the target image into the ResNet18 network model in the plant disease recognition model for feature extraction, and inputting the extracted data into the first DeepLabV3+ network model and the second DeepLabV3+ network model for recognition;

[0131] When the second DeepLabV3+ network model identifies the area in the target image as suffering from the target disease, and there is an overlapping area with the area in the target image identified by the first DeepLabV3+ network model as the target plant, the overlapping area is confirmed as the diseased area where the target plant suffers from the target disease and is used as the identification result.

[0132] Optionally, the recognition device 200 may further include a creation unit, a training unit, and a testing unit.

[0133] A creation unit, configured to create a hybrid model based on deep learning based on a ResNet18 network model and a DeepLabV3+ network model, wherein the hybrid model includes the ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model;

[0134] A training unit, configured to train the ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model using a training data set, wherein the training data set includes multiple hyperspectral images of target plants, each hyperspectral image is annotated with a first image region representing the target plant and a second image region representing the non-target plant, and each hyperspectral image is further annotated with a first marker representing the presence of a target disease in the target plant, or a second marker representing the absence of the target disease; the first image region and the second image region in the hyperspectral image are used to train the ResNet18 network model and the first DeepLabV3+ network model, and the first marker or the second marker in the hyperspectral image is used to train the second DeepLabV3+ network model;

[0135] A testing unit is used to test the ResNet18 network model, the first DeepLabV3+ network model and the second DeepLabV3+ network model in the trained hybrid model using a test data set corresponding to the training data set, and when the hybrid model meets a preset convergence condition, the tested hybrid model is determined as a plant disease recognition model, and the plant disease recognition model is used to detect whether a target plant in a hyperspectral image has a target disease.

[0136] Optionally, the identification device 200 may further include:

[0137] a second acquisition unit, configured to acquire an initial data set, wherein the initial data set includes a plurality of hyperspectral images of the target plant;

[0138] a data enhancement unit, configured to horizontally rotate and / or mirror-flip some or all of the multiple hyperspectral images in the initial data set and / or adjust the reflectance of all channels in the hyperspectral images to specified values using a preset data enhancement algorithm, so as to obtain multiple hyperspectral images after data enhancement;

[0139] a combining unit, configured to use all hyperspectral images in the initial data set and all hyperspectral images after data enhancement as a sample data set;

[0140] A labeling unit is used to label each hyperspectral image in the sample data set with a first image region representing a target plant and a second image region representing a non-target plant, and to label each hyperspectral image with a first label representing the presence of a target disease in the target plant or a second label representing the absence of the target disease, so as to obtain the training data set.

[0141] Optionally, the training unit can be used to:

[0142] Using the hyperspectral image having the first image region and the second image region in the training data set to train the ResNet18 network model and the first DeepLabV3+ network model, so that the trained first DeepLabV3+ network model has the function of detecting whether the target plant exists in the hyperspectral image;

[0143] The ResNet18 network model and the second DeepLabV3+ network model are trained using the hyperspectral images with the first label and the second label in the training data set, so that the trained second DeepLabV3+ network model has the function of detecting whether the target disease exists in the hyperspectral image.

[0144] Optionally, the test unit can be used to:

[0145] Inputting each hyperspectral image in the test data set into the hybrid model to obtain a test result corresponding to each hyperspectral image output by the hybrid model;

[0146] When the test result is different from the first image area and the second image area of the corresponding hyperspectral image, or the first mark or the second mark of the corresponding hyperspectral image, the trained hybrid model is corrected using the first image area and the second image area of the hyperspectral image, or the first mark or the second mark of the hyperspectral image until the hybrid model meets the preset convergence condition.

[0147] In this embodiment, the processing module can be an integrated circuit chip with signal processing capabilities. The above-mentioned processing module can be a general-purpose processor. For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0148] The storage module can be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. In this embodiment, the storage module can be used to store hyperspectral images, recognition results, preset data augmentation algorithms, ResNet18 network models, and DeepLabV3+ network models. Of course, the storage module can also be used to store programs, which the processing module executes after receiving execution instructions.

[0149] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the plant disease identification system described above can refer to the corresponding processes of each step in the aforementioned method, and will not be elaborated here.

[0150] The present application also provides a computer-readable storage medium that stores a computer program that, when executed on a computer, causes the computer to execute the plant disease identification method based on deep learning as described in the above embodiments.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a plant disease identification system, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0152] In summary, the embodiments of the present application provide a plant disease identification method, device and system based on deep learning. In this solution, the target image is input into a trained plant disease recognition model, and the target image is identified and processed by the plant disease recognition model. Among them, the ResNet18 network model in the plant disease recognition model serves as the backbone network for extracting image features of the target image. The first DeepLabV3+ network model and the second DeepLabV3+ network model serve as two branch networks, which are conducive to introducing multi-scale information to improve the accuracy of recognition. That is, the first DeepLabV3+ network model can be used to identify plant species in hyperspectral images, and the second DeepLabV3+ network model can identify whether the target plant has the target disease. The combination of the network models in the hybrid model is conducive to improving the accuracy of plant disease recognition.

[0153] In the embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can also be implemented in other ways. The device, system and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and a part of the module, program segment or code includes one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0154] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A plant disease identification method based on deep learning, characterized in that: The method comprises: Acquire a target image to be identified, where the target image is a hyperspectral image obtained by photographing plants; Inputting the target image into a trained plant disease recognition model to obtain a recognition result of the plant disease recognition model for the target image, wherein the recognition result includes a result characterizing the type of plant and whether the plant has a target disease, and the plant disease recognition model includes a ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model, wherein the first DeepLabV3+ network model is used to identify the type in the target image, and the second DeepLabV3+ network model is used to identify whether the target disease exists in the target image; The step of inputting the target image into a trained plant disease recognition model to obtain a recognition result of the plant disease recognition model for the target image includes: Inputting the target image into the ResNet18 network model in the plant disease recognition model for feature extraction, and inputting the extracted data into the first DeepLabV3+ network model and the second DeepLabV3+ network model for recognition; When the second DeepLabV3+ network model identifies the area in the target image as suffering from the target disease, and there is an overlapping area with the area in the target image identified by the first DeepLabV3+ network model as the target plant, the overlapping area is confirmed as a diseased area in which the target plant suffers from the target disease, and as the identification result, the plant includes the Zanthoxylum bungeanum, and the target disease includes Zanthoxylum bungeanum rust.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining real-time location data of a first drone loaded with pesticide for treating a target disease; When the real-time location data of the first UAV is within the area corresponding to the disease map area, controlling the first UAV to spray pesticides; When the real-time location data of the first UAV is not within the area corresponding to the disease map area, the first UAV is controlled to stop spraying pesticides.

3. The method according to claim 1, characterized in that Obtain the target image to be identified, including: photographing plants in a designated area by a second unmanned aerial vehicle during flight, thereby obtaining a plurality of hyperspectral images arranged in a time sequence, wherein any two hyperspectral images that are adjacent in time sequence among the plurality of hyperspectral images have overlapping image areas; The multiple hyperspectral images are stitched into one hyperspectral image using a preset image stitching algorithm to serve as the target image.

4. The method according to claim 1, wherein Before acquiring the target image to be identified, the method further includes: Based on the ResNet18 network model and the DeepLabV3+ network model, creating a hybrid model based on deep learning, wherein the hybrid model includes the ResNet18 network model as the backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model; The ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model are trained using a training data set, wherein the training data set includes multiple hyperspectral images of target plants, each hyperspectral image is annotated with a first image area representing the target plant and a second image area representing the non-target plant, and each hyperspectral image is also annotated with a first mark representing the presence of a target disease in the target plant, or a second mark representing the absence of the target disease; the first image area and the second image area in the hyperspectral image are used to train the ResNet18 network model and the first DeepLabV3+ network model, and the first mark or the second mark in the hyperspectral image is used to train the second DeepLabV3+ network model; The ResNet18 network model, the first DeepLabV3+ network model and the second DeepLabV3+ network model in the trained hybrid model are tested using a test data set corresponding to the training data set, and when the hybrid model meets the preset convergence condition, the tested hybrid model is determined as a plant disease recognition model, which is used to detect whether the target plant in the hyperspectral image has a target disease.

5. The method according to claim 4, characterized in that Before training the ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model using a training data set, the method further includes: Acquire an initial dataset, the initial dataset comprising a plurality of hyperspectral images of a target plant; By using a preset data enhancement algorithm, some or all of the multiple hyperspectral images in the initial data set are horizontally rotated, and / or mirror-flipped, and / or the reflectances of all channels in the hyperspectral images are adjusted to specified values to obtain multiple hyperspectral images after data enhancement; All hyperspectral images in the initial data set and all hyperspectral images after data enhancement are used as sample data sets; Each hyperspectral image in the sample data set is annotated with a first image region representing a target plant and a second image region representing a non-target plant, and each hyperspectral image is annotated with a first marker representing the presence of a target disease in the target plant or a second marker representing the absence of the target disease, so as to obtain the training data set.

6. The method according to claim 4, characterized in that The ResNet18 network model, the first DeepLabV3+ network model, and the second DeepLabV3+ network model in the hybrid model are trained using a training data set, including: Using the hyperspectral image having the first image region and the second image region in the training data set to train the ResNet18 network model and the first DeepLabV3+ network model, so that the trained first DeepLabV3+ network model has the function of detecting whether the target plant exists in the hyperspectral image; The ResNet18 network model and the second DeepLabV3+ network model are trained using the hyperspectral images with the first label and the second label in the training data set, so that the trained second DeepLabV3+ network model has the function of detecting whether the target disease exists in the hyperspectral image.

7. A plant disease identification device based on deep learning, characterized in that: The device comprises: an acquisition unit, configured to acquire a target image to be identified, wherein the target image is a hyperspectral image obtained by photographing plants; an identification unit, configured to input the target image into a trained plant disease identification model to obtain a recognition result of the plant disease identification model for the target image, wherein the recognition result includes a result characterizing the type of plant and whether the plant has a target disease, the plant disease identification model including a ResNet18 network model as a backbone network, and a first DeepLabV3+ network model and a second DeepLabV3+ network model connected to the ResNet18 network model, the first DeepLabV3+ network model being used to identify the type in the target image, and the second DeepLabV3+ network model being used to identify whether the target disease exists in the target image; The recognition unit is specifically used to input the target image into the ResNet18 network model in the plant disease recognition model for feature extraction, and input the extracted data into the first DeepLabV3+ network model and the second DeepLabV3+ network model for recognition; when the second DeepLabV3+ network model identifies the area in the target image as suffering from the target disease, and there is an overlapping area with the area in the target image identified by the first DeepLabV3+ network model as the target plant, the overlapping area is confirmed as a disease area in which the target plant suffers from the target disease, and as the recognition result, the plant includes the Zanthoxylum bungeanum, and the target disease includes Zanthoxylum bungeanum rust.

8. A plant disease identification system, characterized in that: The plant disease identification system includes a processor and a memory coupled to each other, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the plant disease identification system executes the method according to any one of claims 1 to 6.

Images