Crop disease detection method based on unmanned aerial vehicle and application

Abstract

The invention relates to the technical field of image recognition, in particular to a crop disease detection method based on an unmanned aerial vehicle and application, and the method comprises the steps: obtaining a multispectral image of a crop, carrying out the preprocessing of image annotation, zooming and normalization on the multispectral image, obtaining a training set, carrying out the image annotation of the training set, and obtaining a target image; obtaining a test set containing the multispectral image and the corresponding scab marks; constructing a crop disease detection model according to the DDT model and the Xception structure; inputting the training set into a crop disease detection model to obtain a prediction result; evaluating the trained crop disease detection model according to a preset performance index, and checking prediction conditions of the crop disease detection model in different categories through the confusion matrix; adjusting and optimizing the crop disease detection model by using an Adam optimizer according to an evaluation result; and detecting crop diseases by using the optimized crop disease detection model. Compared with an existing method, the method has the advantages that the processing speed is higher, the resource requirement is lower, and the method has popularization and practical significance.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and specifically provides a crop disease detection method based on an unmanned aerial vehicle and an application thereof. Background Art

[0002] Crop disease marker identification is an important research direction in the field of modern agriculture and an important issue in agricultural production. If it is not discovered and controlled in time, it will lead to reduced crop yields, deterioration in quality, and even death, causing serious economic losses to farmers. At present, the main methods of crop disease marking are manual observation of disease symptoms and enzyme-linked immunosorbent assay, but these two methods are time-consuming and labor-intensive. They are not only highly dependent on experts and professional techniques, but also easily affected by subjective judgment, resulting in low detection efficiency and accuracy. In addition, manual observation or sampling in the field can easily become a medium for the spread of pathogens, directly leading to large-scale outbreaks of field diseases.

[0003] At present, people's identification and marking of crop diseases still mainly rely on sensory organs and experience memory as the basis for classification. However, since the outbreak of artificial intelligence technology, various researches have begun to try to apply artificial intelligence technology to the field of crop disease identification and marking. For example, when a crop is diseased, the corresponding diseased position of the crop leaves will have spots that are obviously different from other normal leaves. The spots will have a large difference in the reflection and absorption performance of some bands of light, especially electromagnetic waves. By using computer vision, image processing and deep learning technologies, the appearance characteristics of crops can be automatically analyzed and identified, so as to quickly and accurately determine whether the crops are affected by diseases. Artificial intelligence technology can also establish a classification model for crop diseases and improve the efficiency of disease identification by learning and analyzing a large amount of historical data. For example, when a crop is diseased, the corresponding diseased position of the crop leaves will have spots that are obviously different from other normal leaves. The spots will have a large difference in the reflection and absorption performance of some bands of light, especially electromagnetic waves.

[0004] In the existing technology, the application of artificial intelligence includes: Everton Castelão Tetila et al. used DJI Phantom 3 to collect data in 2017 and used the SVM classification model to help farmers detect soybean spot, powdery mildew and leafless disease, three diseases with obvious characteristics. Feng Ziheng et al. extracted wheat vegetation index, texture features and canopy temperature information with the help of multispectral technology and thermal imaging technology, and then used multiple linear regression (MLR), back propagation neural network (BP), random forest (RF), extreme learning machine (ELM) algorithm to combine vegetation index (VIs), texture features (TFs) and temperature features (T) to build a monitoring model for wheat powdery mildew disease index. Wang Jiantao et al. used normal citrus leaves, sooty leaves, ulcer leaves, red spider leaves, and glyphosate leaves as research samples, collected 128 bands of hyperspectral images of 400-1000nm and input them into the VGG16 convolutional neural network, which can achieve an accuracy of 98.75% under certain conditions.

[0005] Although the above methods have been proven to achieve remarkable results, there are still some shortcomings, such as low accuracy and low transferability. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a crop disease detection method based on drones, which has a faster processing speed and less resource requirements than existing methods, making it more practical for promotion. It includes the following steps: S1: Obtain multispectral images of crops through the multispectral camera carried by the drone; S2: performing preprocessing of scaling and normalization on the multispectral image to obtain a training set, and then annotating the training set to obtain a test set including the multispectral image and corresponding lesion annotations; S3: constructing a crop disease detection model according to the DDT model and the Xception structure, wherein the DDT model is used to extract a first feature vector from the preprocessed multispectral image, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification; S4: inputting the training set into the crop disease detection model to obtain a prediction result, and training the crop disease detection model with the goal of minimizing the difference between the prediction result and the test set to obtain a trained crop disease detection model; S5: Evaluate the trained crop disease detection model according to the preset performance indicators, and check the prediction of the crop disease detection model on different categories through the confusion matrix; S6: Use the Adam optimizer to tune the crop disease detection model based on the evaluation results to improve the convergence speed and accuracy of the crop disease detection model; S7: Use the optimized crop disease detection model to detect crop diseases.

[0007] Furthermore, the image annotation step includes manual annotation and / or automatic annotation.

[0008] Furthermore, the preprocessing also includes data enhancement, which increases the diversity of the image through rotation, scaling, and flipping operations.

[0009] Furthermore, the DDT model divides the preprocessed multispectral image into multiple small blocks, extracts and classifies features of each small block, and then integrates the features of all the small blocks to obtain the first feature vector of the multispectral image.

[0010] Furthermore, the Xception network extracts the second eigenvector from the preprocessed multispectral image through the global average pooling layer, fuses the first eigenvector and the second eigenvector through the Concatenate layer, and the fused eigenvector is input to the fully connected layer, processed by the ReLU activation layer of 1024 neurons, and finally classified through the output layer to obtain the prediction result.

[0011] Furthermore, step S4 also introduces a cross entropy loss function and a stochastic gradient descent optimizer for training.

[0012] Furthermore, the preset performance indicator includes at least one of accuracy, recall, F1 score and precision.

[0013] Furthermore, the present invention applies the above method to detect tobacco cucumber mosaic virus disease on tobacco leaves, and the multispectral camera includes a 650nm red light sensor.

[0014] Working principle and beneficial effects of the present invention: In the technical solution of the present invention, a multispectral camera carried by an unmanned aerial vehicle is used to collect multispectral images of crops, and the spectral reflectance of each plant canopy in the image in different bands is analyzed to achieve reasonable marking of the diseased parts. The use of this method in agricultural disease prevention and control can achieve rapid and accurate detection and marking of crop diseases, so as to help agricultural technicians predict the occurrence of diseases in advance and realize real-time grasp of the occurrence of field diseases.

[0015] Compared with existing methods, this method has a faster processing speed and requires fewer resources, making it more practical for promotion. At the same time, this method can detect the impact of field crops on diseases in real time, which is an important reference for the formulation of prevention and control plans for field diseases that may break out in tobacco fields or have already broken out, making it easier for managers to make scientific and targeted response measures, so that they can invest in pesticides more reasonably, reduce environmental pollution, and increase the quality and yield of agricultural products. At the same time, through deep learning, key information about crop diseases can be extracted, which can be provided to decision makers and scientific researchers for studying the spread of diseases, predicting disease risks, and formulating scientific agricultural development strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which: Figure 1 It is a schematic flow chart of the main steps of a crop disease detection method based on drone of the present invention. DETAILED DESCRIPTION

[0017] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0018] In the present invention, the crop disease detection model used mainly involves an improved classification and labeling method DDT-Xception, that is, the Deep Deor Transforming technology is used, combined with the convolutional neural network architecture Xception to improve accuracy and reduce running time. DDT technology is an image processing algorithm based on deep learning. It divides the image into multiple small blocks, then extracts and classifies features for each small block, and finally combines the features of all small blocks to obtain the feature representation of the entire image. Compared with traditional machine learning algorithms, DDT has better interpretability and generalization ability, has a good effect when processing images in complex scenes, can accurately detect the target objects in the image, and is very suitable for image processing in complex environments such as farmland. Xception is a lightweight CNN architecture, and its main feature is the use of depthwise separable convolution, which can decompose the convolution operation into two steps: 1. depth convolution; 2. point convolution. This decomposition method can reduce the amount of calculation and the amount of parameters, while improving the accuracy and efficiency of the model. In the present invention, the extracted features are further processed and classified through the Xception structure, and these features are converted into a higher-level abstract representation to adapt to image recognition tasks in complex environments. Example 1

[0019] Figure 1 The figure is a schematic flow chart of the main steps of a crop disease detection method based on drones of the present invention. Figure 1 As shown, a crop disease detection method based on a drone in this embodiment mainly includes the following steps S1 to S7.

[0020] S1: Obtain multispectral images of crops through the multispectral camera carried by the drone.

[0021] In one embodiment, a multispectral image of the crop is obtained by using a multispectral camera carried by a DJI Phantom4 Multispectral drone, using the characteristics that diseased crop leaves reflect light of corresponding wavelengths, while normal leaves strongly absorb light. In this embodiment, both the images used for the training set and the images used for normal detection are obtained by this method.

[0022] S2: Preprocess the multispectral image by scaling and normalizing it to obtain a training set to meet the input requirements of the model, and then annotate the training set to obtain a test set containing the multispectral image and the corresponding lesion annotations.

[0023] In one embodiment, preprocessing also includes data enhancement, which increases the diversity of images through rotation, scaling, and flipping operations to improve the generalization ability of the model.

[0024] In one embodiment, the image annotation step includes manual annotation and / or automatic annotation to ensure the accuracy and consistency of the annotation.

[0025] S3: A crop disease detection model is constructed based on the DDT model and the Xception structure. The DDT model is used to extract the first eigenvector from the preprocessed multispectral image, and the Xception structure is used to receive the first eigenvector output by the DDT model for further processing and classification.

[0026] In one embodiment, the DDT model divides the preprocessed multispectral image into multiple small blocks, extracts and classifies features of each small block, and then integrates the features of all the small blocks to obtain a first feature vector of the multispectral image.

[0027] In one embodiment, the Xception network extracts the second feature vector from the preprocessed multispectral image through the global average pooling layer, fuses the first feature vector and the second feature vector through the Concatenate layer, and inputs the fused feature vector into the fully connected layer, processes it through the ReLU activation layer of 1024 neurons, and finally classifies it through the output layer to obtain the prediction result. The output layer uses the softmax activation function for multi-classification tasks.

[0028] S4: Input the training set into the crop disease detection model to obtain the prediction result, and train the crop disease detection model with the goal of minimizing the difference between the prediction result and the test set to obtain a trained crop disease detection model.

[0029] Furthermore, the cross entropy loss function and stochastic gradient descent optimizer are introduced for training to prevent overfitting.

[0030] S5: Evaluate the trained crop disease detection model based on accuracy, recall, F1 score, and precision, and use the confusion matrix to view the predictions of the crop disease detection model on different categories.

[0031] S6: Based on the evaluation results, the Adam optimizer is used to tune the crop disease detection model to improve the convergence speed and accuracy of the crop disease detection model.

[0032] S7: Use the optimized crop disease detection model to detect crop diseases.

[0033] After model training and evaluation are completed, the crop disease detection model is deployed to actual application scenarios. Due to the portability of this model, for example, the model is used in an automatic detection system for crop diseases and deployed on low-level computing devices such as drones or ground stations. By analyzing the multispectral image data of crops, the disease spots on the leaves of crops can be automatically identified.

[0034] Based on the above steps S1 to S7, a multispectral camera carried by an unmanned aerial vehicle is used to collect multispectral images of crops, and the spectral reflectance of each plant canopy in the image in different bands is analyzed to achieve reasonable marking of diseased parts. The use of this method in agricultural disease prevention and control can achieve rapid and accurate detection and marking of crop diseases, so as to help agricultural technicians predict the occurrence of diseases in advance and realize real-time grasp of the occurrence of field diseases.

[0035] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention. Example 2

[0036] On the basis of the technical solution of Example 1, the crop selected in this embodiment is tobacco, and the disease selected is tobacco cucumber mosaic virus disease. According to the characteristics of large tobacco leaves and obvious disease spots, a detection method for tobacco cucumber mosaic virus disease is proposed. With the help of the 650nm red light sensor carried by the DJI Phantom4 Multispectral UAV, the characteristics that the diseased leaves of tobacco strongly reflect the 650nm wavelength red light, and the normal leaves strongly absorb it, the DDT model (DeepDeor Transforming) is used to mark the diseased spot area with obvious positive correlation on the leaves of the plants in the tobacco field, and it is used as the potential area of ​​the target object to obtain a test set, which is merged with the unlabeled image as a data set for training the crop disease recognition model. Then it is input into the Xception structure for further processing. In this way, DDT plus Xception can achieve high-quality marking effect and retain the structure and details of the image.

[0037] The specific steps include: S1: Obtain red light band images of tobacco fields through the multispectral camera carried by the drone; S2: Perform preprocessing of scaling and normalization on the red light band image to obtain a training set, and then annotate the training set to obtain a test set including the red light band image and the corresponding lesion annotations; S3: A crop disease detection model is constructed based on the DDT model and the Xception structure, wherein the DDT model is used to extract the disease spot features from the preprocessed image to generate a first feature vector, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification; S4: Input the training set into the crop disease detection model to obtain the prediction result, train the crop disease detection model with the goal of minimizing the difference between the prediction result and the test set, and obtain the trained crop disease detection model; train the model through the cross entropy loss function and the stochastic gradient descent optimizer to prevent overfitting; S5: Evaluate the trained crop disease detection model and verify the performance of the model on the test set based on the performance indicators of accuracy and recall; S6: Use the Adam optimizer to tune the crop disease detection model based on the evaluation results to improve the convergence speed and accuracy of the crop disease detection model; S7: Use the trained crop disease detection model to identify the red light band image to be analyzed, and obtain the recognition result of cucumber mosaic virus disease spots.

[0038] This embodiment realizes real-time detection of the impact of field crops on diseases, and serves as an important reference for the formulation of prevention and control plans for field diseases that may break out in tobacco fields or have already broken out, so that managers can make scientific and targeted response decisions, so that they can invest in pesticides more reasonably, reduce environmental pollution, and increase the quality and yield of agricultural products. At the same time, key information about crop diseases is extracted through deep learning. This information can be provided to decision makers and scientific researchers for studying the spread of diseases, predicting disease risks, and formulating scientific agricultural development strategies. However, while this method brings convenience, the method itself may also need to be further verified and improved in practical applications to ensure recognition accuracy and system stability. Example 3

[0039] Based on the technical solution of Example 1, this embodiment proposes a framework code for DDT-Xception integration.

[0040] 1. Load and preprocess the multispectral image through the load_data function. Specific operations include image scaling and cropping to ensure that the data is suitable for the input requirements of the model. The specific code is as follows: # Load the dataset and preprocess it def load_data(): # TODO: Load single-channel image data of the 650nm band and perform preprocessing, such as scaling, cropping, etc. # Here it is assumed that the loaded data is a single-channel image and the corresponding label images = np.random.rand(100, 256, 256, 1) # Sample data labels = np.random.randint(0, 2, size=(100, 1)) # Example labels return images, labels 2. Use the extract_ddt_features function to implement the DDT model to extract the first feature vector from the preprocessed multispectral image. Specifically, use the pre-trained deep convolutional model to extract the depth descriptor of each image, calculate the mean vector and covariance matrix, and obtain the DDT features of the image. These features will be fused with the features extracted by the Xception model in subsequent steps. The specific code is as follows: # Extract DDT features def extract_ddt_features(images): # TODO: Use the pre-trained deep convolutional model to extract the deep descriptor for each image and calculate the mean vector and covariance matrix # Example Implementation return np.random.rand(images.shape[0], 256) 3. In the build_ddt_xception_model function, the construction of the crop disease detection model is implemented. First, the input layer is defined to accept image inputs of shape (None, None, 1). None in the first dimension means that the input image can have any number of rows, None in the second dimension means that the input image can have any number of columns, and 1 in the third dimension means a single-channel 650nm band image, which means that the model can process single-channel images of different sizes (variable width and height). Then, the unpretrained Xception network is used to extract the features of the preprocessed multispectral image, and the second eigenvector is obtained through the global average pooling layer. Subsequently, the DDT features of the preprocessed multispectral image, that is, the first eigenvector, are extracted and fused with the second eigenvector, and feature fusion is achieved through the Concatenate layer. The fused feature vector is input to the fully connected layer, processed by the ReLU activation layer with 1024 neurons, and finally classified by the output layer, which uses the softmax activation function for multi-classification tasks. Finally, through these steps, a DDT-Xception neural network model is constructed for the classification task of crop diseases. The specific code is as follows.

[0041] # Build DDT-Xception structure def build_ddt_xception_model(input_shape, num_classes): # Define the input layer input_layer = tf.keras.Input(shape=input_shape) # Use Xception model to extract features base_model = Xception(weights=None, include_top=False, input_tensor=input_layer) # Do not use pre-trained weights x = base_model.output x = GlobalAveragePooling2D()(x) # Fusion of DDT features with Xception features ddt_features = extract_ddt_features(input_layer) x = Concatenate()([x, ddt_features]) # Add a fully connected layer for classification marking x = Dense(1024, activation='relu')(x) output_layer = Dense(num_classes, activation='softmax')(x) # Build the model model = Model(inputs=input_layer, outputs=output_layer) return model 4. During the training process, the Adam optimizer and sparse classification cross entropy loss function are used for compilation, and the training process is controlled by the set rounds (epochs) and batch size (batch_size). The training data and verification data are input into the model for training and verification respectively, and the model parameters are continuously optimized. The specific code is as follows: # Train the model def train_model(model, train_data, train_labels, validation_data,validation_labels, epochs, batch_size): # Compile the model model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) # Train the model model.fit(train_data, train_labels, validation_data=(validation_data, validation_labels), epochs=epochs, batch_size=batch_size) 5. During the evaluation process, the performance of the crop disease detection model is verified to ensure that the model has a high classification accuracy and completes the automatic detection and classification tasks of crop diseases. The specific code is as follows: # Model evaluation def evaluate_model(model, X_val, y_val): predictions = model.predict(X_val) y_pred = np.argmax(predictions, axis=1) y_true = y_val.flatten() # Make sure labels are the correct shape print("Classification Report:\n", classification_report(y_true,y_pred)) print("Confusion Matrix:\n", confusion_matrix(y_true, y_pred)) 6. In the main program, load the image data and labels by calling the load_data function, then use the train_test_split function to divide the data set into training and validation sets, and use StandardScaler to standardize the data to improve the training effect of the model. Next, call the build_ddt_xception_model function to build the DDT-Xception model. After the model is built, call the train_model function to train the model. Finally, call the evaluate_model function to evaluate the model. The specific code is as follows: # Main program entry if __name__ == '__main__': # Load the dataset images, labels = load_data() # Determine the shape of the input image, for example (none, none, 1) represents a single-channel image input_shape = (none, none, 1) num_classes = 2 # Assume there are 2 categories # Divide into training set and validation set X_train, X_val, y_train, y_val = train_test_split(images, labels,test_size=0.2, random_state=42) # Standardize data scaler = StandardScaler() X_train = scaler.fit_transform(X_train.reshape(-1, X_train.shape[-1])).reshape(X_train.shape) X_val = scaler.transform(X_val.reshape(-1, X_val.shape[-1])).reshape(X_val.shape) # Build DDT-Xception model model = build_ddt_xception_model(input_shape, num_classes) # Train the model train_model(model, X_train, y_train, X_val, y_val, epochs=50,batch_size=32) # Model evaluation evaluate_model(model, X_val, y_val) So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A crop disease detection method based on drones, characterized in that: The following steps are involved: S1: Obtain multispectral images of crops through the multispectral camera carried by the drone; S2: performing image annotation, scaling and normalization preprocessing on the multispectral image to obtain a training set, and then performing image annotation on the training set to obtain a test set including the multispectral image and corresponding lesion annotations; S3: constructing a crop disease detection model according to the DDT model and the Xception structure, wherein the DDT model is used to extract a first feature vector from the preprocessed multispectral image, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification; S4: inputting the training set into the crop disease detection model to obtain a prediction result, and training the crop disease detection model with the goal of minimizing the difference between the prediction result and the test set to obtain a trained crop disease detection model; S5: Evaluate the trained crop disease detection model according to the preset performance indicators, and check the prediction of the crop disease detection model on different categories through the confusion matrix; S6: Use the Adam optimizer to tune the crop disease detection model based on the evaluation results to improve the convergence speed and accuracy of the crop disease detection model; S7: Use the optimized crop disease detection model to detect crop diseases.

2. The crop disease detection method based on drone according to claim 1, characterized in that: The image annotation step in step S1 includes manual annotation and / or automatic annotation.

3. The crop disease detection method based on drone according to claim 1, characterized in that: The preprocessing in step S2 also includes data enhancement, which increases the diversity of the image through rotation, scaling, and flipping operations.

4. The crop disease detection method based on drone according to claim 1, characterized in that: The DDT model in step S3 divides the preprocessed multispectral image into multiple small blocks, extracts and classifies features of each small block, and then combines the features of all the small blocks to obtain the first feature vector of the multispectral image.

5. The crop disease detection method based on drone according to claim 1, characterized in that: The Xception network in step S3 extracts the second eigenvector from the preprocessed multispectral image through the global average pooling layer, fuses the first eigenvector and the second eigenvector through the Concatenate layer, and inputs the fused eigenvector into the fully connected layer, processed by the ReLU activation layer of 1024 neurons, and finally classified by the output layer to obtain the prediction result.

6. The crop disease detection method based on drone according to claim 1, characterized in that: The step S4 also introduces a cross entropy loss function and a stochastic gradient descent optimizer for training.

7. The crop disease detection method based on drone according to claim 1, characterized in that: The performance indicator preset in step S5 includes at least one of accuracy, recall, F1 score and precision.

8. Application of a crop disease detection method based on drone according to any one of claims 1 to 7 in detecting tobacco cucumber mosaic virus disease on tobacco leaves, wherein the multispectral camera comprises a 650nm red light sensor.

Images