An unmanned aerial vehicle-based crop disease detection method and application

Abstract

The application relates to the technical field of image recognition, in particular to a crop disease detection method based on a UAV and application, which comprises the following steps: acquiring a multispectral image of crops, performing image labeling, scaling and normalization preprocessing on the multispectral image, obtaining a training set, performing image labeling on the training set, obtaining a test set containing the multispectral image and corresponding disease spot labeling; constructing a crop disease detection model according to a DDT model and an Xception structure; inputting the training set into the crop disease detection model to obtain a prediction result; evaluating the trained crop disease detection model according to preset performance indexes, and checking the prediction situation of the crop disease detection model on different categories through a confusion matrix; adjusting the crop disease detection model according to an evaluation result by using an Adam optimizer; and realizing crop disease detection by using the optimized crop disease detection model. Compared with the prior art, the method is faster in processing speed, less in resource demand, and more practical and significant in popularization.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically providing a method and application for crop disease detection based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Crop disease marker identification is an important research direction in modern agriculture and a crucial issue in agricultural production. Failure to detect and control diseases in a timely manner can lead to reduced crop yields, decreased quality, and even crop death, causing severe economic losses to farmers. Currently, the main methods for crop disease marker identification are manual observation of disease symptoms and enzyme-linked immunosorbent assay (ELISA). However, these two methods are time-consuming and labor-intensive, heavily reliant on expert and technical expertise, and easily influenced by subjective judgment, resulting in low detection efficiency and accuracy. Furthermore, manual observation or sampling in the field can easily become a medium for the spread of pathogens, directly leading to large-scale outbreaks of diseases in the field.

[0003] Currently, the identification and labeling of crop diseases mainly relies on sensory perception and experiential memory. However, since the emergence of several breakthroughs in artificial intelligence (AI) technology, research in various fields has begun to explore its application in crop disease identification and labeling. For example, when crops are diseased, lesions appear on the affected areas of the leaves, clearly distinguishing them from other normal leaves. These lesions exhibit significant differences in their reflection and absorption of certain wavelengths of light, particularly electromagnetic waves. By utilizing technologies such as computer vision, image processing, and deep learning, the appearance characteristics of crops can be automatically analyzed and identified, allowing for a quick and accurate determination of whether crops are affected by diseases. AI technology can also improve the efficiency of disease identification by learning from and analyzing large amounts of historical data to establish classification models for crop diseases. For example, when crops are diseased, lesions appear on the affected areas of the leaves, clearly distinguishing them from other normal leaves. These lesions exhibit significant differences in their reflection and absorption of certain wavelengths of light, particularly electromagnetic waves.

[0004] In existing technologies, applications of artificial intelligence include: Everton Castelão Tetila et al., in 2017, used DJI Phantom 3 to collect data and employed an SVM classification model to help farmers detect three distinctive diseases of soybean: leaf spot, powdery mildew, and leafless disease. Feng Ziheng et al. used multispectral and thermal imaging technologies to extract wheat vegetation indices, texture features, and canopy temperature information. They then used multiple linear regression (MLR), backpropagation neural network (BP), random forest (RF), and extreme learning machine (ELM) algorithms to combine vegetation indices (VIs), texture features (TFs), and temperature features (T) to construct a monitoring model for wheat powdery mildew disease indices. Wang Jiantao et al. used normal citrus leaves, sooty mold leaves, canker leaves, spider mite leaves, and glyphosate-infected leaves as research samples. They collected 128 bands of hyperspectral images from 400 to 1000 nm and input them into a VGG16 convolutional neural network, achieving an accuracy of 98.75% under specific conditions.

[0005] Although the above methods have been proven to be effective, some shortcomings still exist, such as low accuracy and poor transferability. Summary of the Invention

[0006] The technical problem this invention aims to solve is to provide a crop disease detection method based on unmanned aerial vehicles (UAVs), which, compared to existing methods, has a faster processing speed and requires fewer resources, making it more practical and applicable for widespread use. It includes the following steps:

[0007] S1: Acquire multispectral images of crops using a multispectral camera mounted on a drone;

[0008] S2: The multispectral image is preprocessed by scaling and normalization to obtain a training set. The training set is then labeled to obtain a test set containing multispectral images and corresponding lesion labels.

[0009] S3: Construct a crop disease detection model based on the DDT model and the Xception structure. The DDT model is used to extract the first feature vector from the preprocessed multispectral image, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification.

[0010] S4: Input the training set into the crop disease detection model to obtain the prediction results. With the goal of minimizing the difference between the prediction results and the test set, train the crop disease detection model to obtain the trained crop disease detection model.

[0011] S5: Evaluate the trained crop disease detection model according to the preset performance indicators, and view the prediction performance of the crop disease detection model in different categories through the confusion matrix;

[0012] S6: Based on the evaluation results, use the Adam optimizer to fine-tune the crop disease detection model, thereby improving the convergence speed and accuracy of the crop disease detection model;

[0013] S7: Use the optimized crop disease detection model to detect crop diseases.

[0014] Furthermore, the image annotation step includes manual annotation and / or automatic annotation methods.

[0015] Furthermore, the preprocessing also includes data augmentation, which increases the diversity of the image through rotation, scaling, and flipping operations.

[0016] Furthermore, the DDT model divides the preprocessed multispectral image into multiple small blocks, extracts and classifies features from each small block, and then combines the features of all small blocks to obtain the first feature vector of the multispectral image.

[0017] Furthermore, the Xception network extracts a second feature vector from the preprocessed multispectral image through a global average pooling layer, fuses the first and second feature vectors through a concatenate layer, inputs the fused feature vector into a fully connected layer, processes it through a ReLU activation layer with 1024 neurons, and finally classifies it through an output layer to obtain the prediction result.

[0018] Furthermore, step S4 also introduces a cross-entropy loss function and a stochastic gradient descent optimizer for training.

[0019] Furthermore, the preset performance metrics include at least one of accuracy, recall, F1 score, and precision.

[0020] Furthermore, the present invention uses the above-described method to detect tobacco cucumber mosaic virus disease on tobacco leaves, wherein the multispectral camera includes a 650nm red light sensor.

[0021] The working principle and beneficial effects of this invention:

[0022] In implementing the technical solution of this invention, a multispectral camera mounted on a drone is used to collect multispectral images of crops. By analyzing the spectral reflectance of the canopy of each plant in different wavelength bands, the diseased parts can be reasonably marked. Using this method in agricultural disease control enables rapid and accurate detection and marking of crop diseases, helping agricultural technicians to predict disease occurrence in advance and monitor the disease situation in the field in real time.

[0023] Compared to existing methods, this approach offers faster processing speed and requires fewer resources, making it more widely applicable. Simultaneously, it enables real-time monitoring of crop disease impacts in the field, providing crucial information for developing prevention and control plans for potential or existing field diseases in tobacco fields. This facilitates managers in making scientific and targeted decisions, leading to more rational pesticide application, reduced environmental pollution, and increased agricultural product quality and yield. Furthermore, by extracting key information about crop diseases through deep learning, this information can be provided to policymakers and researchers to study disease transmission patterns, predict disease risks, and formulate scientific agricultural development strategies. Attached Figure Description

[0024] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0025] Figure 1 This is a schematic diagram of the main steps of a crop disease detection method based on unmanned aerial vehicles (UAVs) according to the present invention. Detailed Implementation

[0026] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0027] In this invention, the crop disease detection model mainly involves an improved classification and labeling method, DDT-Xception. This method employs Deep Degree Transforming (DDT) technology combined with the Xception convolutional neural network architecture to improve accuracy and reduce runtime. DDT is a deep learning-based image processing algorithm that segments an image into multiple small blocks, extracts and classifies features from each block, and finally combines the features of all blocks to obtain the feature representation of the entire image. Compared to traditional machine learning algorithms, DDT has better interpretability and generalization ability, and performs well in processing images in complex scenes. It can accurately detect objects in images and is very suitable for image processing in complex environments such as farmland. Xception is a lightweight CNN architecture whose main feature is the use of depthwise separable convolution. This convolution decomposes the convolution operation into two steps: 1. depthwise convolution; 2. pointwise convolution. This decomposition reduces computation and the number of parameters, while improving the model's accuracy and efficiency. In this invention, the extracted features are further processed and classified through the Xception structure, and these features are transformed into a higher-level abstract representation to adapt to image recognition tasks in complex environments. Example 1

[0028] Figure 1 This is a schematic diagram illustrating the main steps of a crop disease detection method based on unmanned aerial vehicles (UAVs) according to the present invention. Figure 1 As shown, a crop disease detection method based on unmanned aerial vehicles (UAVs) in this embodiment mainly includes the following steps S1-S7.

[0029] S1: Acquire multispectral images of crops using a multispectral camera mounted on a drone.

[0030] In one implementation, a multispectral image of the crop is obtained using a multispectral camera mounted on a DJI Phantom4 Multispectral drone, taking advantage of the characteristic that diseased crop leaves reflect light of corresponding wavelengths while healthy leaves strongly absorb it. In this embodiment, both the images used for the training set and the images during normal detection are acquired using this method.

[0031] S2: The multispectral images are scaled and normalized to obtain a training set, which is then used to adapt to the input requirements of the model. The training set is then labeled to obtain a test set containing multispectral images and corresponding lesion labels.

[0032] In one implementation, preprocessing also includes data augmentation, which increases the diversity of the image through rotation, scaling, and flipping operations to improve the model's generalization ability.

[0033] In one implementation, the image annotation step includes manual and / or automatic annotation methods to ensure the accuracy and consistency of the annotations.

[0034] S3: Construct a crop disease detection model based on the DDT model and the Xception structure. The DDT model is used to extract the first feature vector from the preprocessed multispectral image, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification.

[0035] In one implementation, the DDT model divides the preprocessed multispectral image into multiple small blocks, extracts and classifies features from each block, and then combines the features of all blocks to obtain the first feature vector of the multispectral image.

[0036] In one implementation, the Xception network extracts a second feature vector from the preprocessed multispectral image using a global average pooling layer. A concatenation layer fuses the first and second feature vectors. The fused feature vector is then input into a fully connected layer, processed by a 1024-neuron ReLU activation layer, and finally classified through an output layer to obtain the prediction result. The output layer uses the softmax activation function for multi-class classification.

[0037] S4: Input the training set into the crop disease detection model to obtain the prediction results. With the goal of minimizing the difference between the prediction results and the test set, train the crop disease detection model to obtain the trained crop disease detection model.

[0038] Furthermore, a cross-entropy loss function and a stochastic gradient descent optimizer are introduced for training to prevent overfitting.

[0039] S5: Evaluate the trained crop disease detection model based on accuracy, recall, F1 score, and precision, and examine the prediction performance of the crop disease detection model on different categories through the confusion matrix.

[0040] S6: Based on the evaluation results, use the Adam optimizer to fine-tune the crop disease detection model, thereby improving the convergence speed and accuracy of the crop disease detection model.

[0041] S7: Use the optimized crop disease detection model to detect crop diseases.

[0042] After model training and evaluation, the crop disease detection model is deployed to real-world application scenarios. Due to the model's portability, it can be used in automated crop disease detection systems deployed on low-level computing devices such as drones or ground stations. By analyzing multispectral image data of crops, it can automatically identify lesions on crop leaves.

[0043] Based on steps S1-S7 above, multispectral images of crops are acquired using a multispectral camera mounted on a drone. By analyzing the spectral reflectance of the canopy of each plant in different wavelength bands, diseased parts can be appropriately marked. Using this method in agricultural disease control enables rapid and accurate detection and marking of crop diseases, helping agricultural technicians to predict disease occurrence in advance and monitor the disease situation in the field in real time.

[0044] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention. Example 2

[0045] Based on the technical solution of Example 1, this example uses tobacco as the crop and tobacco-cucumber mosaic virus as the disease. A detection method for tobacco-cucumber mosaic virus is proposed based on the characteristics of large tobacco leaves and obvious lesions. Using a 650nm red light sensor mounted on a DJI Phantom4 Multispectral UAV, the method leverages the strong reflection of 650nm red light by diseased tobacco leaves and the strong absorption by normal leaves. DDT (Deep Degree Transforming) model is used to label lesion areas with a significant positive correlation on the leaves of tobacco plants in the field, and these areas are used as potential target regions. A test set is obtained and merged with unlabeled images to form a dataset for training the crop disease recognition model. This dataset is then input into an Xception structure for further processing. In this way, DDT combined with Xception achieves high-quality labeling while preserving the structure and details of the image.

[0046] Specifically, the following steps are included:

[0047] S1: Acquire red-band images of the tobacco field using a multispectral camera mounted on a drone;

[0048] S2: The red band image is scaled and normalized to obtain the training set. The training set is then labeled to obtain the test set containing the red band image and the corresponding lesion label.

[0049] S3: Construct a crop disease detection model based on the DDT model and the Xception structure. The DDT model is used to extract lesion features from the preprocessed image to generate the first feature vector, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification.

[0050] S4: Input the training set into the crop disease detection model to obtain the prediction results. With the goal of minimizing the difference between the prediction results and the test set, train the crop disease detection model to obtain the trained crop disease detection model. Train the model using the cross-entropy loss function and stochastic gradient descent optimizer to prevent overfitting.

[0051] S5: Evaluate the trained crop disease detection model and verify its performance on the test set based on the performance metrics of accuracy and recall.

[0052] S6: Based on the evaluation results, use the Adam optimizer to fine-tune the crop disease detection model, thereby improving the convergence speed and accuracy of the crop disease detection model;

[0053] S7: Use the trained crop disease detection model to identify the red light band image to be analyzed and obtain the identification results of cucumber mosaic virus disease spots.

[0054] This embodiment enables real-time monitoring of crop disease impacts in the field, providing crucial reference for developing prevention and control plans for potential or existing field diseases in tobacco fields. This facilitates managers in making scientific and targeted decisions, leading to more rational pesticide application, reduced environmental pollution, and increased agricultural product quality and yield. Simultaneously, deep learning extracts key information about crop diseases, which can be provided to decision-makers and researchers to study disease transmission patterns, predict disease risks, and formulate scientific agricultural development strategies. However, while this method offers significant advantages, it may require further validation and improvement in practical applications to ensure accuracy and system stability. Example 3

[0055] Based on the technical solution of Embodiment 1, this embodiment proposes a framework code for DDT-Xception integration.

[0056] 1. The multispectral image is loaded and preprocessed using the `load_data` function. Specific operations include image scaling and cropping to ensure the data fits the model's input requirements. The code is as follows:

[0057] # Load the dataset and preprocess it

[0058] def load_data():

[0059] # TODO: Load single-channel image data in the 650nm band and perform preprocessing operations such as scaling and cropping.

[0060] # Here we assume the loaded data consists of a single-channel image and its corresponding label.

[0061] images = np.random.rand(100, 256, 256, 1) # Example data

[0062] labels = np.random.randint(0, 2, size=(100, 1)) # Example labels

[0063] return images, labels

[0064] 2. The `extract_ddt_features` function is used to implement the DDT model to extract the first feature vector from the preprocessed multispectral image. Specifically, a pre-trained deep convolutional model is used to extract the depth descriptor for each image, and the mean vector and covariance matrix are calculated to obtain the DDT features of the image. These features will be fused with the features extracted by the Xception model in subsequent steps. The specific code is as follows:

[0065] # Extracting DDT features

[0066] def extract_ddt_features(images):

[0067] # TODO: Extract the depth descriptor for each image using a pre-trained deep convolutional model, and compute the mean vector and covariance matrix.

[0068] # Example Implementation

[0069] return np.random.rand(images.shape[0], 256)

[0070] 3. The `build_ddt_xception_model` function implements the construction of a crop disease detection model. First, an input layer is defined, accepting image input of shape (None, None, 1). The first dimension, None, indicates that the input image can have any number of rows; the second dimension, None, indicates that the input image can have any number of columns; and the third dimension, 1, represents a single-channel 650nm band image. This means the model can handle single-channel images of different sizes (variable width and height). Then, an untrained Xception network is used to extract features from the preprocessed multispectral image, and a second feature vector is obtained through a global average pooling layer. Subsequently, DDT features (the first feature vector) of the preprocessed multispectral image are extracted and fused with the second feature vector using a concatenation layer. The fused feature vector is input to a fully connected layer, processed through a 1024-neuron ReLU activation layer, and finally classified through an output layer using a softmax activation function for multi-class classification. Ultimately, these steps construct a DDT-Xception neural network model for crop disease classification. The specific code is as follows.

[0071] # Constructing the DDT-Xception structure

[0072] def build_ddt_xception_model(input_shape, num_classes):

[0073] # Define the input layer

[0074] input_layer = tf.keras.Input(shape=input_shape)

[0075] # Extracting features using the Xception model

[0076] base_model = Xception(weights=None, include_top=False, input_tensor=input_layer) # Do not use pre-trained weights

[0077] x = base_model.output

[0078] x = GlobalAveragePooling2D()(x)

[0079] # Fusing DDT features with Xception features

[0080] ddt_features = extract_ddt_features(input_layer)

[0081] x = Concatenate()([x, ddt_features])

[0082] # Add a fully connected layer for classification and labeling

[0083] x = Dense(1024, activation='relu')(x)

[0084] output_layer = Dense(num_classes, activation='softmax')(x)

[0085] # Building a Model

[0086] model = Model(inputs=input_layer, outputs=output_layer)

[0087] return model

[0088] 4. During training, the Adam optimizer and sparse classification cross-entropy loss function are used for compilation. The training process is controlled by setting the number of epochs and the batch size. Training and validation data are input into the model for training and validation respectively, continuously optimizing the model parameters. The specific code is as follows:

[0089] # Training the model

[0090] def train_model(model, train_data, train_labels, validation_data,validation_labels, epochs, batch_size):

[0091] # Compilation Model

[0092] model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

[0093] # Training the model

[0094] model.fit(train_data, train_labels, validation_data=(validation_data, validation_labels), epochs=epochs, batch_size=batch_size)

[0095] 5. During the evaluation process, the performance of the crop disease detection model is verified to ensure that the model has a high classification accuracy and can complete the automatic detection and classification tasks of crop diseases. The specific code is as follows:

[0096] # Model Evaluation

[0097] def evaluate_model(model, X_val, y_val):

[0098] predictions = model.predict(X_val)

[0099] y_pred = np.argmax(predictions, axis=1)

[0100] y_true = y_val.flatten() # Ensure the label is the correct shape

[0101] print("Classification Report:\n", classification_report(y_true,y_pred))

[0102] print("Confusion Matrix:\n", confusion_matrix(y_true, y_pred))

[0103] 6. In the main program, the `load_data` function loads image data and labels, then the `train_test_split` function divides the dataset into training and validation sets, and `StandardScaler` is used to standardize the data to improve model training performance. Next, the `build_ddt_xception_model` function is called to build the DDT-Xception model. After model building, the `train_model` function is called to train the model. Finally, the `evaluate_model` function is called to evaluate the model. The specific code is as follows:

[0104] # Main program entry point

[0105] if __name__ == '__main__':

[0106] # Load dataset

[0107] images, labels = load_data()

[0108] # Determine the shape of the input image, for example, (none, none, 1) represents a single-channel image.

[0109] input_shape = (none, none, 1)

[0110] num_classes = 2 # Assuming there are 2 categories

[0111] # Divide the training set and validation set

[0112] X_train, X_val, y_train, y_val = train_test_split(images, labels,test_size=0.2, random_state=42)

[0113] # Standardized Data

[0114] scaler = StandardScaler()

[0115] X_train = scaler.fit_transform(X_train.reshape(-1, X_train.shape[-1])).reshape(X_train.shape)

[0116] X_val = scaler.transform(X_val.reshape(-1, X_val.shape[-1])).reshape(X_val.shape)

[0117] # Building the DDT-Xception Model

[0118] model = build_ddt_xception_model(input_shape, num_classes)

[0119] # Training the model

[0120] train_model(model, X_train, y_train, X_val, y_val, epochs=50,batch_size=32)

[0121] # Model Evaluation

[0122] evaluate_model(model, X_val, y_val)

[0123] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for detecting crop diseases based on unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1: Acquire multispectral images of crops using a multispectral camera mounted on a drone; S2: Perform image annotation, scaling and normalization preprocessing on the multispectral image to obtain a training set, and then perform image annotation on the training set to obtain a test set containing multispectral images and corresponding lesion annotations; S3: Construct a crop disease detection model based on the DDT model and the Xception structure. The DDT model is used to extract the first feature vector from the preprocessed multispectral image, and the Xception structure is used to receive the first feature vector output by the DDT model for further processing and classification. S4: Input the training set into the crop disease detection model to obtain the prediction results. With the goal of minimizing the difference between the prediction results and the test set, train the crop disease detection model to obtain the trained crop disease detection model. S5: Evaluate the trained crop disease detection model according to the preset performance indicators, and view the prediction performance of the crop disease detection model in different categories through the confusion matrix; S6: Based on the evaluation results, use the Adam optimizer to fine-tune the crop disease detection model, thereby improving the convergence speed and accuracy of the crop disease detection model; S7: Use the optimized crop disease detection model to detect crop diseases.

2. The crop disease detection method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The image annotation step in step S1 includes manual annotation and / or automatic annotation.

3. The crop disease detection method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The preprocessing in step S2 also includes data augmentation, which increases the diversity of the image through rotation, scaling, and flipping operations.

4. The crop disease detection method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The DDT model in step S3 divides the preprocessed multispectral image into multiple small blocks, extracts and classifies features from each small block, and then combines the features of all small blocks to obtain the first feature vector of the multispectral image.

5. A method for detecting crop diseases based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, In step S3, the Xception network extracts the second feature vector from the preprocessed multispectral image through a global average pooling layer, fuses the first and second feature vectors through a concatenate layer, inputs the fused feature vector into a fully connected layer, processes it through a ReLU activation layer with 1024 neurons, and finally classifies it through the output layer to obtain the prediction result.

6. The crop disease detection method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Step S4 also introduces the cross-entropy loss function and stochastic gradient descent optimizer for training.

7. The crop disease detection method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The preset performance indicators in step S5 include at least one of accuracy, recall, F1 score, and precision.

8. The application of the UAV-based crop disease detection method according to any one of claims 1 to 7 in detecting tobacco cucumber mosaic virus disease on tobacco leaves, wherein the multispectral camera includes a 650nm red light sensor.

Images