Identification method of pine wood nematode disease based on UAV remote sensing images

Abstract

The present invention discloses a method for identifying pine wilt disease based on drone remote sensing imagery, primarily addressing the problem of poor pine wilt disease identification results in existing technologies. The method comprises the following steps: using drones to capture images in pine wilt disease-infected areas to obtain pine wilt disease training and test sets; performing mirroring, rotation, color conversion, scaling, nonlinear scaling, translation, salt and pepper noise, and fisheye enhancement on the training sets to generate eight enhanced training sets; using these training sets to iteratively train pine wilt disease recognition models; inputting the test sets into each of the eight trained pine wilt disease recognition models to obtain image sets output by each model, labeling the pine wilt disease category and confidence level; calculating the average precision of the eight output image sets, and selecting the image set with the highest average precision as the pine wilt disease identification result. The present invention avoids manual intervention, improves the accuracy of pine wilt disease detection, and can be used for monitoring, early warning, and prevention of pine wilt disease and insect pests in pine farming and forestry.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method for identifying pine wood nematode disease, which can be used for monitoring, early warning, and prevention of agricultural and forestry pests and diseases. Background Art

[0002] Pine trees are a key component of my country's forest resources, accounting for over 70% of the country's planted forest area. However, in recent years, pine trees have been severely damaged by the invasive pine wood nematode, an invasive pest. Given the high pathogenicity, rapid onset, rapid spread, and difficulty in control, the most effective approach currently is to promptly detect and accurately locate infected trees and dispose of them promptly. Due to the limitations of manual surveys and satellite remote sensing in monitoring complex forest areas, drone remote sensing, with its real-time, low-cost, and high spatial resolution, has become widely used for low-altitude remote sensing in agriculture and forestry.

[0003] In recent years, deep learning has achieved breakthroughs in image recognition. However, it often requires a large amount of labeled data as a sample set. In most practical applications, due to the lack of labeled data, neural networks are prone to overfitting or poor generalization. This phenomenon is particularly evident in small-scale datasets. The overfitting problem is mainly caused by three reasons: model complexity, image noise, and limited training data. The quality of image recognition algorithms is related to the quality and quantity of the dataset. However, due to time and financial cost constraints, the image datasets obtained are insufficient, of poor quality, and have an uneven sample distribution, which makes the image recognition task difficult. Currently, most researchers mainly use rotation and flipping to enhance diseased pine tree samples.

[0004] Data augmentation is an important means of improving the training performance of convolutional neural networks. It includes a range of augmentation methods, including rotation, flipping, scaling, translation, and noise addition. Traditional data augmentation strategies are based on camera models and imaging principles. By using different augmentation methods, they can simulate the effects of lenses, focal length, and aperture on images in real-world scenarios. For example, a fisheye lens distorts images but allows for a wider scene within a smaller one; focal length blurs the background and affects image resolution; and aperture affects image brightness and color. The fisheye effect can simulate images captured with a fisheye lens. Scaling can simulate different focal lengths and shooting distances; color transformation can simulate images with different apertures and lighting conditions; translation and rotation can simulate different perspectives; and Gaussian and salt-and-pepper noise can simulate noise generated by camera sensors. While augmentation strategies can effectively increase the number of training sets, sample diversity, model accuracy, and reduce the time and financial costs of data acquisition, different augmentation strategies have varying degrees of impact on model recognition results. Summary of the Invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose a pine wood nematode disease identification method based on UAV remote sensing images to reduce the probability of overfitting, enhance the generalization and learning capabilities of the network model, and improve the recognition accuracy of pine wood nematode disease.

[0006] To achieve the above objectives, the technical solutions of the present invention include the following:

[0007] (1) Using drones in pine wilt disease epidemic areas, we obtain a pine wilt disease dataset and perform cropping preprocessing on it to obtain the preprocessed pine wilt disease dataset B.

[0008] (2) Mark the pine wood nematode-infected trees in the preprocessed dataset B and divide them into a training set T and a test set S in a ratio of 8:2;

[0009] (3) Perform data enhancement on the training set T to generate the enhanced training set:

[0010] (3a) Perform eight enhancements on the training set T using mirroring, rotation, color transformation, scaling, nonlinear scaling, translation, salt and pepper noise, and fisheye effects, generating eight enhanced training sets.

[0011] (3b) Each enhanced training set is combined with the original training set to form eight new training sets:

[0012] in represents the jth new training set, j = 1, 2, ..., 8;

[0013] (4) Select the existing Faster R-CNN target detection network as the pine wood nematode disease recognition model M;

[0014] (5) Each new training set Input into the pine wilt disease recognition model M, and use the gradient descent method to iteratively train the pine wilt disease recognition model to obtain eight trained pine wilt disease recognition models:

[0015]

[0016] in represents the trained pine wood nematode disease recognition model corresponding to the j-th new training set;

[0017] (6) Input the test set S into the eight trained pine wood nematode disease recognition models respectively In the example, we obtain the image set of labeled pine wilt disease categories and confidence information output by each pine wilt disease recognition model:

[0018] Q={Q1,Q2,…,Qj ,Q8,},

[0019] where Q j represents the image set corresponding to the j-th trained pine wilt disease recognition model;

[0020] (7) Calculate the average precision AP of the output image sets Q of the eight pine wood nematode disease recognition models, compare them, and select the image set Q' with the highest average precision AP as the pine wood nematode disease recognition result.

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] 1) The eight data enhancement techniques used in this paper depend on the UAV camera model, imaging principle, and changes in the external environment, and are suitable for the identification of pine wood nematode disease under different shooting angles, different lighting conditions, and different camera lenses;

[0023] 2) The present invention uses eight trained pine wood nematode disease recognition models By testing pine wilt-infected trees in pine wilt-infected areas and obtaining detection frames marked with pine wilt disease, the network model's generalization and learning capabilities were improved, and the selected Faster R-CNN network achieved high accuracy.

[0024] 3) The present invention selects the image set Q with the highest average precision AP as the pine wood nematode disease recognition result, which not only saves time and financial costs, but also improves the accuracy of pine wood nematode disease recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is an implementation flow chart of the present invention;

[0026] Figure 2 It is a simulation result diagram of the present invention. DETAILED DESCRIPTION

[0027] The specific embodiments and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0028] Reference Figure 1 , the implementation steps of this example are as follows

[0029] Step 1: Use drones to obtain the pine wilt disease dataset in the pine wilt disease epidemic area, and perform cropping and label preprocessing on it to obtain the pine wilt disease training sample set T:

[0030] 1.1) Use a drone to capture images of pine wilt disease in an epidemic area. Crop the acquired pine wilt disease images with a 640*640 pixel window and a step size of 480. Extract 150 images of pine wilt diseased trees from them, A = {A1, A2, …, Ak ,A N}, where A k represents the kth image containing pine wood nematode-infected wood, N = 150;

[0031] 1.2) Use labellmg labeling software to label each image A k The pine wood nematode diseased trees are labeled to obtain the label set L containing the pine wood nematode diseased trees information L = {L1, L2, ..., L k ,L N}, L k Indicates A k The corresponding label containing information about pine wood nematode-infected trees;

[0032] 1.3) The label set L containing the information of pine wood nematode diseased trees is set to {L1, L2, ..., L k ,L N} and the corresponding images of pine wood nematode diseased wood A={A1,A2,…,A k ,A N} are combined to form a sample set E={E1,E2,…,E k ,E N}, where E k For L k and A k The kth sample formed by the combination;

[0033] 1.4) Divide the sample set E into the training set T = {T1, T2, ..., T m ,T I} and test set S={S1,S2,…,S n ,S J}, where T m represents the mth training sample containing pine wood nematode diseased wood, S n represents the nth test sample containing wood infected by pine wood nematodes, I and J are the total number of samples in the training set and the test set, respectively, I = 120, J = 30.

[0034] Step 2: Perform data enhancement on the training set T to generate an enhanced training set.

[0035] 2.1) Eight strategies for data augmentation of training set T:

[0036] This step uses eight strategies, namely mirroring, rotation, color transformation, scaling, nonlinear scaling, translation, salt and pepper noise, and fisheye effects, to enhance the training set T and generate eight enhanced training sets. The implementation of each enhancement strategy is as follows:

[0037] The mirror enhancement of the training set T is to swap the left and right parts of each image in the training set T with its vertical midline as the axis, or swap the upper and lower parts of the image with its horizontal midline as the axis, to obtain the enhanced first training set T'1;

[0038] The rotation enhancement of the training set T is to randomly rotate each image in the training set T by 90°, 180° or 270° clockwise to obtain an enhanced second training set T'2;

[0039] The color transformation enhancement of the training set T is to randomly adjust the brightness, contrast or saturation of each image in the training set T to obtain an enhanced third training set T'3;

[0040] The scaling enhancement of the training set T is to linearly enlarge or reduce each image in the training set T, crop the area that exceeds the size of the original image after enlargement, and fill the empty part between the boundary of the reduced image and the original image as the background to obtain the enhanced fourth training set T'4;

[0041] The nonlinear scaling enhancement of the training set T is to enlarge or reduce each image in the training set T with an aspect ratio different from that of the original image to obtain an enhanced fifth training set T'5;

[0042] The translation enhancement of the training set T is to move each image in the training set T to the right by 50 pixels along the horizontal axis to obtain the enhanced sixth training set T'6;

[0043] The salt and pepper noise enhancement of the training set T is to randomly select some pixels from each image in the training set T and change their values ​​to 0 or 255 to obtain the enhanced seventh training set T'7;

[0044] The fisheye special effect enhancement of the training set T is to simulate the fisheye effect on each image in the training set T according to the imaging principle of a fisheye camera to obtain an enhanced eighth training set T'8.

[0045] 2.2) Each training set after enhancement T'={T'1,T'2,…,T' j ,T'8} are combined with the original training set T to form eight new training sets:

[0046]

[0047] in, T' j The jth new training set formed by combining with T, j = 1, 2, ..., 8;

[0048] Step 3: Select the existing Faster R-CNN target detection network as the pine wood nematode disease recognition model M.

[0049] The Faster R-CNN target detection network includes a backbone network, a target feature network, and a detection network.

[0050] The backbone network is composed of a Resnet50 network, which includes a Conv Block module and an Identity Block module, and is composed of a convolutional layer Conv, a batch normalization layer BN, and an activation function Relu.

[0051] Target feature network Neck, composed of feature pyramid module FPN;

[0052] The detection network Head consists of RPN module and RCNN module;

[0053] The backbone network Backbone, target feature network Neck and detection network Head are cascaded to form a pine wood nematode disease recognition model.

[0054] Step 4: Eight new training sets The data is input into the pine wood nematode disease recognition model M, and the pine wood nematode disease recognition model is iteratively trained.

[0055] This example uses the gradient descent method to iteratively train the pine wood nematode disease recognition model to achieve the following

[0056] 4.1) Set the maximum number of iterations to X = 1125 and initialize the number of iterations to y = 0;

[0057] 4.2) The new training set As a model for pine wood nematode disease detection The multi-scale feature map C1 is obtained through the Conv Block module and Identity Block module in the backbone network Resnet50. j 、C2 j 、C3 j 、C4 j 、C5 j ,in is the pine wilt disease detection model of the yth iteration corresponding to the jth training set, j = 1, 2, …, 8;

[0058] 4.3) The multi-scale feature map C2 obtained in 4.2) j 、C3 j 、C4 j 、C5 jThe data is sent to the top-down feature pyramid FPN structure, and the number of channels of each feature map is changed to 256 through 1*1 convolution. Then, the high-level features and the low-level features are fused through upsampling to obtain a fused feature map.

[0059] 4.4) The fused feature map obtained in 4.3) is fed into the existing detection network head. The RPN module in the network performs a 3x3 sliding window operation on the feature map to generate nine anchors of different proportions and scales. Non-maximum suppression is used to select target candidate boxes with an IOU greater than 0.5.

[0060] 4.5) Using the cross entropy loss function and the Smooth L1 Loss loss function, calculate the classification and regression loss value Loss1 of the RPN module:

[0061]

[0062] Among them, P i Indicates the probability that the i-th anchor is predicted to be the true label, when the predicted sample is a positive sample is 1 when the predicted sample is a negative sample is 0, t i Represents the bounding box regression parameters for predicting the i-th anchor, represents the true bounding box parameter corresponding to the i-th anchor, L cls Indicates the number of samples in a mini-bath, L reg Indicates the number of anchor positions;

[0063] 4.6) Project the candidate box obtained in 4.4) to C1 obtained in 4.2) j The corresponding feature matrix is ​​obtained from the feature map, and then the obtained feature matrix is ​​passed through the fully connected layer and softmax of the RCNN module in the network to obtain a feature vector with category and position information;

[0064] 4.7) Using the cross entropy loss function and the Smooth L1 Loss loss function, calculate the classification and regression loss values ​​Loss2 of the RCNN module:

[0065]

[0066] Where p is the softmax probability distribution predicted by the classifier, u is the true label of the target, and t u is the regression parameter of the bounding box regressor corresponding to u, and v is the bounding box regression parameter of the true target;

[0067] 4.8) Add the results obtained in 4.6) and 4.7) to obtain the pine wood nematode disease identification model The loss value Loss, and then the model is The weights are updated to obtain the pine wood nematode disease recognition model after the y+1th iteration

[0068] 4.9) Determine whether the current number of iterations has reached the set maximum number of iterations X:

[0069] If so, the trained pine wood nematode disease recognition model is obtained

[0070] Otherwise, set y=y+1 and return to step 4.3).

[0071] 4.10) For each new training set obtained in step 2 Repeat steps 4.2) to 4.9) to obtain eight trained pine wood nematode disease recognition models. in Represents the trained pine wood nematode disease recognition model corresponding to the j-th new training set.

[0072] Step 5: Use the trained pine wood nematode disease recognition model to obtain the category and confidence information image set of the image to be tested.

[0073] The test set S is input into eight trained pine wood nematode disease recognition models respectively. In the example, we obtain the image set of labeled pine wilt disease categories and confidence information output by each pine wilt disease recognition model:

[0074] Q={Q1,Q2,…,Q j ,Q8,},

[0075] where Q j Represents the image set corresponding to the j-th trained pine wilt disease recognition model.

[0076] Step 6: Calculate the average precision AP of the output image set Q of the eight pine wilt disease recognition models to obtain the pine wilt disease recognition results.

[0077] 7a) Output image set Q for each pine wilt disease recognition model j , calculate the intersection-over-union (IOU) of each predicted target box and sort each predicted target box according to its confidence level;

[0078] 7b) Calculate the precision P and recall R of each predicted target box at different confidence thresholds:

[0079]

[0080]

[0081] Among them, TP represents the positive sample predicted by the model as the positive class, FP represents the negative sample predicted by the model as the positive class, FN represents the positive sample predicted by the model as the negative class, and TN represents the negative sample predicted by the model as the negative class;

[0082] 7c) Draw a PR curve based on the precision P and recall R. Integrate the PR curve to obtain the area of ​​the image enclosed by the curve, which is the average precision AP of the output image set Q of the pine wilt disease recognition model.

[0083]

[0084] 7d) Select the image set Q' with the highest average precision AP as the pine wood nematode disease recognition result, completing the recognition of pine wood nematode disease.

[0085] The effects of the present invention are further illustrated by the following simulation results.

[0086] Use PyCharm software to simulate the average precision AP of each trained pine wood nematode disease recognition model output image set Q and draw a curve graph, as shown in the following figure: Figure 2 shown.

[0087] The statistics of eight trained pine wood nematode disease recognition models and their corresponding average precision AP values ​​are shown in Table 1;

[0088] Table 1 Eight trained pine wilt disease recognition models and their corresponding average precision AP

[0089] Pine wood nematode disease identification model Average precision AP 1 0.721 2 0.630 3 0.683 4 0.894 5 0.798 6 0.648 7 0.720 8 0.777

[0090] from Figure 2 As can be seen from Table 1, the output image set of the fourth trained pine wilt disease recognition model has the highest average precision AP, so its corresponding image set is selected as the pine wilt disease recognition result.

Claims

1. A method for identifying pine wood nematode disease based on UAV remote sensing images, characterized in that: The steps include: (1) Using drones in pine wilt disease epidemic areas, we obtain a pine wilt disease dataset and perform cropping preprocessing on it to obtain the preprocessed pine wilt disease dataset B. (2) Mark the pine wood nematode-infected trees in the preprocessed dataset B and divide them into a training set T and a test set S in a ratio of 8:2; (3) Perform data enhancement on the training set T to generate the enhanced training set: (3a) Perform eight enhancements on the training set T using mirroring, rotation, color transformation, scaling, nonlinear scaling, translation, salt and pepper noise, and fisheye effects, generating eight enhanced training sets. (3b) Each enhanced training set is combined with the original training set to form eight new training sets: in represents the jth new training set, j = 1, 2, ..., 8; (4) Select the existing Faster R-CNN target detection network as the pine wood nematode disease recognition model M; (5) Each new training set Input into the pine wilt disease recognition model M, and use the gradient descent method to iteratively train the pine wilt disease recognition model to obtain eight trained pine wilt disease recognition models: in represents the trained pine wood nematode disease recognition model corresponding to the j-th new training set; (6) Input the test set S into the eight trained pine wood nematode disease recognition models respectively In the example, we obtain the image set of labeled pine wilt disease categories and confidence information output by each pine wilt disease recognition model: Q={Q1,Q2,…,Q j ,Q8,}, where Q j represents the image set corresponding to the j-th trained pine wilt disease recognition model; (7) Calculate the average precision AP of the output image sets Q of the eight pine wood nematode disease recognition models, compare them, and select the image set Q' with the highest average precision AP as the pine wood nematode disease recognition result.

2. The method according to claim 1, wherein the preprocessing of clipping the pine wilt disease dataset in step (1) is implemented as follows: (1a) The acquired pine wilt disease dataset is cropped with a 640*640 window with a step size of 480, and N images of pine wilt diseased wood are extracted from it, A={A1,A2,…,A k ,A N }, where A k Indicates that the kth image contains an image of a wood infected with pine wood nematodes; (1b) Use labellmg labeling software to label each image A k The pine wood nematode diseased trees are labeled to obtain the label set L containing the pine wood nematode diseased trees information L = {L1, L2, ..., L k ,L N }, L k Indicates A k The corresponding label contains information about pine wood nematode-infected trees.

3. The method according to claim 1, wherein Step (3a) performs eight enhancement strategies on the training set T, including mirroring, rotation, color transformation, scaling, nonlinear scaling, translation, salt and pepper noise, and fisheye special effects, as follows: The mirror enhancement is to swap the left and right parts of each image in the training set T with its vertical midline as the axis, or swap the upper and lower parts of the image with its horizontal midline as the axis; The rotation enhancement is to randomly rotate each image in the training set T by 90°, 180° or 270° clockwise; The color transformation enhancement is to randomly adjust the brightness, contrast or saturation of each image in the training set T; The scaling enhancement is to linearly enlarge or reduce each image in the training set T, and crop the area that exceeds the size of the original image after enlargement, and fill the empty part between the boundary of the reduced image and the original image as the background; The nonlinear scaling enhancement is to enlarge or reduce each image in the training set T with an aspect ratio different from that of the original image; The translation enhancement is to move each image in the training set T to the right by 50 pixels along the horizontal axis; The salt and pepper noise enhancement is to randomly select some pixels from each image in the training set T and change their values ​​to 0 or 255; The fisheye special effect enhancement is to simulate the fisheye effect for each image in the training set T according to the imaging principle of the fisheye camera.

4. The method according to claim 1, wherein The Faster R-CNN target detection network selected in step (4) is composed of a cascade of the backbone network, the target feature network Neck, and the detection network Head; The backbone network Backbone is composed of a Resnet50 network, which includes a Conv Block module and an Identity Block module, and is composed of a convolutional layer Conv, a batch normalization layer BN, and an activation function Relu; The target feature network Neck is composed of a feature pyramid module FPN; The detection network Head consists of an RPN module and an RCNN module.

5. The method according to claim 1, wherein in (5), the pine wood nematode disease recognition model is iteratively trained using a gradient descent method, and is implemented as follows: 5a) Set the maximum number of iterations to X = 1125 and initialize the number of iterations to y = 0; 5b) Eight new training sets As a detection model for pine wood nematode disease The input is passed through the backbone network Resnet50 to obtain the multi-scale feature map C1 j 、C2 j 、C3 j 、C4 j 、C5 j ,in is the pine wilt disease detection model of the yth iteration corresponding to the jth training set, j = 1, 2, …, 8; 5c) The multi-scale feature map C2 obtained in 5b) j 、C3 j 、C4 j 、C5 j The data is sent to the target feature network Neck, and through the top-down feature pyramid FPN structure, the high-level features are fused with the low-level features through upsampling to obtain a fused feature map; 5d) The fused feature map obtained in 5c) is fed into the existing detection network Head, and the target candidate box is generated by the RPN module in the network, and the candidate box is projected onto the C1 obtained in 5a) j The corresponding feature matrix is ​​obtained from the feature map, and then the obtained feature matrix is ​​passed through the RCNN module in the network to obtain the feature vector with category and position information; 5f) Using the cross entropy loss function and the Smooth L1 Loss loss function, the classification and regression loss values ​​of the RPN module and the RCNN module are calculated. The Loss is then used to identify the pine wood nematode disease model. The weights are updated to obtain the pine wood nematode disease recognition model after the y+1th iteration 5g) Determine whether the current number of iterations has reached the set maximum number of iterations X: If so, the trained pine wood nematode disease recognition model is obtained Otherwise, set y = y + 1 and return to 5c).

6. The method according to claim 1, wherein in step (7), the average precision AP of the eight pine wilt disease recognition model output image sets Q is calculated as follows: 7a) Output image set Q for each pine wilt disease recognition model j , calculate the intersection-over-union (IOU) of each predicted target box and sort each predicted target box according to its confidence level; 7b) Calculate the precision P and recall R of each predicted target box at different confidence thresholds, and draw a PR curve based on the precision P and recall R. 7c) Integrate the PR curve and find the area of ​​the image enclosed by it, which is the average precision AP.

Images