Unmanned aerial vehicle image pine wood nematode disease epidemic wood identification method based on improved U-Net

Abstract

The invention relates to an unmanned aerial vehicle (UAV) image pine wood nematode disease epidemic wood identification method based on improved U-Net. The method comprises the following steps: acquiring a forest region image through an UAV and preprocessing the forest region image; performing data enhancement on the preprocessed image data and the annotated image data; the U-Net network model is improved, and an improved U-Net network model is obtained; inputting the training set into the improved U-Net network model for training; and inputting a to-be-identified image into the improved U-Net network model to obtain an identification result. According to the invention, the Swin Transformer encoder is fully utilized to make up for the defect of the U-Net network model itself in capturing long-distance dependence, so that the precision of identifying the pine wood infected with the pine wood nematode disease is improved; a bottleneck layer of a U-Net network model is replaced by a cavity space pyramid pooling module, image features of different scales are fully captured, the receptive field is increased, context information is fully utilized, feature information loss is reduced, more valuable information is captured, and the precision of pine wood nematode disease infected diseased wood identification is improved.

Description

Technical Field

[0001] The present invention relates to the field of pest disease epidemic identification and deep learning technology, and in particular to a method for identifying pine wood nematode diseased trees using drone images based on an improved U-Net. Background Art

[0002] Pine wilt disease, also known as pine wilt disease, is a systemic infectious disease caused by pine nematodes. It has the characteristics of rapid spread, fast onset, high mortality rate and great difficulty in prevention and control. Since it was introduced into my country in 1982, at least 17 pine species, including black pine, red pine, masson pine, Ryukyu pine and white pine, have been infected and eventually withered and died. Therefore, accurate monitoring of pine wilt disease is of great significance to prevent its rapid spread. Traditional survey methods mainly rely on manual ground surveys, but they are time-consuming and labor-intensive, and are difficult to carry out over large areas due to the limitations of complex terrain such as mountainous areas. This leads to the need for a detection method that can monitor a wide area. High-resolution satellite remote sensing data can provide regional pine forest damage status, but it is difficult to identify individual diseased trees due to low spatial resolution. In recent years, with the continuous development of drone remote sensing technology, drone spectral images have been widely used as a data source in the identification of diseased trees.

[0003] Deep learning algorithms can directly process raw data and automatically learn feature representations to achieve end-to-end recognition. They have been widely used in the identification of diseased trees. However, traditional deep learning algorithms are mainly constructed using convolutional neural networks, which have limited receptive fields, and deep networks weaken the long-range dependencies of features, leading to problems such as missed detection of small targets. In addition, individual diseased trees vary in size and are distributed in a jumping manner in the image, the forest landscape is complex, and the background information is highly interfering. In addition, traditional deep learning algorithms are prone to losing target information during the downsampling process, resulting in blurred boundaries, loss of small targets, and other problems, affecting the recognition effect. Summary of the invention

[0004] In order to solve the problems of blurred boundaries and target omission in the identification of pine wilt diseased trees using traditional deep learning methods, the purpose of the present invention is to provide a method for identifying pine wilt diseased trees using drone images based on an improved U-Net, which reduces the loss of feature information, captures more valuable information, and improves the accuracy of identifying pine wilt diseased trees.

[0005] To achieve the above object, the present invention adopts the following technical solution: a method for identifying pine wood nematode diseased trees using drone images based on improved U-Net, the method comprising the following steps in order:

[0006] (1) Obtain forest area images through drones and preprocess them, mark pine wood nematode-infected trees in the forest area images, and obtain preprocessed image data and marked image data;

[0007] (2) Performing data enhancement on the preprocessed image data and the annotated image data to obtain a pine wood nematode diseased tree dataset, and dividing the pine wood nematode diseased tree dataset into a training set, a validation set, and a test set;

[0008] (3) Improving the U-Net network model to obtain an improved U-Net network model;

[0009] (4) Input the training set into the improved U-Net network model for training;

[0010] (5) The image to be recognized is input into the improved U-Net network model, and the improved U-Net network model performs recognition to obtain a recognition result.

[0011] The step (1) specifically refers to: acquiring forest area images through a drone, preprocessing the acquired forest area images, wherein the preprocessing includes image denoising, image registration and radiation correction, marking pine wood nematode diseased trees in the forest area images, generating marked images, and adjusting the preprocessed image data and the corresponding marked image data to a pixel size of 256×256.

[0012] The step (2) specifically refers to: performing data enhancement operations of horizontal flipping, vertical flipping, and rotating 45 degrees on the preprocessed image data and the annotated image data to form a remote pine wood nematode disease-infected tree data set, and dividing the pine wood nematode disease-infected tree data set into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0013] The step (3) comprises the following steps in order:

[0014] (2a) The U-Net network model consists of an encoder, a bottleneck layer, a decoder and a skip connection module; the encoder is a convolutional encoder, on the basis of which a Swin Transformer encoder and a feature fusion module are added, the convolutional encoder, the Swin Transformer encoder and the feature fusion module together form a parallel dual encoder, the convolutional encoder is responsible for extracting local features, and the Swin Transformer encoder is responsible for extracting global features;

[0015] (2b) The bottleneck layer is replaced by a dilated spatial pyramid pooling module; the dilated spatial pyramid pooling module includes five branches, the first branch is a 1×1 convolution; the second, third and fourth branches are a 3×3 dilated convolution with a dilation rate of 3, a 3×3 dilated convolution with a dilation rate of 6, and a 3×3 dilated convolution with a dilation rate of 12, respectively; the fifth branch is global average pooling.

[0016] The step (4) specifically refers to: during the training process, the optimizer used is a stochastic gradient descent optimizer, the initial learning rate is 0.01, the learning rate is adjusted using a polynomial decay strategy, the momentum is 0.9, the weight decay is 1e-4, the rounds of the training model are set to 300, the batch size is set to 12, and the loss function used is a focal loss function. The formula of the focal loss function FL(pt) is:

[0017] FL(pt)=-αt(1-pt) γ log(pt)

[0018] In the formula, pt is the model's predicted probability for the correct category; for positive samples, pt = p, for negative samples, pt = 1-p, p is the predicted probability, αt is a balancing factor used to control the importance of positive and negative samples; γ is a regulating factor.

[0019] In step (2a), the convolution encoder has three layers, each of which consists of two layers of 3×3 convolution and one layer of 2×2 maximum pooling. The Swin Transformer encoder also has three layers. The input image is first segmented into 4×4 blocks by the image block segmentation (PatchPartition) module, and then enters the first layer of the Swin Transformer encoder, which consists of a linear embedding layer and two layers of stacked Swin Transformer blocks. The linear embedding layer projects the feature dimension to 64, and the Swin Transformer block performs global feature extraction. The second and third layers of the Swin Transformer encoder are both composed of two layers of stacked Swin Transformer blocks and an image block merging layer. The image block merging layer is responsible for downsampling and dimensionality increase. The Swin The Transformer block is used for global feature extraction; the feature fusion module is used to fuse local and global features. The global features extracted by the SwinTransformer encoder are upsampled and then concatenated with the local features extracted by the convolutional encoder, and then the number of channels is modified using 1×1 convolution; the output of the first layer of the convolutional encoder enters the input of the second layer of the convolutional encoder, and the output of the second layer of the convolutional encoder enters the input of the third layer of the convolutional encoder; the output of the first layer of the Swin Transformer encoder enters the input of the second layer of the Swin Transformer encoder, and the output of the second layer of the Swin Transformer encoder enters the input of the third layer of the Swin Transformer encoder; the output of the third layer of the convolutional encoder and the output of the third layer of the Swin Transformer encoder enter the bottleneck layer after feature fusion. At the same time, the convolutional encoder and Swin Transformer encoder of each layer are jump-connected with the corresponding decoder after feature fusion.

[0020] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention is improved on the basis of the U-Net network model, and makes full use of the Swin Transformer encoder to make up for the defects of the U-Net network model itself in capturing long-distance dependencies, thereby improving the accuracy of identifying pine wood nematode disease-susceptible and diseased trees; second, the present invention replaces the bottleneck layer of the U-Net network model with a void space pyramid pooling module, which fully captures image features of different scales, increases the receptive field, makes full use of contextual information, reduces feature information loss, captures more valuable information, and improves the accuracy of identifying pine wood nematode disease-susceptible and diseased trees. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of the method of the present invention;

[0022] Figure 2 It is a structural diagram of the improved U-Net network model;

[0023] Figure 3 It is a schematic diagram of the structure of the dilated space pyramid pooling module;

[0024] Figure 4 It is a schematic diagram of the test results. DETAILED DESCRIPTION

[0025] like Figure 1 As shown, a method for identifying pine wood nematode-infected trees using drone images based on an improved U-Net comprises the following steps in order:

[0026] (1) Obtain forest area images through drones and preprocess them, mark pine wood nematode-infected trees in the forest area images, and obtain preprocessed image data and marked image data;

[0027] (2) Performing data enhancement on the preprocessed image data and the annotated image data to obtain a pine wood nematode diseased tree dataset, and dividing the pine wood nematode diseased tree dataset into a training set, a validation set, and a test set;

[0028] (3) Improving the U-Net network model to obtain an improved U-Net network model;

[0029] (4) Input the training set into the improved U-Net network model for training;

[0030] (5) The image to be recognized is input into the improved U-Net network model, and the improved U-Net network model performs recognition to obtain a recognition result.

[0031] The step (1) specifically refers to: obtaining forest area images through drones, preprocessing the obtained forest area images to improve image data and enhance image features. The preprocessing includes image denoising, image registration and radiation correction, marking pine wood nematode diseased trees in the forest area images, generating marked images, and adjusting the preprocessed image data and the corresponding marked image data to a pixel size of 256×256.

[0032] The step (2) specifically refers to: performing data enhancement operations of horizontal flipping, vertical flipping, and rotating 45 degrees on the preprocessed image data and the annotated image data to form a remote pine wood nematode disease-infected tree data set, and dividing the pine wood nematode disease-infected tree data set into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0033] like Figure 2 As shown, step (3) includes the following steps in order:

[0034] (2a) The U-Net network model consists of an encoder, a bottleneck layer, a decoder and a skip connection module; the encoder is a convolutional encoder, on the basis of which a Swin Transformer encoder and a feature fusion module are added, the convolutional encoder, the Swin Transformer encoder and the feature fusion module together form a parallel dual encoder, the convolutional encoder is responsible for extracting local features, and the Swin Transformer encoder is responsible for extracting global features;

[0035] (2b) Replace the bottleneck layer with a dilated spatial pyramid pooling module; Figure 3 As shown in the figure, the dilated spatial pyramid pooling module includes 5 branches. The first branch is a 1×1 convolution; the second, third and fourth branches are 3×3 dilated convolutions with a dilation rate of 3, 3×3 dilated convolutions with a dilation rate of 6, and 3×3 dilated convolutions with a dilation rate of 12, respectively; the fifth branch is global average pooling.

[0036] The step (4) specifically refers to: during the training process, the optimizer used is a stochastic gradient descent optimizer, the initial learning rate is 0.01, the learning rate is adjusted using a polynomial decay strategy, the momentum is 0.9, the weight decay is 1e-4, the rounds of the training model are set to 300, the batch size is set to 12, and the loss function used is a focal loss function. The formula of the focal loss function FL(pt) is:

[0037] FL(pt)=-αt(1-pt) γ log(pt)

[0038] In the formula, pt is the model's predicted probability for the correct category; for positive samples, pt = p, for negative samples, pt = 1-p, p is the predicted probability, αt is a balancing factor used to control the importance of positive and negative samples; γ is a regulating factor.

[0039] In step (2a), the convolution encoder has three layers, each of which consists of two layers of 3×3 convolution and one layer of 2×2 maximum pooling. The Swin Transformer encoder also has three layers. The input image is first segmented into 4×4 blocks by the image block segmentation (PatchPartition) module, and then enters the first layer of the Swin Transformer encoder, which consists of a linear embedding layer and two layers of stacked Swin Transformer blocks. The linear embedding layer projects the feature dimension to 64, and the Swin Transformer block performs global feature extraction. The second and third layers of the Swin Transformer encoder are both composed of two layers of stacked Swin Transformer blocks and an image block merging layer. The image block merging layer is responsible for downsampling and dimensionality increase. The Swin The Transformer block is used for global feature extraction; the feature fusion module is used to fuse local and global features. The global features extracted by the SwinTransformer encoder are upsampled and then concatenated with the local features extracted by the convolutional encoder, and then the number of channels is modified using 1×1 convolution; the output of the first layer of the convolutional encoder enters the input of the second layer of the convolutional encoder, and the output of the second layer of the convolutional encoder enters the input of the third layer of the convolutional encoder; the output of the first layer of the Swin Transformer encoder enters the input of the second layer of the Swin Transformer encoder, and the output of the second layer of the Swin Transformer encoder enters the input of the third layer of the Swin Transformer encoder; the output of the third layer of the convolutional encoder and the output of the third layer of the Swin Transformer encoder enter the bottleneck layer after feature fusion. At the same time, the convolutional encoder and Swin Transformer encoder of each layer are jump-connected with the corresponding decoder after feature fusion.

[0040] like Figure 4 As shown in the figure, from left to right, the first column is the drone images of different distributions of pine wilt diseased trees, the second column is the ground truth, and the real labels of pine wilt diseased trees. The third to sixth columns are the improved U-Net, U-Net 、 Recognition results of DeepLabV3+ and FCN network models. Figure 4The part marked with red circles is the case of missed detection. It can be seen that the improved U-Net network model has more accurate and clearer target contours than other models, and the segmentation effect of the model is closer to the real label. It can accurately segment more delicate diseased wood targets, significantly improving the problem of missed detection of small targets, proving the superiority of the model proposed in the present invention.

[0041] In summary, the present invention makes improvements on the basis of the U-Net network model, makes full use of the SwinTransformer encoder to make up for the defects of the U-Net network model itself in capturing long-distance dependencies, and improves the accuracy of identifying pine wood nematode disease-susceptible and diseased trees; the present invention replaces the bottleneck layer of the U-Net network model with a void space pyramid pooling module, fully captures image features of different scales, increases the receptive field, makes full use of contextual information, reduces feature information loss, captures more valuable information, and improves the accuracy of identifying pine wood nematode disease-susceptible and diseased trees.

Claims

1. A method for identifying pine wood nematode diseased trees using drone images based on improved U-Net, characterized by: The method comprises the following steps in order: (1) Obtain forest area images through drones and preprocess them, mark pine wood nematode-infected trees in the forest area images, and obtain preprocessed image data and marked image data; (2) Performing data enhancement on the preprocessed image data and the annotated image data to obtain a pine wood nematode diseased tree dataset, and dividing the pine wood nematode diseased tree dataset into a training set, a validation set, and a test set; (3) Improving the U-Net network model to obtain an improved U-Net network model; (4) Input the training set into the improved U-Net network model for training; (5) The image to be recognized is input into the improved U-Net network model, and the improved U-Net network model performs recognition to obtain a recognition result.

2. The method for identifying pine wood nematode diseased trees using drone images based on improved U-Net according to claim 1, characterized in that: The step (1) specifically refers to: acquiring forest area images through a drone, preprocessing the acquired forest area images, wherein the preprocessing includes image denoising, image registration and radiation correction, marking pine wood nematode diseased trees in the forest area images, generating marked images, and adjusting the preprocessed image data and the corresponding marked image data to a pixel size of 256×256.

3. The method for identifying pine wood nematode diseased trees using drone images based on improved U-Net according to claim 1, characterized in that: The step (2) specifically refers to: performing data enhancement operations of horizontal flipping, vertical flipping, and rotating 45 degrees on the preprocessed image data and the annotated image data to form a remote pine wood nematode disease-infected tree data set, and dividing the pine wood nematode disease-infected tree data set into a training set, a validation set, and a test set in a ratio of 6:2:

2.

4. The method for identifying pine wood nematode diseased trees using drone images based on improved U-Net according to claim 1, characterized in that: The step (3) comprises the following steps in order: (2a) The U-Net network model consists of an encoder, a bottleneck layer, a decoder and a skip connection module; the encoder is a convolutional encoder, on the basis of which a Swin Transformer encoder and a feature fusion module are added, the convolutional encoder, the Swin Transformer encoder and the feature fusion module together form a parallel dual encoder, the convolutional encoder is responsible for extracting local features, and the Swin Transformer encoder is responsible for extracting global features; (2b) The bottleneck layer is replaced by a dilated spatial pyramid pooling module; the dilated spatial pyramid pooling module includes five branches, the first branch is a 1×1 convolution; the second, third and fourth branches are a 3×3 dilated convolution with a dilation rate of 3, a 3×3 dilated convolution with a dilation rate of 6, and a 3×3 dilated convolution with a dilation rate of 12, respectively; the fifth branch is global average pooling.

5. The method for identifying pine wood nematode diseased trees using drone images based on improved U-Net according to claim 1, characterized in that: The step (4) specifically refers to: during the training process, the optimizer used is a stochastic gradient descent optimizer, the initial learning rate is 0.01, the learning rate is adjusted using a polynomial decay strategy, the momentum is 0.9, the weight decay is 1e-4, the rounds of the training model are set to 300, the batch size is set to 12, and the loss function used is a focal loss function. The formula of the focal loss function FL(pt) is: FL(pt)=-αt(1-pt) γ log(pt) In the formula, pt is the model's predicted probability for the correct category; for positive samples, pt = p, for negative samples, pt = 1-p, p is the predicted probability, αt is a balancing factor used to control the importance of positive and negative samples; γ is a regulating factor.

6. The method for identifying pine wood nematode diseased trees using drone images based on improved U-Net according to claim 4, characterized in that: In step (2a), the convolution encoder has three layers, each of which consists of two layers of 3×3 convolution and one layer of 2×2 maximum pooling. The Swin Transformer encoder also has three layers. The input image is first segmented into 4×4 blocks by the image block segmentation (PatchPartition) module, and then enters the first layer of the Swin Transformer encoder, which consists of a linear embedding layer and two stacked Swin Transformer blocks. The linear embedding layer projects the feature dimension to 64, and the Swin Transformer block performs global feature extraction. The second and third layers of the Swin Transformer encoder are composed of two stacked Swin Transformer blocks and an image block merging layer. The image block merging layer is responsible for downsampling and dimensionality increase, and the Swin Transformer block is used for global feature extraction; the feature fusion module is used to fuse local and global features. The global features extracted by the Swin Transformer encoder are upsampled and then concatenated with the local features extracted by the convolutional encoder, and then the number of channels is modified using 1×1 convolution; the output of the first layer of the convolutional encoder enters the input of the second layer of the convolutional encoder, and the output of the second layer of the convolutional encoder enters the input of the third layer of the convolutional encoder; the output of the first layer of the Swin Transformer encoder enters the input of the second layer of the Swin Transformer encoder, and the output of the second layer of the Swin Transformer encoder enters the input of the third layer of the Swin Transformer encoder; the third layer of the convolutional encoder The output of the layer and the output of the third layer of the Swin Transformer encoder enter the bottleneck layer after feature fusion. At the same time, the convolutional encoder and Swin Transformer encoder of each layer are fused with features. Make a jump connection with the corresponding decoder.

Images