A detection method based on small-scale pine wood nematode disease trees

Abstract

A detection method based on small-scale pine wilt diseased trees, comprising the following steps: Step 1: Use drone aerial photography to obtain remote sensing images, mark pine wilt diseased trees, and make a data set; Step 2: Input the data set into the underlying feature batch fusion and multiple feature reuse network model, perform feature extraction, obtain a recognition model, and identify pine wilt diseased trees; Step 3: Vectorize the recognition results of pine wilt diseased trees to obtain the longitude and latitude coordinate files of pine wilt diseased trees; Step 4: Upload the longitude and latitude coordinate information of pine wilt diseased trees to the pine wilt diseased tree supervision platform, and check the geographical distribution of diseased trees through the supervision platform, and manually cut them down. The purpose of the patent of this invention is to solve the problem of missed detection of small-scale pine wilt diseased trees under the complex background of remote sensing images, and propose a detection method for pine wilt diseased trees based on underlying feature batch fusion and multiple feature reuse network models.

Description

Technical Field

[0001] The invention relates to the technical field of pine wood nematode diseased tree detection, and in particular to a detection method based on small-scale pine wood nematode diseased trees. Background Art

[0002] Forest pests and diseases are the old enemies of forest health and forestry production. Pine wood nematodes are one of the most harmful invasive species to pine trees. After infection with pine wood nematode disease, the pine needles will turn yellow-brown or reddish-brown, and the entire tree will dry up and die within 6 months. It reproduces quickly, spreads widely, and is difficult to prevent and control. It is an important pest that destroys large areas of pine forests. Therefore, it is crucial to detect pine wood nematode diseased trees in a timely manner.

[0003] At present, the main means of monitoring pine wilt disease are ground survey, satellite remote sensing monitoring, multispectral drone imagery and drone remote sensing monitoring. The traditional survey method of pine wilt disease is low efficiency, high cost and easy to cause omissions through manual census; satellite remote sensing has low spatial resolution and poor timeliness; multispectral drone image acquisition efficiency is low and the resolution is high. The acquisition of full-color band images by drone is currently the most effective. The full-color band images of drones can not only obtain images with higher spatial resolution, but also ensure the efficiency of image acquisition. By extracting the color, shape, texture and other features of individual diseased trees on high spatial resolution images, high detection accuracy can be obtained through deep learning to meet the needs of governance.

[0004] Since AlexNet proposed by Hinton et al. won the ImageNet Large Scale Visual Recognition Challenge in 2012, deep learning has developed rapidly and has been used more and more widely in the field of computer vision. More and more scholars are applying deep learning to target detection in remote sensing scenarios. Wang et al. proposed an improved SSD algorithm combined with feature pyramid (FPN) for small target detection in remote sensing images. This algorithm has improved the speed and accuracy of detection to a certain extent. Yao Qunli et al. proposed a multi-scale network remote sensing target detection framework-MSCNN. This algorithm has good results for multi-scale remote sensing large-scale targets, but poor results for small-scale target detection under multi-scale.

[0005] In order to achieve efficient detection of small remote sensing targets under complex backgrounds, this patent proposes a method for detecting pine wood nematode diseased trees using a batch fusion of underlying features and a multiple feature reuse network model. This method considers and redesigns the feature fusion module, integrating the shallow feature map that is mainly responsible for small target detection with the deep feature map with rich semantic information. It uses a batch modular feature fusion method to enhance and reuse shallow feature information, improves the feature extraction capability for small-scale targets, and adopts the idea of ​​transfer learning. First, the model is trained on relatively small small-scale diseased tree samples at low resolution, and then the trained model is used as a pre-trained model for training a higher resolution 1:500 data set, thereby improving the network model's ability to detect small-scale target diseased trees. Summary of the invention

[0006] The purpose of this patent is to solve the problem of missed detection of small-scale pine wilt diseased trees in the complex background of remote sensing images, and propose a detection method for pine wilt diseased trees based on underlying feature batch fusion and multiple feature multiplexing network model.

[0007] A detection method based on small-scale pine wood nematode diseased trees specifically comprises the following steps:

[0008] Step 1: Obtain images and mark the trees infected with pine wilt to create a data set;

[0009] Step 2: Input the data set into the underlying feature batch fusion and multiple feature reuse network model to extract features, obtain the recognition model, and identify the pine wood nematode diseased trees;

[0010] Step 3: vectorize the identification results of the pine wilt diseased trees to obtain the longitude and latitude coordinate information of the pine wilt diseased trees;

[0011] Step 4: Upload the latitude and longitude coordinate information of the pine wilt diseased trees to the pine wilt diseased tree supervision platform, check the geographical distribution of the diseased trees through the supervision platform, and cut them down manually.

[0012] In step 2, the data set is input into the underlying feature batch fusion and multiple feature reuse network model to extract features and obtain a recognition model. Specifically, the following steps are used:

[0013] 2-1: Select and preprocess the drone image sample set;

[0014] 2-2: Classify the dataset according to diseased tree images of different scales and perform migration training on the network model separately;

[0015] 2-3: Input the data set into the underlying feature batch fusion and multiple feature multiplexing network model to extract multi-feature information, obtain the pine wood nematode disease tree recognition model, and identify the pine wood nematode diseased trees.

[0016] In step 2-2, the data sets are classified according to different scales and the migration training network models are performed separately, which specifically includes the following sub-steps:

[0017] 2-2-1: In the data set samples with scale value A, the scale of the diseased trees is relatively small, which is conducive to the model's feature extraction of small-scale diseased trees. First, the data set with scale value A is input into the network model, and the pine wood nematode disease tree recognition model is trained;

[0018] 2-2-2: Then input the data set samples with scale value B into the network model, and use the recognition model obtained by training with scale value A as the pre-trained model trained with the data set samples with scale value B for migration training to obtain the final pine wood nematode disease tree recognition model.

[0019] In step 2-3, the underlying feature batch fusion and multiple feature reuse network model is constructed, which specifically includes the following sub-steps:

[0020] 2-3-1: Input the image to be detected into the feature extraction backbone network, and extract features from shallow to deep layers in a bottom-up manner, as follows:

[0021] Downsample the input image through convolution to obtain feature map L1;

[0022] The feature map L1 is subjected to a maximum pooling downsampling operation through convolution, and then subjected to multiple convolution residual block operations in sequence to obtain the feature map L2; the feature map L2 is subjected to multiple convolution residual block operations to obtain the feature map L3; the feature map L3 is subjected to multiple convolution residual block operations to obtain the feature map L4; the feature map L4 is subjected to multiple convolution residual block operations to obtain the feature map L5.

[0023] 2-3-2: The underlying feature batch processing is fused and the deep feature is reused to enhance the model's feature extraction ability for pine wilt diseased trees of different scales, and then the pine wilt diseased trees are identified. The specific steps are as follows:

[0024] The bottom feature maps L1, L2, and L3 are convolved to make their number of channels the same, and then feature fusion is performed to obtain feature map F2;

[0025] Use a 3*3 convolution kernel and a step size of 1 to perform a convolution operation on the feature map F2 to eliminate the aliasing effect after fusion and obtain the final prediction feature map Ps1 for detecting small-scale diseased trees;

[0026] The deep feature maps L4 and L5 are subjected to convolution operations so that the number of channels of the feature maps L4 and L5 is the same as the number of channels of the feature map L3; then feature fusion is performed to obtain the feature map F3;

[0027] The feature map F3 is convolved with a 3*3 convolution kernel and a step size of 1 to eliminate the aliasing effect after fusion and obtain the final predicted feature map Ps2, which is used to detect small-scale diseased trees that are not detected by the predicted feature map Ps1.

[0028] The feature map L4 is merged with the feature map F2 that has been downsampled and the feature map L5 that has been upsampled to obtain the feature map F4;

[0029] The feature map F4 is convolved with a 3*3 convolution kernel and a step size of 1 to eliminate the aliasing effect after fusion and obtain the predicted feature map Pm, which is used to detect medium-scale diseased trees.

[0030] The feature map L5 is fused with the feature map F2 that has been downsampled to obtain the feature map F5, and the feature map F5 is convolved to obtain the predicted feature map P1, which is used to detect large-scale diseased trees;

[0031] In step 2, the loss function used in the underlying feature batch fusion and multiple feature reuse network model is the focal loss function, as follows:

[0032]

[0033] The meanings of each parameter are as follows: y is the true label value (positive sample value is 1, negative sample value is 0); y′ is the predicted category probability given by the model; γ is a hyperparameter, γ>0 reduces the loss of easy-to-classify samples and makes the model pay more attention to difficult and misclassified samples. Changing the size of γ can also adjust the rate at which the weight of simple samples decreases. When γ is 0, it is the cross entropy loss function. When γ increases, the influence of the adjustment factor also increases. This γ will automatically reduce the weight of the contribution of simple samples (samples with clearer categories) to the loss during training, and quickly make the model pay more attention to difficult samples (samples with difficult categories to distinguish). Here, the optimal value for γ is 2; α is a balancing factor hyperparameter used to balance the uneven proportion of positive and negative samples. Here, the value is 0.25.

[0034] In step 4, the following sub-steps are specifically included:

[0035] 4-1: Import the latitude and longitude coordinate information of the identified diseased trees into the pine wood nematode diseased tree supervision platform;

[0036] 4-2: The geographical distribution of diseased trees can be viewed on the pine wilt diseased tree supervision platform. According to the diseased tree navigation information on the supervision platform, the diseased trees can be found, and the diseased pine trees can be surveyed and manually cut down.

[0037] Compared with the prior art, the present invention has the following technical effects:

[0038] The present invention is a detection method based on small-scale pine wilt diseased trees. Currently, there is only detection of overall pine wilt diseased trees, and there is no research on small-scale diseased tree detection. In the complex background of remote sensing images, there is a situation where small-scale pine wilt diseased trees are missed. Therefore, a detection method based on a network model of underlying feature batch fusion and multiple feature reuse is proposed. The method designs an underlying feature batch fusion module, performs adjacent feature fusion on the shallow feature map mainly responsible for small target detection, and at the same time fuses the deep features with rich semantic information with the shallow features, thereby improving the model's feature extraction capability for small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0040] Figure 1 is a flow chart of the present invention;

[0041] Figure 2 This is a network structure diagram of the present invention. DETAILED DESCRIPTION

[0042] like Figure 1 As shown, a detection method based on small-scale pine wood nematode diseased trees includes the following steps:

[0043] Step 1: Use drones to obtain remote sensing images, use the labelimg tool to mark pine wood nematode trees, and create a data set;

[0044] Step 2: Input the data set into the underlying feature batch fusion and multiple feature reuse network model to extract features, obtain the recognition model, and identify the pine wood nematode diseased trees;

[0045] Step 3: Vectorize the identification results of the pine wilt diseased trees to obtain the longitude and latitude coordinate files of the pine wilt diseased trees;

[0046] Step 4: Upload the latitude and longitude coordinate information of the pine wilt diseased trees to the pine wilt diseased tree supervision platform, check the geographical distribution of the diseased trees through the supervision platform, and cut them down manually.

[0047] In step 1, use drone aerial photography to obtain remote sensing images, use labelimg tool to mark diseased trees in remote sensing images, and make a data set. The specific steps are as follows:

[0048] (1) Use DJI drones to obtain clear 1:1000 and 1:500 images of pine trees in September, the peak period of pine wilt disease. Crop the 1:500 high-definition pine tree images to 1000*1000 and 1:1000 pine tree images to 1500*1500, and use the labelimg tool to mark the diseased trees in the images.

[0049] (2) The labeled samples are divided into two data sets with different scales of 1:500 and 1:1000.

[0050] like Figure 2 As shown, in step 2, the data set is input into the underlying feature batch fusion and multiple feature reuse network model to extract features and obtain a recognition model, which specifically includes the following steps:

[0051] (1) The input image is downsampled by a 7*7 convolution with a step size of 2 to obtain feature map L1; feature map L1 is downsampled by a 3*3 convolution with a step size of 2, and then it is sequentially passed through three 1*1, 3*3, and 1*1 convolution residual blocks to obtain feature map L2; feature map L2 is passed through four convolution residual blocks to obtain feature map L3; feature map L3 is passed through six convolution residual blocks to obtain feature map L4; feature map L4 is passed through three convolution residual blocks to obtain feature map L5.

[0052] The bottom feature maps L1, L2, and L3 are convolved by 1*1 to make their channels the same, and then feature fusion is performed to obtain feature map F2. A 3*3 convolution operation is performed on feature map F2 to obtain the final prediction feature map Ps1, which is responsible for detecting small-scale diseased trees.

[0053] The deep feature maps L4 and L5 are convolved with 1*1 to make the number of channels of feature maps L4 and L5 the same as that of feature map L3. Then feature fusion is performed to obtain feature map F3. A 3*3 convolution operation is performed on feature map F3 to obtain the final prediction feature map Ps2, which is responsible for detecting small-scale diseased trees missed by feature map Ps1.

[0054] The feature map L4 is fused with the feature map F2 which is a fusion of the underlying feature maps L1, L2 and L3, and the feature map L5 which is an upsampling operation, to obtain the feature map F4. A 3*3 convolution operation is performed on the feature map F4 to obtain the predicted feature map Pm, which is responsible for detecting medium-scale diseased trees.

[0055] The feature map L5 is fused with the down-sampled feature map F2 to obtain the feature map F5. A 3*3 convolution operation is performed on the feature map F5 to obtain the predicted feature map Pl, which is responsible for detecting large-scale diseased trees.

[0056] The traditional way of predicting small-scale targets is to output the prediction results of small-scale targets in the third layer. However, the results of the first and second convolution layers are also helpful for the fusion of small-scale features, and the second layer has more small-scale target information. Therefore, the features of the first and third layers are fused and added to the second layer to make a prediction of small-scale targets, which enhances the small-scale feature extraction capability.

[0057] On the fourth layer of the deep feature map, the traditional feature fusion method is to directly fuse the upsampled feature map of the fifth layer through the convolution layer, which loses the underlying geometric information. The present invention fuses the underlying information of the first, second and third layers on the basis of the existing fusion, so that the fourth layer of the deep feature map adds the underlying geometric information and enhances the feature extraction capability.

[0058] On the fifth layer of the deep feature map, the traditional method is to directly obtain the prediction map through convolution operation on the fifth layer feature map, while the present invention also integrates the shallow layer features into the fifth layer feature map, thereby improving the feature extraction capability of the model;

[0059] (2) Input the samples of the data set into the constructed network model and perform repeated training. Each time, the samples that are misidentified are removed and used as negative samples to be input into the network model for further training. During the recognition process, due to the complex terrain of the mountain forest, there will be other red car heads, roofs, bare land and pine-like trees that interfere with the detection of pine wilt diseased trees. Each time during the recognition process, the samples that are misidentified need to be removed and re-labeled as negative samples, put into the data set, and then trained again until a stable and relatively high-precision model recognition rate is achieved.

[0060] (3) In the 1:1000 data sample, the scale of diseased trees is relatively small, which is conducive to the identification of small-scale diseased trees. The 1:1000 data set is first input into the network model to train the pine wood nematode diseased tree identification model. The model obtained by the 1:1000 training is then used as the pre-training model for the 1:500 data sample training to obtain the final recognition model.

[0061] (4) The loss function used in the network model is focal loss, which is as follows:

[0062] The focal loss is a modification of the cross entropy loss function. The binary cross entropy loss is as follows:

[0063]

[0064] Among them, y is the true label value (the true sample value is 1, and the negative sample value is 0), y′ is the predicted category probability given by the model, and the cross entropy loss is that for positive samples, the greater the output probability, the smaller the loss. For negative samples, the smaller the output probability, the smaller the loss. At this time, the loss function is relatively slow in the iterative process of a large number of simple samples and may not be optimized to the optimal level.

[0065] The focal loss adds a factor to the original one, as shown below:

[0066]

[0067] Among them, γ is a hyperparameter. When γ>0, the loss of easy-to-classify samples is reduced, and the model is more focused on difficult and misclassified samples. Changing the size of γ can also adjust the rate at which the weight of simple samples is reduced. When γ is 0, it is the cross entropy loss function. When γ increases, the influence of the adjustment factor also increases. This γ will automatically reduce the weight of the contribution of simple samples (samples with clear categories) to the loss during training, and quickly make the model pay more attention to difficult samples (samples with difficult categories to distinguish). Here, the optimal value of γ is 2.

[0068] In addition, a balance factor α is added to balance the uneven ratio of positive and negative samples, as shown below:

[0069]

[0070] In steps 3 and 4, the identification results of pine wilt diseased trees are vectorized to obtain the latitude and longitude coordinate files of pine wilt diseased trees; the latitude and longitude coordinate information of pine wilt diseased trees is uploaded to the pine wilt diseased tree supervision platform, and the geographical distribution of diseased trees is checked through the supervision platform, and manual felling is carried out, which specifically includes the following steps:

[0071] (1) Input the remote sensing image of the pine forest that needs to be identified with coordinate information into the detection model to obtain the identification results of the pine wilt diseased trees and the location information of the diseased trees, and convert the identification results of the pine wilt diseased trees into longitude and latitude coordinate information. Import the longitude and latitude information of the pine wilt diseased trees into the ArcGis software to view the distribution of diseased trees for statistical analysis;

[0072] (2) Import the latitude and longitude coordinate information of the identified diseased trees into the pine wood nematode diseased tree supervision platform;

[0073] (3) The pine wilt diseased tree supervision platform can check the geographical distribution of diseased trees, find diseased trees based on the diseased tree navigation information on the supervision platform, and conduct inspections and cleanup of diseased pine trees.

[0074] The present invention designs the modular fusion of underlying feature batch processing and the reuse fusion of deep features, which fuses the shallow feature map mainly responsible for small target detection with the deep feature map with rich semantic information. The combination of underlying feature adjacency batch processing fusion and deep feature reuse fusion is adopted to enhance the shallow information and improve the model's ability to extract small target features.

[0075] At the same time, the present invention adopts a migration experiment method, first training small-scale diseased trees at low resolution, and then migrating to a data set at a higher resolution for training, so that the model can first learn the characteristics of small-scale diseased trees and improve the network model's detection ability for small-scale diseased trees.

Claims

1. A method for detecting small-scale pine wilt diseased trees, comprising the following steps: Step 1: Obtain images and mark the trees infected with pine wilt to create a data set; Step 2: Input the data set into the underlying feature batch fusion and multiple feature reuse network model to extract features, obtain the recognition model, and identify the pine wood nematode diseased trees; Step 3: vectorize the identification results of the pine wilt diseased trees to obtain the longitude and latitude coordinate information of the pine wilt diseased trees; Step 4: Upload the latitude and longitude coordinates of the pine wilt diseased trees to the pine wilt diseased tree supervision platform, check the geographical distribution of the diseased trees through the supervision platform, and cut them down manually; In step 2, the data set is input into the underlying feature batch fusion and multiple feature reuse network model to extract features and obtain a recognition model. Specifically, the following steps are used: 2-1: Select and preprocess the drone image sample set; 2-2: Classify the dataset according to the different scales of the diseased tree images and perform migration training on the network model separately; 2-3: Input the data set into the underlying feature batch fusion and multiple feature reuse network model to extract multi-feature information, obtain the pine wood nematode disease tree recognition model, and identify the pine wood nematode diseased trees; In step 2-2, the dataset is classified according to the different scales of the diseased tree images, and the migration training network model is performed separately, which specifically includes the following sub-steps: 2-2-1: In the data set samples with scale value A, the scale of the diseased trees is relatively small, which is conducive to the model's feature extraction of small-scale diseased trees. First, the data set with scale value A is input into the network model, and the pine wood nematode disease tree recognition model is trained; 2-2-2: Then, the data set samples with scale value B are input into the network model, and the network model obtained by training with scale value A is used as the pre-trained model trained with the data set samples with scale value B for migration training to obtain the final pine wood nematode disease tree recognition model; In step 2-3, the underlying feature batch fusion and multiple feature reuse network model is constructed, which specifically includes the following sub-steps: 2-3-1: Input the image to be detected into the feature extraction backbone network, and extract features from shallow to deep layers in a bottom-up manner, as follows: Downsample the input image through convolution to obtain feature map L1; The feature map L1 is convolved to perform a maximum pooling downsampling operation, and then sequentially passed through multiple convolution residual blocks to obtain the feature map L2; The feature map L2 is operated through multiple convolution residual blocks to obtain the feature map L3; the feature map L3 is operated through multiple convolution residual blocks to obtain the feature map L4; the feature map L4 is operated through multiple convolution residual blocks to obtain the feature map L5; 2-3-2: The underlying feature batch processing is fused and the deep feature is reused to enhance the model’s ability to extract features of pine wilt diseased trees of different scales, and then the pine wilt diseased trees are identified.

2. The method according to claim 1, characterized in that In step 2-3-2, the specific steps are as follows: The bottom feature maps L1, L2, and L3 are convolved to make their number of channels the same, and then feature fusion is performed to obtain feature map F2; Use convolution to perform convolution operation on the feature map F2 to eliminate the mixed superposition effect of the fused features and obtain the final prediction feature map Ps1, which is used to detect small-scale diseased trees; The deep feature maps L4 and L5 are subjected to convolution operations so that the number of channels of the feature maps L4 and L5 is the same as the number of channels of the feature map L3; then feature fusion is performed to obtain the feature map F3; Use convolution to perform convolution operation on feature map F3 to eliminate the mixed superposition effect of fused features, and obtain the final prediction feature map Ps2, which is used to detect small-scale diseased trees that are not detected by the prediction feature map Ps1; The feature map L4 is merged with the feature map F2 that has been downsampled and the feature map L5 that has been upsampled to obtain the feature map F4; Use convolution to perform convolution operation on the feature map F4 to eliminate the mixed superposition effect of the fused features and obtain the predicted feature map Pm, which is used to detect medium-scale diseased trees; The feature map L5 is fused with the feature map F2 that has undergone a downsampling operation to obtain the feature map F5, and a convolution operation is performed on the feature map F5 to obtain the predicted feature map P1, which is used to detect large-scale diseased trees.

3. The method according to claim 1, characterized in that Step 4 specifically includes the following sub-steps: 4-1: Import the latitude and longitude coordinate information of the identified diseased trees into the pine wood nematode diseased tree supervision platform; 4-2: The geographical distribution of diseased trees can be viewed on the pine wilt diseased tree supervision platform. According to the diseased tree navigation information on the supervision platform, the diseased trees can be found, and the diseased pine trees can be surveyed and manually cut down.

Images