Two-stage pine wood nematode disease epidemic tree general survey method based on aerial images

Abstract

The invention discloses a two-stage pine wood nematode disease epidemic tree general survey method based on an aerial image, and the method comprises the steps: collecting an image of a target mountain forest land through an unmanned plane, and carrying out the splicing, cutting and marking of the image, thereby obtaining an epidemic tree data set; an object detection model with a high recall rate is selected, the epidemic wood data set is trained, then the trained epidemic wood detection model is used to detect and identify all the segmented small images in the target mountain forest land, and all suspected epidemic wood positions and latitude and longitude information thereof are obtained; associating each suspected epidemic wood with a unique identifier, intercepting all suspected epidemic wood block diagrams, and carrying out epidemic wood sample classification; selecting an image classification model, performing model training by taking the obtained various block diagrams as a data set, performing classification, and reserving a final classification result as a block diagram of the epidemic wood; and extracting the unique identifiers of all the reserved epidemic wood block diagrams, obtaining the longitude and latitude coordinates of all the corresponding epidemic wood, and counting the final number and distribution of the epidemic wood in the target mountain forest land to complete the general survey of the pine wood nematode disease epidemic wood.

Description

Technical Field

[0001] The invention relates to a two-stage pine wood nematode diseased tree census method based on aerial images. Background Art

[0002] Pine wilt disease is a highly contagious forest disease, known as "pine cancer". When a pine tree is infected, its needles will gradually change color and wither, and it will die in as fast as 40 days. The current diseased tree management process mostly adopts the method of manual census and manual acceptance after the diseased trees are cleared. This method has the following problems: 1) It is difficult to master the full amount of data: In view of the complexity of mountain forest terrain and the vast area of ​​forest areas, manual census is not only difficult but also has a huge workload. It is difficult to achieve a comprehensive statistics of the number of diseased trees and accurate mapping of the distribution of diseased points in the epidemic area. This lack of data directly weakens the ability of management departments to formulate precise prevention and control strategies, which in turn affects the thoroughness and effectiveness of diseased tree removal, making it difficult to achieve the goal of "all that should be removed"; 2) It is difficult to clear and accept diseased trees: After the diseased trees are cleared, the forestry department needs to carry out extensive acceptance work. This process covers many points and a wide range. The traditional manual acceptance method not only consumes a lot of human resources and has low implementation efficiency, but also the acceptance results are easily interfered with by human factors, affecting the fairness and accuracy of the acceptance, and bringing uncertainty to the subsequent evaluation of the management effect.

[0003] Among the existing solutions, there are also methods of acquiring large-scale aerial images of target mountain forests from a bird's-eye view by using drones, splicing them into orthophotos with latitude and longitude coordinates, and then using a deep learning-based object detection model to identify and locate diseased trees in all orthophotos, and counting the number and distribution of diseased trees to complete a census of diseased trees with pine wood nematode disease in the target mountain forests. However, in actual pine wood nematode disease diseased trees census scenarios, the census scope is usually in counties (hundreds of square kilometers or thousands of square kilometers), and large-scale mountain forest target images need to be collected. Due to the limitations of the collection time window and the collection cost, the quality of the collected mountain forest images often cannot reach the ideal state. Although the current object detection model can better realize the accurate identification and positioning of diseased trees with pine wood nematode disease in high-quality mountain forest images, in the mountain forest images obtained in the large-scale actual diseased tree census scenarios, due to the weakening of the appearance features such as color and texture of the diseased trees, it is often difficult to effectively distinguish them from other objects with similar appearance in the mountain forest, and it is difficult to obtain a high recall rate and precision at the same time. Summary of the invention

[0004] The present invention aims to solve the problem that it is difficult to simultaneously obtain a high recall rate and precision in the existing technology for identifying and surveying diseased trees. To this end, the present invention proposes a two-stage survey method for diseased trees with pine wood nematodes based on aerial images.

[0005] A technical solution adopted by an embodiment of the present invention to solve the technical problem is: a two-stage pine wood nematode diseased tree survey method based on aerial images, comprising:

[0006] S1. Use a drone to collect images of the target mountain forest land in a bird's-eye view aerial photography mode, stitch the acquired aerial photos into a two-dimensional orthophoto with precise latitude and longitude coordinates, and evenly divide the orthophoto into small images of appropriate sizes;

[0007] S2. Divide the target mountain forest into different areas and select several representative small pictures from multiple areas as training and evaluation samples. Label the selected small pictures with samples of pine wood nematode infected trees to form an infected tree dataset. The infected tree dataset is divided into a training set, a validation set, and a test set.

[0008] S3, select a deep learning object detection model with a higher recall rate as the benchmark model, train, evaluate and optimize the object detection model with the diseased tree data set marked in step S2, use the trained diseased tree detection model to detect and identify all segmented small pictures in the target mountain forest, and obtain the locations of all suspected diseased trees and their latitude and longitude information;

[0009] S4, each suspected diseased tree is associated with a unique identifier, all suspected diseased tree frame images selected in step S3 are intercepted, each frame image is named with its unique identifier and saved as a picture file, and several suspected diseased tree frame image files are selected for diseased tree classification samples;

[0010] S5, select a refined deep learning image classification model, use the various frame diagrams obtained in step S4 as data sets for model training, and input all suspected diseased wood frame diagrams into the classification network model for classification, and only retain the frame diagrams whose final classification results are diseased wood;

[0011] S6. Extract the unique identifiers of all retained diseased tree frames in step S5, obtain the corresponding longitude and latitude coordinates of all diseased trees, count the final number and distribution of diseased trees in the target mountain forest, and complete the two-stage pine wood nematode disease diseased tree census.

[0012] As one of the preferred embodiments of the present invention, in step S2, the target mountain forest land is divided into different areas based on at least the weather conditions, sunlight intensity, light angle and / or diseased tree distribution density when the image is collected.

[0013] As one of the preferred embodiments of the present invention, in step S4, the diseased wood classification categories at least include normal pine trees, broad-leaved trees with similar colors to the diseased wood, other dead tree species, misidentified background images, and trees diseased by pine nematode.

[0014] Beneficial effects of the present invention: The present invention divides the identification and positioning process into two independent stages, each of which focuses on only one indicator, and can perform targeted model selection and strategy optimization for the single indicator:

[0015] 1. In the first stage of obtaining all suspected diseased trees, the diseased tree detection model only needs to consider how to improve the recall rate of diseased trees. When selecting a benchmark detection model, a model with a higher recall rate can be considered. Various strategies can also be used to improve the recall rate index during model training and diseased tree identification;

[0016] 2. In the second stage, the stage of eliminating misidentified diseased trees, the main consideration is to eliminate misidentified diseased trees. Various types of misidentification results can be effectively removed through steps such as the preparation of detailed diseased tree classification data sets and the selection of high-precision diseased tree classification models, and ultimately a high-accuracy census of pine wood nematode disease-infected trees in large-scale target areas will be completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0018] Figure 1 This is a flowchart of a two-stage pine wood nematode diseased tree census method based on aerial images;

[0019] Figure 2 A schematic diagram of the area where some orthophotos of mountains and forests are collected and spliced;

[0020] Figure 3 Annotate the schematic diagram for the diseased wood dataset;

[0021] Figure 4 This is a schematic diagram of the test results of suspected diseased trees;

[0022] Figure 5 This is a list of intercepted suspected epidemic wood frame pictures;

[0023] Figure 6 This is a schematic diagram of the classification samples of suspected infected trees. DETAILED DESCRIPTION

[0024] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0025] In the description of the present invention, the meaning of "above", "below", "exceed", etc. is not inclusive of the number itself, and the meaning of "above", "below", "within", etc. is inclusive of the number itself. If there is a description of "first" or "second", it is only used to distinguish the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0026] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0027] In the present invention, unless otherwise clearly defined, the words "set", "install", "connect" and the like should be understood in a broad sense, for example, they can be directly connected or indirectly connected through an intermediate medium; they can be fixedly connected or detachably connected or integrally formed; they can be mechanically connected; they can be the internal connection of two elements or the interaction relationship between two elements. Those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0028] Reference Figure 1 , a two-stage pine wood nematode-infected tree survey method based on aerial images, including:

[0029] S1. Use a drone to collect images of the target mountain forest land in a bird's-eye view aerial photography mode, stitch the acquired aerial photos into a two-dimensional orthophoto with precise latitude and longitude coordinates, and evenly divide the orthophoto into small images of appropriate sizes;

[0030] S2. Divide the target mountain forest into different areas and select several representative small pictures from multiple areas as training and evaluation samples. Label the selected small pictures with samples of pine wood nematode infected trees to form an infected tree dataset. The infected tree dataset is divided into a training set, a validation set, and a test set.

[0031] S3, select a deep learning object detection model with a higher recall rate as the benchmark model, train, evaluate and optimize the object detection model with the diseased tree data set marked in step S2, use the trained diseased tree detection model to detect and identify all segmented small pictures in the target mountain forest, and obtain the locations of all suspected diseased trees and their latitude and longitude information;

[0032] S4, each suspected diseased tree is associated with a unique identifier, all suspected diseased tree frame images selected in step S3 are intercepted, each frame image is named with its unique identifier and saved as a picture file, and several suspected diseased tree frame image files are selected for diseased tree classification samples;

[0033] S5, select a refined deep learning image classification model, use the various frame diagrams obtained in step S4 as data sets for model training, and input all suspected diseased wood frame diagrams into the classification network model for classification, and only retain the frame diagrams whose final classification results are diseased wood;

[0034] S6. Extract the unique identifiers of all retained diseased tree frames in step S5, obtain the corresponding longitude and latitude coordinates of all diseased trees, count the final number and distribution of diseased trees in the target mountain forest, and complete the two-stage pine wood nematode disease diseased tree census.

[0035] In the present invention:

[0036] 1) Target mountain forest image acquisition

[0037] Reference Figure 2 In the large-scale forest image acquisition, fixed-wing UAVs are selected, and the flight route is set according to the target forest land planning. The UAV is controlled to fly at a constant speed at a fixed altitude. The UAV is equipped with an image sensor device to take an original image of the forest area at a fixed time interval. The heading overlap rate between adjacent original films is set to 80%, and the lateral overlap rate between adjacent routes is set to 60%. The forest ground resolution is about 5cm-10cm; after obtaining all the original aerial photos, the image stitching software or stitching algorithm is used to reconstruct and stitch the two-dimensional images to generate a two-dimensional orthophoto of the target forest land. Each coordinate point in the orthophoto is accompanied by precise latitude and longitude coordinates. Due to the huge pixel value of the orthophoto, in order to facilitate subsequent sample annotation and model training recognition, it is evenly divided into small images of 768*768 pixels. Each small image is named in the form of "XXXX_YYYY.j pg", where XXXX is the row number of the small image in the orthophoto ti ff whole image, and YYYY is the row number of the small image in the orthophoto ti ff whole image. ff is the column number in the entire image, which is used to save the position information of each small image in the entire image.

[0038] 2) Labeling of diseased tree detection samples and data set creation

[0039] Reference Figure 3In order to make the selected diseased tree samples more extensive and representative, the target mountain forest was divided into different areas according to various factors such as weather conditions, sunshine intensity, light angle, and diseased tree distribution density during image acquisition, and a certain number of small images were randomly selected from each area as candidate sample images; the image annotation tool labe II mg was used to manually select all diseased trees in the candidate images as training and evaluation samples; in order to make the recognition results of the subsequent detection model have a higher recall rate, the diseased tree identification standard was lowered accordingly during sample annotation, and samples with suspected diseased trees were first selected, so that the samples contained diseased trees of various appearance types as much as possible; the number of annotated diseased tree detection images was greater than 5,000, and the number of diseased tree samples was greater than 10,000. All annotated diseased tree samples were divided into training set, validation set, and test set in a ratio of about 8:1:1.

[0040] 3) Training of diseased tree detection model and acquisition of suspected diseased tree information

[0041] Reference Figure 4 , because the accuracy requirement for diseased wood detection and identification in this scenario is higher than the detection and identification speed requirement, therefore, in the selection of diseased wood detection benchmark models, consider using a two-stage object detection model with higher accuracy, or a large-scale network structure in the YOLO series of models, such as YOLOv8x; in order to further increase the diversity of training samples, a variety of data augmentation strategies are adopted. In this scenario, the detection target object is only one type of object, pine wood nematode disease diseased wood, and the value of the last full-link layer of the benchmark detection model needs to be set to 1; all the segmented small images of the target mountain forest are input into the trained detection model for diseased wood Detection and identification, output all suspected diseased tree identification result information, including the identification frame containing the position and size information of the suspected diseased tree in the segmented small image (represented by the coordinates of the center point of the identification frame and the length and width information of the identification frame) and the confidence level. According to the row number and column number in the orthophoto contained in the segmented small image file name, the position of the center point of the identification frame in the small image is converted into the pixel position in the orthophoto, thereby obtaining the longitude and latitude coordinates of the center point of the identification frame of each suspected diseased tree. In order to maximize the recall rate, the confidence threshold is set to 0.01, that is, all suspected diseased tree information with a confidence level greater than 0.01 is saved.

[0042] 4) Capture and save the suspected epidemic wood frame

[0043] Reference Figure 5, extract all the suspected diseased tree related information preserved in the above step 3), obtain the center point position of the suspected diseased tree identification frame from the orthophoto according to the longitude and latitude information of the extracted suspected diseased tree, and intercept a square area image of a fixed size (such as 100*100 pixels) with this point as the center point to represent the suspected diseased tree; since the flight altitude of the drone is relatively fixed during aerial image acquisition, the sizes of the diseased trees in the collected images are different but the overall difference is not large, so the main content in each intercepted frame diagram is usually the suspected diseased tree; each suspected diseased tree frame diagram is associated with a unique identifier (UID) and saved as a picture file, and all suspected diseased tree frame diagram files are named with the unique identifier (UID) corresponding to the suspected diseased tree, so that the frame diagram is associated with the longitude and latitude coordinates and confidence of the corresponding suspected diseased tree.

[0044] 5) Sample selection and labeling of suspected diseased trees

[0045] Reference Figure 6 The diseased tree detection model can identify and locate most of the diseased trees in the target forest area with a high recall rate. However, due to the low recognition standard and low confidence threshold when selecting training sample images in the detection model, the output results will contain a large number of misidentifications. In order to improve the accuracy of the model, the second stage of this method screens out the misidentification results by inputting all suspected diseased tree frame maps into the refined classification model for further refined classification. Specifically, several images are randomly selected from the suspected diseased tree frame map as candidate classification samples, and the candidate samples are manually classified and labeled by relevant professionals with rich industry knowledge. All candidate samples are divided into five different categories: "normal pine trees", "broad-leaved forests with similar colors", "other dead tree species", "other background images that are misidentified" and "pine wilt diseased trees". There are about 5,000 sample images of "pine wilt diseased trees" in the final labeled frame map, and about 500-1,000 sample images of other categories.

[0046] 6) Diseased tree classification model training and non-disease tree removal

[0047] A refined deep learning image classification model is selected, and the various frame diagrams obtained in the above 5) are used as data sets for model training, and all the suspected diseased wood frame diagrams obtained in the above 4) are input into the trained classification network model for classification; after the classification is completed, the four types of frame diagrams output by the model, such as "normal pine trees", "broad-leaved forests with similar colors", "other dead tree species", and "other background images that are misidentified", are screened out, and only the frame diagrams whose final classification results are "pine wood nematode disease diseased trees" are retained.

[0048] 7) Obtaining the results of the diseased tree census

[0049] According to the classification results of 6) above, the UID corresponding to the box diagram of all classification categories "pine wilt diseased trees" is obtained, and according to the relationship between the U ID and the longitude and latitude coordinates saved in 4) above, the longitude and latitude coordinates of all corresponding diseased trees are obtained, and the final number and distribution of diseased trees in the target forest area are counted to complete the two-stage precise census of diseased trees in pine wilt disease.

[0050] Of course, the present invention is not limited to the above-mentioned embodiments, and those skilled in the art may make equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications and substitutions are all included in the scope defined by the claims of this application.

Claims

1. A two-stage pine wood nematode diseased tree survey method based on aerial images, characterized in that: include: S1. Use a drone to collect images of the target mountain forest land in a bird's-eye view aerial photography mode, stitch the acquired aerial photos into a two-dimensional orthophoto with precise latitude and longitude coordinates, and evenly divide the orthophoto into small images of appropriate sizes; S2. Divide the target mountain forest into different areas and select several representative small pictures from multiple areas as training and evaluation samples. Label the selected small pictures with samples of pine wood nematode infected trees to form an infected tree dataset. The infected tree dataset is divided into a training set, a validation set, and a test set. S3, select a deep learning object detection model with a higher recall rate as the benchmark model, train, evaluate and optimize the object detection model with the diseased tree data set marked in step S2, use the trained diseased tree detection model to detect and identify all segmented small pictures in the target mountain forest, and obtain the locations of all suspected diseased trees and their latitude and longitude information; S4, each suspected diseased tree is associated with a unique identifier, all suspected diseased tree frame images selected in step S3 are intercepted, each frame image is named with its unique identifier and saved as a picture file, and several suspected diseased tree frame image files are selected for diseased tree classification samples; S5, select a refined deep learning image classification model, use the various frame diagrams obtained in step S4 as data sets for model training, and input all suspected diseased wood frame diagrams into the classification network model for classification, and only retain the frame diagrams whose final classification results are diseased wood; S6. Extract the unique identifiers of all retained diseased tree frames in step S5, obtain the corresponding longitude and latitude coordinates of all diseased trees, count the final number and distribution of diseased trees in the target mountain forest, and complete the two-stage pine wood nematode disease diseased tree census.

2. The two-stage pine wood nematode-infected tree survey method based on aerial images according to claim 1 is characterized by: In step S2, the target mountain forest land is divided into different areas based on at least the weather conditions, sunshine intensity, light angle and / or diseased tree distribution density when the image is collected.

3. The two-stage pine wood nematode diseased tree survey method based on aerial images according to claim 1 is characterized by: In step S4, the diseased wood classification categories include at least normal pine trees, broad-leaved trees with similar colors to the diseased wood, other dead tree species, misidentified background images, and trees diseased by pine wilt.

Images