Automatic forest damage tree detection device, automatic forest damage tree detection method and program

Abstract

To recognize the position of a damaged tree present in a survey-target forest, by a simple method.SOLUTION: A device for automatically detecting a damaged forest tree receives the input of aerial image data and information specifying the detection range of damaged trees included in the aerial image, generates a grid within the specified range and cuts out a plurality of rectangular images from the aerial image, and detects the damaged tree included in the rectangular images. In the detection of the damaged tree, it is determined whether or not the rectangular images include the damaged tree, and the position information of the damaged tree is calculated. Supervised learning of an object detection model using training data is performed, and whether or not the rectangular images include the damage tree is determined using the learnt object detection model. The training data includes a training data aerial image including the damaged tree and labeling performed on a rectangle circumscribing the damaged tree. When it is determined that the cut out rectangular images includes the damaged tree, the object detection model outputs the rectangle circumscribing the damaged tree, and the position information of the damaged tree detected based on the rectangle is calculated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an automatic forest damaged tree detection device, a forest damaged tree automatic detection method, and a program. [Background technology]

[0002] In recent years, due in part to the effects of global warming, damage caused by pine wilt caused by the Japanese pine sawyer beetle and oak wilt caused by the long-horned oak beetle has been spreading nationwide. Damage is becoming more severe in Nagano Prefecture and the Tohoku region, which are located at high altitudes and latitudes. To prevent the damage from spreading, it is necessary to detect and treat affected trees early. Understanding the distribution of damage and accurately controlling infected trees at the forefront are effective ways to prevent the damage from spreading. For example, with regard to pine wilt damage, which is the largest forest pest in Japan, currently most of the damage is assessed by personnel through on-site surveys and visual inspections, which takes time and effort but is inaccurate. To address these issues, research has been conducted in recent years into analyzing pine wilt damage using remote sensing techniques such as drones and satellites.

[0003] For example, in the technology described in Patent Document 1, image data of red pine forests, which is created based on the spectral reflectance characteristics of tree species contained in image data obtained by photographing the area containing the red pine forests being surveyed from the air, is used, and the number of red pine trees contained in the red pine forests is calculated by damage classification.

[0004] On the other hand, there are few cases where damaged trees are analyzed using remote sensing, such as the technology described in Patent Document 1, and no method that is both simple and widespread has been established. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6544582 Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the above-mentioned problems, the present invention aims to provide an automatic forest damaged tree detection device, an automatic forest damaged tree detection method, and a program that can determine whether or not there are damaged trees in a forest area being surveyed and the location of damaged trees in the forest area being surveyed using a simple method. [Means for solving the problem]

[0007] One aspect of the present invention is an automatic forest damaged tree detection device that automatically detects damaged trees in a forest area under investigation, and includes a data input processing unit that accepts input of data on an aerial image of the forest area under investigation and detection range designation information that designates the range within which damaged trees included in the aerial image are to be detected, an image clipping processing unit that clips out a plurality of rectangular images from the aerial image by generating a grid within the range within which damaged trees are to be detected that is designated by the detection range designation information, and a damaged tree detection processing unit that detects damaged trees included in each of the plurality of rectangular images clipped out by the image clipping processing unit, and the damaged tree detection processing unit includes a damaged tree search processing unit that determines whether a damaged tree is included in each of the plurality of rectangular images clipped out by the image clipping processing unit, and a damaged tree position calculation processing unit that calculates position information of the damaged tree when the damaged tree search processing unit determines that a damaged tree is included in any of the plurality of rectangular images clipped out by the image clipping processing unit, and The unit performs supervised learning of an object detection model using training data, and uses the object detection model after supervised learning to determine whether or not a damaged tree is included in each of a plurality of rectangular images cut out by the image cutout processing unit. The training data used for supervised learning of the object detection model includes aerial images for training data that include damaged trees, and labeling applied to damaged tree circumscribing rectangles for training data that are rectangles circumscribing damaged trees included in the aerial images for training data. When the damaged tree search processing unit determines that a damaged tree is included in any of the plurality of rectangular images cut out by the image cutout processing unit, the object detection model outputs a damaged tree circumscribing rectangle to be detected that is a rectangle circumscribing a damaged tree included in any of the plurality of rectangular images cut out by the image cutout processing unit. The damaged tree position calculation processing unit calculates position information of the damaged tree detected by the damaged tree detection processing unit based on the damaged tree circumscribing rectangle to be detected output by the object detection model. death , The damaged tree position calculation processing unit calculates the center positions in the vertical and horizontal directions of the circumscribing rectangle of the detected damaged tree output by the object detection model as the position of the damaged tree detected by the damaged tree detection processing unit, and the image clipping processing unit clips a plurality of rectangular images from the aerial image so that a part of one of two rectangular images adjacent in the vertical direction overlaps with a part of the other, and a part of one of two rectangular images adjacent in the horizontal direction overlaps with a part of the other. This is an automatic detection device for damaged forest trees.

[0010] In one embodiment of the forest damaged tree automatic detection device of the present invention, the damaged tree detection processing unit is provided with a matching processing unit that matches the damaged tree based on the damaged tree's position information calculated by the damaged tree position calculation processing unit when the damaged tree search processing unit determines that a damaged tree is included in any of the multiple rectangular images cut out by the image cutout processing unit, and the matching processing unit may consider the damaged tree included in one of the two vertically adjacent rectangular images and the damaged tree included in the other of the two vertically adjacent rectangular images to be the same damaged tree if the distance between the position of the damaged tree included in one of the two vertically adjacent rectangular images and the position of the damaged tree included in the other of the two vertically adjacent rectangular images is within a first threshold, and may consider the damaged tree included in one of the two horizontally adjacent rectangular images and the damaged tree included in the other of the two horizontally adjacent rectangular images to be the same damaged tree if the distance between the position of the damaged tree included in one of the two horizontally adjacent rectangular images and the position of the damaged tree included in the other of the two horizontally adjacent rectangular images is within the first threshold.

[0011] In one embodiment of the automatic forest damaged tree detection device of the present invention, the vertical overlap width, which is the overlap width between a part of one of two adjacent rectangular images in the vertical direction and a part of the other, may be greater than the first threshold value, and the horizontal overlap width, which is the overlap width between a part of one of two adjacent rectangular images in the horizontal direction and a part of the other, may be greater than the first threshold value.

[0012] In one embodiment of the forest damaged tree automatic detection device of the present invention, the damaged tree search processing unit uses the object detection model after supervised learning to calculate the reliability that each of the multiple rectangular images cut out by the image cutout processing unit contains a damaged tree, and the damaged tree detection processing unit is equipped with a noise removal processing unit, and if the damaged tree search processing unit determines that one of the multiple rectangular images cut out by the image cutout processing unit contains a damaged tree and the reliability calculated by the damaged tree search processing unit is less than or equal to a second threshold, the noise removal processing unit may remove the damaged tree having a reliability less than or equal to the second threshold as noise.

[0013] In one aspect of the forest damaged tree automatic detection device of the present invention, the aerial image data received as input by the data input processing unit is orthoimage data generated from RGB color images in the visible light range captured using a drone, the damaged tree search processing unit determines whether or not a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit, and determines whether or not a dead or damaged tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit, the damaged tree position calculation processing unit calculates position information of the dead tree when the damaged tree search processing unit determines that a dead tree is included in any of the plurality of rectangular images cut out by the image cutout processing unit, and calculates position information of the dead or damaged tree when the damaged tree search processing unit determines that a dead or damaged tree is included in any of the plurality of rectangular images cut out by the image cutout processing unit, and the training data used for supervised learning of the object detection model includes the aerial images for training data that include dead trees and dead or damaged trees, or the aerial images for training data that include dead trees and the aerial images for training data that include dead trees and the aerial images for training data that include dead trees, and the aerial images for training data that include dead or damaged trees. The object detection model includes labeling applied to a dead tree circumscribing rectangle for training data, which is a rectangle circumscribing a dead tree included in the image, and labeling applied to a dead tree circumscribing rectangle for training data, which is a rectangle circumscribing a dead tree included in the aerial photograph image for training data, and when the damaged tree search processing unit determines that a dead tree and / or a dead tree is included in any of the multiple rectangular images cut out by the image cutout processing unit, the object detection model a dead tree circumscribing rectangle for a dead tree to be detected, which is a rectangle circumscribing a dead tree and / or a dead tree; the damaged tree position calculation processing unit calculates position information of the dead tree detected by the damaged tree detection processing unit based on the dead tree circumscribing rectangle for the dead tree to be detected output by the object detection model; and a ground height filter processing unit determines whether the detection result of the dead tree and / or dead tree by the damaged tree detection processing unit is a false detection;The ground height filter processing unit may acquire digital elevation model data of the forest area to be surveyed and digital surface model data of the forest area to be surveyed, and may determine whether the damaged tree detection processing unit has erroneously detected the ground surface as a dead tree and / or a dead tree based on the detection results of dead and / or damaged trees by the damaged tree detection processing unit, the digital elevation model data, and the digital surface model data.

[0014] In one aspect of the forest damaged tree automatic detection device of the present invention, the aerial image data received as input by the data input processing unit is orthoimage data generated from multispectral images taken by an artificial satellite or an aircraft, the damaged tree search processing unit executes the following: determining whether an infected tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; determining whether a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; and determining whether a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; the damaged tree position calculation processing unit calculates position information of the infected tree when the damaged tree search processing unit determines that an infected tree is included in any of the plurality of rectangular images cut out by the image cutout processing unit; calculating position information of the dead tree when the damaged tree search processing unit determines that a dead tree is included in any of the plurality of rectangular images cut out by the image cutout processing unit; and calculating the position information of the dead tree when the damaged tree search processing unit determines that a dead tree is included in any of the plurality of rectangular images cut out by the image cutout processing unit. When the search processing unit determines that the dead tree is infected, the location information of the dead tree is calculated. The training data used for supervised learning of the object detection model includes the training data aerial photograph images including infected trees, dead trees, and dead trees, labeling applied to the training data infected tree circumscribing rectangle, which is a rectangle circumscribing the infected tree included in the training data aerial photograph images, labeling applied to the training data dead tree circumscribing rectangle, which is a rectangle circumscribing the dead tree included in the training data aerial photograph images, and labeling applied to the training data dead tree circumscribing rectangle, which is a rectangle circumscribing the dead tree included in the training data aerial photograph images. and when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes an infected tree, a dead tree, and / or a damaged tree, the object detection model outputs a detection target infected tree circumscribing rectangle which is a rectangle circumscribing an infected tree included in any of the multiple rectangular images cut out by the image cutout processing unit, a detection target dead tree circumscribing rectangle which is a rectangle circumscribing a dead tree, and / or a detection target dead tree circumscribing rectangle which is a rectangle circumscribing a damaged tree, and the damaged tree position calculation processing unitThe location information of the infected tree detected by the damaged tree detection processing unit may be calculated based on the circumscribing rectangle of the infected tree to be detected output by the object detection model, the location information of the dead tree detected by the damaged tree detection processing unit may be calculated based on the circumscribing rectangle of the dead tree to be detected output by the object detection model, and the location information of the dead tree detected by the damaged tree detection processing unit may be calculated based on the circumscribing rectangle of the dead tree to be detected output by the object detection model.

[0015] In one aspect of the forest damaged tree automatic detection device of the present invention, the data input processing unit receives input of data of visible light range orthoimages, which are orthoimages generated from RGB color images in the visible light range taken using a drone, and data of multispectral orthoimages, which are orthoimages generated from multispectral images taken by an artificial satellite or an aircraft, as the aerial image data, the image clipping processing unit clips a plurality of rectangular images from the visible light range orthoimage and also clips a plurality of rectangular images from the multispectral orthoimage, the damaged tree search processing unit determines whether or not each of the plurality of rectangular images clipped from the visible light range orthoimage includes a damaged tree and calculates a reliability that each of the plurality of rectangular images clipped from the visible light range orthoimage includes a damaged tree, determines whether or not each of the plurality of rectangular images clipped from the multispectral orthoimage includes a damaged tree and calculates a reliability that each of the plurality of rectangular images clipped from the multispectral orthoimage includes a damaged tree, and the damaged tree position calculation processing unit performs the steps of: The damaged tree search processing unit calculates the location information of the damaged tree that is determined by the damaged tree search processing unit to be included in one of the multiple rectangular images cut out from the visible light range orthoimage, and the location information of the damaged tree that is determined by the damaged tree search processing unit to be included in one of the multiple rectangular images cut out from the multispectral orthoimage.If the distance between the location of the damaged tree included in one of the multiple rectangular images cut out from the visible light range orthoimage and the location of the damaged tree included in one of the multiple rectangular images cut out from the multispectral orthoimage is within the first threshold, the damaged tree included in one of the multiple rectangular images cut out from the visible light range orthoimage and the damaged tree included in one of the multiple rectangular images cut out from the multispectral orthoimage are considered to be the same damaged tree.The damaged tree search processing unit may use the location information of the damaged tree included in one of the multiple rectangular images cut out from the visible light range orthoimage and the location information of the damaged tree included in one of the multiple rectangular images cut out from the multispectral orthoimage, whichever has a higher reliability calculated by the damaged tree search processing unit.

[0016] One aspect of the present invention is a method for automatically detecting damaged trees in a forest area to be surveyed, the method comprising: a data input process step for receiving input of data on an aerial image of the forest area to be surveyed and detection range designation information that designates a range within which damaged trees included in the aerial image are to be detected; an image clipping process step for clipping a plurality of rectangular images from the aerial image by generating a grid within the range within which damaged trees are to be detected that is designated by the detection range designation information; and a damaged tree detection process step for detecting damaged trees included in each of the plurality of rectangular images clipped in the image clipping process step, the damaged tree detection process step including a damaged tree search process step for determining whether or not a damaged tree is included in each of the plurality of rectangular images clipped by the image clipping processing unit; and a damaged tree position calculation process step for calculating position information of the damaged tree when it is determined in the damaged tree search process step that a damaged tree is included in any of the plurality of rectangular images clipped in the image clipping process step, In the processing step, supervised learning of an object detection model using training data is performed, and by using the object detection model after supervised learning, it is determined whether or not a damaged tree is included in each of the multiple rectangular images cut out in the image cutting out processing step. The training data used for supervised learning of the object detection model includes aerial images for training data that include damaged trees, and labeling applied to training data damaged tree circumscribing rectangles that are rectangles circumscribing damaged trees included in the aerial images for training data. When it is determined in the damaged tree search processing step that a damaged tree is included in any of the multiple rectangular images cut out in the image cutting out processing step, the object detection model outputs a detection target damaged tree circumscribing rectangle that is a rectangle circumscribing a damaged tree included in any of the multiple rectangular images cut out in the image cutting out processing step. In the damaged tree position calculation processing step, position information of the damaged tree detected in the damaged tree detection processing step is calculated based on the detection target damaged tree circumscribing rectangle output by the object detection model. In the damaged tree position calculation processing step, the center positions in the vertical and horizontal directions of the circumscribing rectangle of the detected damaged tree output by the object detection model are calculated as the position of the damaged tree detected in the damaged tree detection processing step, and in the image cutout processing step, a plurality of rectangular images are cut out from the aerial image so that a part of one of two rectangular images adjacent in the vertical direction overlaps with a part of the other, and a part of one of two rectangular images adjacent in the horizontal direction overlaps with a part of the other. This is a method for automatically detecting damaged trees in forests.

[0017] One aspect of the present invention is a program for causing a computer to execute a data input processing step for accepting input of data on an aerial photograph of a forest area to be surveyed and detection range designation information, which is information for designating the range within which detection of damaged trees contained in the aerial photograph is to be carried out; an image clipping processing step for clipping a plurality of rectangular images from the aerial photograph by generating a grid within the range within which detection of damaged trees is to be carried out, which is designated by the detection range designation information; and a damaged tree detection processing step for detecting damaged trees contained in each of the plurality of rectangular images clipped in the image clipping processing step, wherein the damaged tree detection processing step includes a damaged tree search processing step for determining whether or not a damaged tree is included in each of the plurality of rectangular images clipped by the image clipping processing unit, and a damaged tree position calculation processing step for calculating position information of the damaged tree when it is determined in the damaged tree search processing step that a damaged tree is included in any of the plurality of rectangular images clipped in the image clipping processing step, and in the damaged tree search processing step: Supervised learning of an object detection model is performed using training data, and by using the object detection model after supervised learning, it is determined whether or not a damaged tree is included in each of the multiple rectangular images cut out in the image cutout processing step, the training data used for supervised learning of the object detection model includes aerial images for training data that include damaged trees and labeling applied to training data damaged tree circumscribing rectangles that are rectangles circumscribing damaged trees included in the aerial images for training data, and when it is determined in the damaged tree search processing step that a damaged tree is included in any of the multiple rectangular images cut out in the image cutout processing step, the object detection model outputs a detection target damaged tree circumscribing rectangle that is a rectangle circumscribing a damaged tree included in any of the multiple rectangular images cut out in the image cutout processing step, and in the damaged tree position calculation processing step, position information of the damaged tree detected in the damaged tree detection processing step is calculated based on the detection target damaged tree circumscribing rectangle output by the object detection model, In the damaged tree position calculation processing step, the center positions in the vertical and horizontal directions of the circumscribing rectangle of the detected damaged tree output by the object detection model are calculated as the position of the damaged tree detected in the damaged tree detection processing step, and in the image cutout processing step, a plurality of rectangular images are cut out from the aerial image so that a part of one of two rectangular images adjacent in the vertical direction overlaps with a part of the other, and a part of one of two rectangular images adjacent in the horizontal direction overlaps with a part of the other. It is a program. [Effects of the Invention]

[0018] According to the present invention, it is possible to provide an automatic forest damaged tree detection device, an automatic forest damaged tree detection method, and a program that can determine whether or not there are damaged trees in a forest area being surveyed and the location of damaged trees in the forest area being surveyed using a simple method. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an example of an automatic forest damaged tree detection device 1 according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of the relationship between an aerial image (orthoimage) AM input by the data input processing unit 11 and the "area AR where damaged tree detection is performed." [Figure 3] 10 is a diagram for explaining an example of processing by the image cutout processing unit 12. FIG. [Figure 4] 10 is a diagram for explaining an example of overlapping between a plurality of rectangular images RM1, RM2, RM3, and RM4 cut out by the image cutout processing unit 12. FIG. [Figure 5] 5 is a diagram for explaining why rectangular images RMi (i=1 to N) as shown in FIG. 4 are cut out by the image cutout processing unit 12. FIG. [Figure 6] 1 is a diagram illustrating an example of aerial images TM for training data, damaged tree bounding rectangles TER for training data, and labeling used in supervised learning of an object detection model. FIG. [Figure 7] 10 is a diagram showing an example of a detection target damaged tree circumscribing rectangle DER that circumscribes a damaged tree DT included in a rectangular image RMp output by an object detection model. FIG. [Figure 8] FIG. 10 is a diagram for explaining an example of calculation of the position of a damaged tree DT performed by a damaged tree position calculation processing unit 13B. [Figure 9] 10 is a diagram showing an example of an analysis result output by an analysis result output processing unit 14. FIG. [Figure 10] 10 is a diagram for explaining an example of processing by the ground height filter processing unit 15. FIG. [Figure 11] 1 is a diagram showing an example of the detection results of damaged trees DT (dead trees DTA, dead and damaged trees DTB) in a forest area to be surveyed, output by the automatic forest damaged tree detection device 1 of the first embodiment. FIG. [Figure 12] 1 is a flowchart illustrating an example of processing executed in the automatic forest damaged tree detection device 1 according to the first embodiment. [Figure 13] 13 is a flowchart for explaining in detail an example of processing executed in step S13 of FIG. 12. [Figure 14] FIG. 10 is a diagram illustrating an example of an automatic forest damaged tree detection device 1 according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of an automatic forest damaged tree detection device, an automatic forest damaged tree detection method, and a program according to the present invention will be described.

[0021] First Embodiment FIG. 1 is a diagram showing an example of an automatic forest damaged tree detection device 1 according to the first embodiment. In the example shown in Fig. 1, the automatic forest damaged tree detection device 1 of the first embodiment automatically detects damaged trees in a forest area to be surveyed. In detail, the automatic forest damaged tree detection device 1 of the first embodiment distinguishes (identifies) and detects damaged trees into two types: dead trees, such as pine trees whose leaves have turned reddish-brown and died, and dead and damaged trees that have been dead for several years, lost their leaves, and are now only left with thick branches and trunks, turning into skeletons. Dead trees are damaged trees that require urgent action to prevent the spread of infection, while dead and damaged trees are damaged trees that are at high risk of losing branches and falling. The automatic forest damaged tree detection device 1 comprises a data input processing unit 11, an image clipping processing unit 12, a damaged tree detection processing unit 13, an analysis result output processing unit 14, and a ground height filter processing unit 15. The data input processing unit 11 receives input of data of an aerial image AM of a forest area to be surveyed. The aerial image AM of the forest area to be surveyed is an orthoimage generated from an RGB color image in the visible light range captured using a drone. The data input processing unit 11 also receives input of detection range designation information, which is information that designates the "area AR within which damaged tree detection is carried out" included in the aerial photograph image AM.

[0022] Figure 2 is a diagram showing an example of the relationship between the aerial image (orthoimage) AM that is input to the data input processing unit 11 and the "area AR where damaged tree detection is performed." In detail, Figure 2(A) shows the aerial image (orthoimage) AM, and Figure 2(B) shows a polygon (polygon) corresponding to the "area AR where damaged tree detection is performed." 2, the "area AR where damaged tree detection is performed" is specified by a polygon. That is, the data input processing unit 11 receives input of a polygon file as detection area specification information that specifies the "area AR where damaged tree detection is performed" included in the aerial image AM.

[0023] In the example shown in Figure 1, the image cutout processing unit 12 cuts out multiple rectangular images RMi (i = 1 to N) from the aerial image AM by generating a grid within the ``range AR where damaged tree detection is performed'' specified by the detection range designation information (polygon file).

[0024] FIG. 3 is a diagram for explaining an example of processing by the image cutout processing unit 12. In FIG. As shown in Figures 2 and 3, the image cutout processing unit 12 generates a grid within the ``area AR where damaged trees are detected'' specified by the polygon shown in Figure 2(B) in the aerial image (orthoimage) AM shown in Figure 2(A), and cuts out multiple rectangular images RMi (i = 1 to N). In detail, the image cutout processing unit 12 cuts out multiple rectangular images from the aerial image so that a part of one of two adjacent rectangular images in the vertical direction overlaps with a part of the other, and a part of one of two adjacent rectangular images in the horizontal direction overlaps with a part of the other.

[0025] Figure 4 is a diagram illustrating an example of overlap between multiple rectangular images RM1, RM2, RM3, and RM4 cut out by the image cutout processing unit 12. In detail, Figure 4(A) shows an enlarged view of a portion of the aerial image AM shown in Figure 2(A) within the "area AR where damaged trees are detected" shown in Figure 2(B), and Figure 4(B) shows multiple rectangular images RM1, RM2, RM3, and RM4 cut out by the image cutout processing unit 12 from the aerial image AM shown in Figure 4(A). In Figure 4, DT indicates damaged trees (more specifically, dead or damaged trees). In the example shown in Fig. 4, rectangular images RM1 and RM3 are adjacent to each other in the vertical direction. Rectangular images RM2 and RM4 are adjacent to each other in the vertical direction. Rectangular images RM1 and RM2 are adjacent to each other in the horizontal direction. Rectangular images RM3 and RM4 are adjacent to each other in the horizontal direction. 50% of the bottom side of rectangular image RM1 (bottom side in FIG. 4(B)) overlaps with 50% of the top side of rectangular image RM3 (top side in FIG. 4(B)). 50% of the bottom side of rectangular image RM2 (bottom side in FIG. 4(B)) overlaps with 50% of the top side of rectangular image RM4 (top side in FIG. 4(B)). 50% of the right side of rectangular image RM1 (right side in FIG. 4(B)) overlaps with 50% of the left side of rectangular image RM2 (left side in FIG. 4(B)). 50% of the right side of rectangular image RM3 (right side in FIG. 4(B)) overlaps with 50% of the left side of rectangular image RM4 (left side in FIG. 4(B)).

[0026] That is, in the example shown in FIG. 4, the image cropping processing unit 12 determines whether a part of one rectangular image RM1 (the lower part of FIG. 4(B)) of two vertically adjacent rectangular images RM1 and RM3 overlaps with a part of the other rectangular image RM3 (the upper part of FIG. 4(B)), and whether a part of one rectangular image RM2 (the lower part of FIG. 4(B)) of two vertically adjacent rectangular images RM2 and RM4 overlaps with a part of the other rectangular image RM4 (the upper part of FIG. 4(B)). Multiple rectangular images RM1, RM2, RM3, and RM4 are cut out from the aerial image AM so that a portion of one rectangular image RM1 (the right side of Figure 4(B)) of two adjacent rectangular images RM1 and RM2 overlaps with a portion of the other rectangular image RM2 (the left side of Figure 4(B)), and so that a portion of one rectangular image RM3 (the right side of Figure 4(B)) of two horizontally adjacent rectangular images RM3 and RM4 overlaps with a portion of the other rectangular image RM4 (the left side of Figure 4(B)).

[0027] In the example shown in FIG. 4, the image cutout processing unit 12 cuts out rectangular images RM1, RM2, RM3, and RM4, each having a square of, for example, 10 m×10 m, from the aerial photograph image AM. In other examples, the image cutout processing unit 12 may cut out square rectangular images RM1, RM2, RM3, and RM4 with a side length other than 10 m from the aerial image AM, or the image cutout processing unit 12 may cut out rectangular images RM1, RM2, RM3, and RM4 from the aerial image AM.

[0028] In the example shown in Figure 4, the vertical overlap width, which is the overlap width between a portion of one rectangular image RM1 of two adjacent rectangular images RM1 and RM3 in the vertical direction and a portion of the other rectangular image RM3, is 5m (=10m x 50%), and the horizontal overlap width, which is the overlap width between a portion of one rectangular image RM1 of two adjacent rectangular images RM1 and RM2 in the horizontal direction and a portion of the other rectangular image RM2, is 5m (=10m x 50%). In another example, the overlap width in the vertical direction and the overlap width in the horizontal direction may be a value other than 5 m.

[0029] In the example shown in FIG. 4, when the image cropping processor 12 crops out a rectangular image RM1 from an aerial image AM, information indicating the absolute coordinate values ​​(X1, Y1) (e.g., latitude and longitude) of the origin of rectangular image RM1 (e.g., the upper left corner of rectangular image RM1 shown in FIG. 4(B)) is included in the data of rectangular image RM1. Furthermore, when the image cropping processor 12 crops out a rectangular image RM2 from an aerial image AM, information indicating the absolute coordinate values ​​(X2, Y2) of the origin of rectangular image RM2 is included in the data of rectangular image RM2. Similarly, when the image cropping processor 12 crops out a rectangular image RM3 from an aerial image AM, information indicating the absolute coordinate values ​​(X3, Y3) of the origin of rectangular image RM3 is included in the data of rectangular image RM3, and when the image cropping processor 12 crops out a rectangular image RM4 from an aerial image AM, information indicating the absolute coordinate values ​​(X4, Y4) of the origin of rectangular image RM4 is included in the data of rectangular image RM4. That is, rectangular images RMi (i=1 to N) are associated with absolute coordinate values ​​and cut out from the aerial image AM.

[0030] FIG. 5 is a diagram for explaining why the image cutout processing unit 12 cuts out rectangular images RMi (i=1 to N) as shown in FIG. As shown in Figures 5(A) and 5(B), if a portion of one of two rectangular images RMA and RMB adjacent to each other in the left-right direction does not overlap with a portion of the other rectangular image RMB, the damaged tree DT (crown) located on the boundary between the rectangular images RMA and RMB will be divided into two rectangular images RMA and RMB, and there is a risk that the damaged tree DT will not be detected correctly. On the other hand, as shown in Figures 5(C) and 5(D), when a portion of one of two rectangular images RMA, RMB adjacent to each other in the horizontal direction overlaps with a portion of the other rectangular image RMB, and when a portion of one of two rectangular images RMB, RMC adjacent to each other in the horizontal direction overlaps with a portion of the other rectangular image RMC, the entire damaged tree DT (crown of the tree) is contained in one rectangular image RMB, so the automatic forest damaged tree detection device 1 can reliably detect the damaged tree DT by using the rectangular image RMB (see Figure 5(D)). That is, in the automatic forest damaged tree detection device 1 of the first embodiment, a vertical overlap width and a horizontal overlap width are provided as shown in Fig. 4. As a result, in the automatic forest damaged tree detection device 1 of the first embodiment, it is possible to reduce the risk that damaged tree DTs located on the boundary line between two horizontally adjacent rectangular images will not be detected correctly because two horizontally adjacent rectangular images do not overlap each other, as in the examples shown in Fig. 5(A) and Fig. 5(B), and it is possible to reduce the risk that damaged tree DTs located on the boundary line between two vertically adjacent rectangular images will not be detected correctly because two vertically adjacent rectangular images do not overlap each other.

[0031] In the example shown in Fig. 1, the damaged tree detection processing unit 13 detects damaged trees DT contained in each of a plurality of rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. The damaged tree detection processing unit 13 includes a damaged tree search processing unit 13A, a damaged tree position calculation processing unit 13B, a matching processing unit 13C, and a noise removal processing unit 13D. The damaged tree search processing unit 13A determines whether or not each of the multiple rectangular images RMi (i=1 to N) cut out by the image cutout processing unit 12 includes a damaged tree DT. In detail, the damaged tree search processing unit 13A performs supervised learning of an object detection model using training data before determining whether or not a damaged tree DT is included in each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. Furthermore, the damaged tree search processing unit 13A uses the object detection model after supervised learning to determine whether or not a damaged tree DT is included in each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. That is, the damaged tree search processing unit 13A detects damaged trees DT using an object detection method. The object detection method is a type of AI (artificial intelligence) (more specifically, a type of deep learning), and by providing training data to an object detection model and having it learn in advance, it becomes possible to automatically analyze rectangular images RMi (i = 1 to N) input to the object detection model. The training data used for supervised learning of the object detection model includes aerial images TM for training data (see Figure 6) that include damaged trees, and labeling applied to the damaged tree circumscribing rectangle TER for training data (see Figure 6), which is a rectangle that circumscribing the damaged tree DT (see Figure 6) included in the aerial images TM for training data.

[0032] Figure 6 is a diagram illustrating an example of aerial image TM for training data, damaged tree circumscribing rectangle TER for training data, and labeling used in supervised learning of an object detection model. In detail, Figure 6(A) shows aerial image TM for training data used in supervised learning of an object detection model, and Figure 6(B) shows two damaged tree circumscribing rectangles TER for training data that circumscribing each of two damaged trees DT (dead tree DTA, damaged tree DTB) included in the aerial image TM for training data shown in Figure 6(A), and the labeling applied to them. In the example shown in Figure 6, an orthoimage generated from an RGB color image in the visible light range taken using a drone or the like in a forest area different from the "area AR where damaged trees are detected" shown in Figure 2 is used as the training data aerial image TM shown in Figure 6. The size and shape of the training data aerial image TM shown in Figure 6 are set to a 10m x 10m square, similar to the size and shape of the rectangular images RM1, RM2, RM3, and RM4 shown in Figure 4(B). The two damaged trees DT included in the training data aerial image TM shown in Figure 6 are a dead tree DTA and a dead and damaged tree DTB.

[0033] In the example shown in Figure 6, a process commonly referred to as "annotation" is performed to create training data to be used in supervised learning of an object detection model. Specifically, in this process, as shown in Figure 6(B), a training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) is created that circumscribing the dead tree DTA included in the training data aerial image TM, and the training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) is labeled as a "dead tree." In addition, a training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) is created that circumscribing the dead and damaged tree DTB included in the training data aerial image TM, and the training data dead and damaged tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) is labeled as a "dead tree." In the example shown in FIG. 6, the training data used for supervised learning of the object detection model includes a training data aerial photograph image TM including dead trees DTA and dead and damaged trees DTB, a labeling "dead tree" applied to a training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) that circumscribing the dead and damaged tree DTA included in the training data aerial photograph image TM, and a labeling "dead tree" applied to a training data dead and damaged tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) that circumscribing the dead and damaged tree DTB included in the training data aerial photograph image TM. However, in other examples, the training data used for supervised learning of the object detection model includes a dead tree DTA and a damaged tree DTB included in the training data aerial photograph image TM. The training data may include an aerial photograph image (not shown) containing only the dead tree DTA and labeling "dead tree" applied to a training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) for dead trees that circumscribing the dead tree DTA included in the aerial photograph image for training data, and an aerial photograph image (not shown) containing only the dead and damaged tree DTB and labeling "dead and damaged tree" applied to a training data dead and damaged tree circumscribing rectangle (training data damaged tree circumscribing rectangle TER) for dead trees that circumscribing the dead and damaged tree DTB included in the aerial photograph image for training data (in other words, one aerial photograph image TM for training data may include only one of the dead tree DTA and the dead and damaged tree DTB).

[0034] 1, supervised learning of the object detection model is performed by using, as training data, a plurality of labeled aerial images TM for training data, such as those shown in Fig. 6(B). Furthermore, by using the object detection model after supervised learning, it is determined whether or not a damaged tree DT (dead tree DTA, dead tree DTB) is included in each of a plurality of rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. In detail, the damaged tree search processing unit 13A determines whether or not each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cut-out processing unit 12 contains a dead tree DTA, and determines whether or not each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cut-out processing unit 12 contains a dead or damaged tree DTB.

[0035] When the damaged tree search processing unit 13A determines that a damaged tree DT (dead tree DTA, dead and damaged tree DTB) is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the object detection model outputs the detected damaged tree circumscribing rectangle DER (see Figure 7), which is a rectangle circumscribing the damaged tree DT included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. In detail, when the damaged tree search processing unit 13A determines that a dead tree DTA is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the object detection model outputs a detection target dead tree circumscribing rectangle (detection target damaged tree circumscribing rectangle DER), which is a rectangle circumscribing the dead tree DTA included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. In other words, the object detection model identifies the area where the dead tree DTA exists. Furthermore, when the damaged tree search processing unit 13A determines that a dead or damaged tree DTB is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the object detection model outputs a detection target dead or damaged tree circumscribing rectangle (detection target damaged tree circumscribing rectangle DER), which is a rectangle circumscribing the dead or damaged tree DTB included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. In other words, the object detection model identifies the area where the dead or damaged tree DTB is located.

[0036] FIG. 7 is a diagram showing an example of a detection target damaged tree circumscribing rectangle DER that circumscribes a damaged tree DT included in a rectangle image RMp output by the object detection model. As shown in FIG. 7, the object detection model outputs the area in which the detected damaged tree DT appears as a circumscribed rectangle DER of the detected target damaged tree.

[0037] In the example shown in Figure 1, when the damaged tree search processing unit 13A determines that any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a damaged tree DT (dead tree DTA, dead and damaged tree DTB), it calculates the reliability that each rectangular image RMi (i = 1 to N) contains the damaged tree DT (dead tree DTA, dead and damaged tree DTB) by using an object detection model after supervised learning. The damaged tree position calculation processing unit 13B calculates the position information of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) when the damaged tree search processing unit 13A determines that the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. In detail, the damaged tree position calculation processing unit 13B calculates the center positions in the vertical and horizontal directions of the circumscribing rectangle DER (see Figure 7) of the detected damaged tree output by the object detection model as the position of the damaged tree DT detected by the damaged tree detection processing unit 13.

[0038] FIG. 8 is a diagram for explaining an example of calculation of the position of a damaged tree DT performed by the damaged tree position calculation processing unit 13B. In the example shown in Figure 8, the damaged tree position calculation processing unit 13B calculates the position of the damaged tree DT contained in the rectangular image RMq cut out by the image cutout processing unit 12 based on the circumscribed rectangle DER of the detected damaged tree output by the object detection model. As described above, when the image cropping processing unit 12 crops out the rectangular image RMq, information indicating the absolute coordinate values ​​(X, Y) (e.g., latitude and longitude) of the origin of the rectangular image RMq (e.g., the upper left corner of the rectangular image RMq shown in Figure 8) is included in the data of the rectangular image RMq. In addition, when the object detection model outputs the bounding rectangle DER of the target damaged tree, information indicating, for example, the relative coordinate values ​​(x1, y1) of the upper left corner and the relative coordinate values ​​(x2, y2) of the lower right corner of the bounding rectangle DER of the target damaged tree relative to the origin (absolute coordinate values ​​(X, Y)) of the rectangular image RMq is included in the data of the rectangular image RMq. The damaged tree position calculation processing unit 13B calculates the position (X+(x1+x2) / 2, Y+(y1+y2) / 2) of the damaged tree DT based on the origin (absolute coordinate value (X, Y)) of the rectangular image RMq and the relative coordinate values ​​(x1, y1) and (x2, y2) of the circumscribing rectangle DER of the damaged tree to be detected.

[0039] In more detail, when the damaged tree search processing unit 13A determines that a dead tree DTA is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the damaged tree position calculation processing unit 13B calculates the position information of the dead tree DTA. The damaged tree position calculation processing unit 13B calculates the position information of the dead tree DTA based on the detection target dead tree circumscribing rectangle (detection target damaged tree circumscribing rectangle DER) output by the object detection model. Specifically, the damaged tree position calculation processing unit 13B calculates the center positions in the vertical and horizontal directions of the detection target dead tree circumscribing rectangle (detection target damaged tree circumscribing rectangle DER) output by the object detection model as the position of the dead tree DTA detected by the damaged tree detection processing unit 13. Furthermore, when the damaged tree search processing unit 13A determines that a dead or damaged tree DTB is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the damaged tree position calculation processing unit 13B calculates the position information of the dead or damaged tree DTB. The damaged tree position calculation processing unit 13B calculates the position information of the dead or damaged tree DTB based on the detection target dead or damaged tree circumscribing rectangle (detection target damaged tree circumscribing rectangle DER) output by the object detection model. Specifically, the damaged tree position calculation processing unit 13B calculates the center positions in the vertical and horizontal directions of the detection target dead or damaged tree circumscribing rectangle (detection target damaged tree circumscribing rectangle DER) output by the object detection model as the position of the dead or damaged tree DTB detected by the damaged tree detection processing unit 13.

[0040] In the example shown in Figure 1, when the damaged tree search processing unit 13A determines that a damaged tree DT is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the matching processing unit 13C matches the damaged tree DT based on the position information of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B. In the example shown in Figure 4(B), there is a risk that the damaged tree search processing unit 13A will determine that the rectangular image RM1 cut out by the image cut-out processing unit 12 contains the damaged tree DT, that the damaged tree search processing unit 13A will determine that the rectangular image RM2 cut out by the image cut-out processing unit 12 contains the damaged tree DT, that the damaged tree search processing unit 13A will determine that the rectangular image RM3 cut out by the image cut-out processing unit 12 contains the damaged tree DT, and that the damaged tree search processing unit 13A will determine that the rectangular image RM4 cut out by the image cut-out processing unit 12 contains the damaged tree DT.In other words, there is a risk that the damaged tree search processing unit 13A will mistakenly count one damaged tree DT as multiple damaged trees DT (four in the example shown in Figure 4(B)).

[0041] In consideration of this, in the example shown in Figure 1, if the distance between the position of the damaged tree DT included in one of two vertically adjacent rectangular images (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) and the position of the damaged tree DT included in the other of the two vertically adjacent rectangular images (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) is within a first threshold value (e.g., 3 m), the matching processing unit 13C considers the damaged tree DT included in one of the two vertically adjacent rectangular images and the damaged tree DT included in the other of the two vertically adjacent rectangular images to be the same damaged tree. The overlap width (5 m), which is the width of overlap between a part of one of the two vertically adjacent rectangular images and a part of the other, is greater than the first threshold value (3 m). Furthermore, if the distance between the position of the damaged tree DT included in one of two horizontally adjacent rectangular images (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) and the position of the damaged tree DT included in the other of the two horizontally adjacent rectangular images (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) is within a first threshold value (3 m), the matching processing unit 13C considers the damaged tree DT included in one of the two horizontally adjacent rectangular images and the damaged tree DT included in the other of the two horizontally adjacent rectangular images to be the same damaged tree. The horizontal overlap width (5 m), which is the overlap width between a part of one of the two horizontally adjacent rectangular images and a part of the other, is greater than the first threshold value (3 m).

[0042] 4(B), the distance between the position of the damaged tree DT included in one rectangular image RM1 of two vertically adjacent rectangular images RM1 and RM3 (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) and the position of the damaged tree DT included in the other rectangular image RM3 of the two vertically adjacent rectangular images RM1 and RM3 (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) is within the first threshold value (3 m). Therefore, the matching processing unit 13C considers that the damaged tree DT included in one rectangular image RM1 of the two vertically adjacent rectangular images RM1 and RM3 and the damaged tree DT included in the other rectangular image RM3 of the two vertically adjacent rectangular images RM1 and RM3 are the same damaged tree, and discards the position information of the damaged tree DT included in rectangular image RM1 (i.e., considers that the damaged tree DT included in rectangular image RM1 does not exist). 4(B), the distance between the position of the damaged tree DT included in one rectangular image RM3 of the two horizontally adjacent rectangular images RM3 and RM4 (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) and the position of the damaged tree DT included in the other rectangular image RM4 of the two horizontally adjacent rectangular images RM3 and RM4 (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) is within the first threshold value (3 m). Therefore, the matching processing unit 13C considers that the damaged tree DT included in one rectangular image RM3 of the two horizontally adjacent rectangular images RM3 and RM4 and the damaged tree DT included in the other rectangular image RM4 of the two horizontally adjacent rectangular images RM3 and RM4 are the same damaged tree, and discards the position information of the damaged tree DT included in rectangular image RM3 (i.e., considers that the damaged tree DT included in rectangular image RM3 does not exist).

[0043] 4(B), the distance between the position of the damaged tree DT included in one rectangular image RM2 of two vertically adjacent rectangular images RM2 and RM4 (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) and the position of the damaged tree DT included in the other rectangular image RM4 of the two vertically adjacent rectangular images RM2 and RM4 (the position of the damaged tree DT calculated by the damaged tree position calculation processing unit 13B) is within the first threshold value (3 m). Therefore, the matching processing unit 13C considers that the damaged tree DT included in one rectangular image RM2 of the two vertically adjacent rectangular images RM2 and RM4 and the damaged tree DT included in the other rectangular image RM4 of the two vertically adjacent rectangular images RM2 and RM4 are the same damaged tree, and discards the position information of the damaged tree DT included in rectangular image RM4 (i.e., considers that the damaged tree DT included in rectangular image RM4 does not exist). In the example shown in Figure 4(B), the position information of the damaged tree DT contained in rectangular image RM1 is discarded, the position information of the damaged tree DT contained in rectangular image RM3 is discarded, and the position information of the damaged tree DT contained in rectangular image RM4 is discarded, so that only the position information of the damaged tree DT contained in rectangular image RM2 remains, and the matching processing unit 13C determines that one damaged tree DT is contained in each of the four rectangular images RM1, RM2, RM3, and RM4.

[0044] In the example shown in Figure 1, as described above, when the damaged tree search processing unit 13A determines that one of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a damaged tree DT, it calculates the reliability that each rectangular image RMi (i = 1 to N) contains a damaged tree DT by using the object detection model after supervised learning. The noise removal processing unit 13D performs processing to remove the damaged tree DT as noise based on the reliability calculated by the damaged tree search processing unit 13A. Specifically, when the damaged tree search processing unit 13A determines that a damaged tree DT is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, and the reliability calculated by the damaged tree search processing unit 13A using the object detection model after supervised learning is less than or equal to a second threshold value (e.g., "0.5"), the noise removal processing unit 13D removes the damaged tree DT having a reliability less than or equal to the second threshold value ("0.5") as noise. The analysis result output processing unit 14 outputs the analysis results by the damaged tree detection processing unit 13 (for example, the position of the damaged tree DT within the ``range AR where damaged tree detection is performed'' shown in Figure 2, and the identification result of whether the damaged tree DT is a dead tree DTA or a dead and damaged tree DTB).

[0045] Figure 9 shows an example of the analysis results output by the analysis result output processing unit 14. In detail, Figure 9(A) shows a damaged tree information list, and Figure 9(B) shows an enlarged portion of the aerial photograph image (orthoimage) to which damaged tree information has been added. In the example shown in Figure 9, the analysis result output processing unit 14 converts the analysis results (information shown as "damaged tree information list" in Figure 9(A)) by the damaged tree detection processing unit 13 into a shapefile and outputs it. A shapefile is generally said to be a collection of "map data files with graphic information and attribute information." In the "Damaged Tree Information List" shown in Figure 9(A), the damaged tree DT located at coordinates (x1, y1) is identified as a dead tree DTA, the damaged tree DT located at coordinates (x2, y2) is identified as a dead tree DTB, and the damaged tree DT located at coordinates (x3, y3) is identified as a dead tree DTA. The analysis result output processing unit 14 places a symbol "●" indicating a dead tree DTA at a position corresponding to the coordinates (x1, y1) on the aerial image (orthoimage) shown in Figure 9(B). The analysis result output processing unit 14 also places a symbol "○" indicating a dead tree DTB at a position corresponding to the coordinates (x2, y2), and places a symbol "●" indicating a dead tree DTA at a position corresponding to the coordinates (x3, y3).

[0046] In the example shown in Figure 9, the analysis result output processing unit 14 places the symbol "●" indicating a dead tree DTA and the symbol "○" indicating a dead tree DTB on one aerial image (orthoimage), but in other examples, the analysis result output processing unit 14 may separately generate and output an aerial image (orthoimage) on which the symbol "●" indicating a dead tree DTA is placed and an aerial image (orthoimage) on which the symbol "○" indicating a dead tree DTB is placed.

[0047] In the example shown in Fig. 1, the ground height filter processing unit 15 determines whether or not the detection result of the dead tree DTA by the damaged tree detection processing unit 13 is a false detection. The ground height filter processing unit 15 also determines whether or not the detection result of the dead or damaged tree DTB by the damaged tree detection processing unit 13 is a false detection. Specifically, the ground height filter processor 15 acquires digital elevation model (DEM) data of the forest area to be surveyed from, for example, the website indicated by the following URL. https: / / fgd.gsi.go.jp / download / menu.php Furthermore, the ground height filter processing unit 15 acquires digital surface model (DSM) data of the forest area to be surveyed from, for example, the website indicated by the following URL. https: / / www.geospatial.jp / ckan / dataset / aw3d-alljapan

[0048] The ground height filter processing unit 15 determines whether the damaged tree detection processing unit 13 has mistakenly detected the ground surface as a dead tree DTA based on the detection result of the dead tree DTA by the damaged tree detection processing unit 13, the digital elevation model (DEM) data, and the digital surface model (DSM) data. For example, if the DEM value and the DSM value at the position where the damaged tree detection processing unit 13 detected the dead tree DTA are equal, the ground height filter processing unit 15 determines that the damaged tree detection processing unit 13 has mistakenly detected the reddish-brown ground surface as a dead tree DTA. The ground height filter processing unit 15 determines whether the damaged tree detection processing unit 13 has mistakenly detected the ground surface as a dead or damaged tree DTB based on the detection result of the dead or damaged tree DTB by the damaged tree detection processing unit 13, the digital elevation model (DEM) data, and the digital surface model (DSM) data. For example, if the DEM value and the DSM value at the position where the dead or damaged tree DTB was detected by the damaged tree detection processing unit 13 are equal, the ground height filter processing unit 15 determines that the damaged tree detection processing unit 13 has mistakenly detected the white ground surface as a dead or damaged tree DTB.

[0049] Fig. 10 is a diagram for explaining an example of processing by the ground height filter processing unit 15. In detail, Fig. 10(A) shows the state before processing by the ground height filter processing unit 15 is performed (the state after the analysis result output processing unit 14 has output the analysis result), and Fig. 10(B) shows the state after processing by the ground height filter processing unit 15 has been performed. In the example shown in Figure 10, the symbol "○" indicating three dead and damaged tree DTBs is located in the circled area in Figure 10(A) of the analysis result (aerial image) output by the analysis result output processing unit 14. In other words, the damaged tree detection processing unit 13 has mistakenly detected the white ground surface as three dead and damaged tree DTBs. The ground height filter processing unit 15 determines whether the damaged tree detection processing unit 13 has mistakenly detected the ground surface as a dead or damaged tree DTB based on the detection result of the damaged tree detection processing unit 13 (detection result indicating the presence of three dead or damaged tree DTBs in the circled area in Figure 10(A)), the digital elevation model data, and the digital surface model data, and outputs a determination result that the damaged tree detection processing unit 13 has mistakenly detected the white ground surface in the circled area in Figure 10(A) as three dead or damaged tree DTBs. As a result, as shown in Figure 10(B), the symbols "○" indicating the three mistakenly detected dead or damaged tree DTBs are deleted.

[0050] In the example shown in Figure 1, in order to reduce the risk of the damaged tree detection processing unit 13 mistakenly detecting the ground surface as a damaged tree DT (dead tree DTA, dead and damaged tree DTB), the damaged tree circumscribing rectangle TER for training data (see Figure 6) of the aerial photograph image TM for training data is one in which the ground surface is not present around the damaged tree DT (dead tree DTA, dead and damaged tree DTB), but one in which healthy trees are present around the damaged tree DT (dead tree DTA, dead and damaged tree DTB).

[0051] FIG. 11 is a diagram showing an example of the detection results of damaged trees DT (dead trees DTA, dead and damaged trees DTB) in the forest area under investigation output by the automatic forest damaged tree detection device 1 of the first embodiment. In the example shown in Figure 11, the automatic forest damaged tree detection device 1 outputs, as a detection result of damaged trees DT (dead trees DTA, dead and damaged trees DTB) in the forest area under investigation, an aerial image AM (see Figure 2(A)) of the forest area under investigation accepted as input by the data input processing unit 11, with the symbol ``●'' indicating the dead trees DTA detected by the automatic forest damaged tree detection device 1 and the symbol ``○'' indicating the dead and damaged trees DTB placed on it.

[0052] Fig. 12 is a flowchart for explaining an example of processing executed in the automatic forest damaged tree detection device 1 of the first embodiment. Fig. 13 is a flowchart for explaining in detail an example of processing executed in step S13 of Fig. 12. In the example shown in Figures 12 and 13, in step S11, the data input processing unit 11 accepts input of data on an aerial image AM of the forest area to be surveyed and detection range designation information (polygon file) that specifies the ``range AR in which damaged trees are detected'' contained in the aerial image AM. Next, in step S12, the image cutout processing unit 12 cuts out multiple (N) rectangular images RMi (i = 1 to N) from the aerial image AM by generating a grid within the "range AR where damaged tree detection is performed" specified by the detection range designation information. Next, in step S13, the damaged tree detection processing unit 13 detects damaged trees DT (dead trees DTA, damaged trees DTB) included in each of the multiple rectangular images RMi (i=1 to N) cut out by the image cutout processing unit 12.

[0053] In detail, when step S13 starts, first, in step S13-1, the damaged tree detection processing unit 13 sets the number i of the rectangular image RMi of the detection target of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) to 1 (i=1). Next, in step S13-2, the damaged tree detection processing unit 13 determines whether the number i of the rectangular image RMi of the detection target of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is less than or equal to N. If the number i of the rectangular image RMi of the detection target of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is less than or equal to N (i.e., there is a rectangular image RMi for which the detection process of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) has not been completed), the process proceeds to step S13-3. On the other hand, if the number i of the rectangular image RMi of the detection target of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is greater than N (i.e., there is no rectangular image RMi for which the detection process of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) has not been completed), the process proceeds to step S14.

[0054] In step S13-3, the damaged tree search processing unit 13A reads the i-th rectangular image RMi from among the plurality of rectangular images RM1 to RMN cut out by the image cutout processing unit 12. Next, in step S13A, the damaged tree search processing unit 13A uses the object detection model after supervised learning to determine whether or not the i-th rectangular image RMi contains a damaged tree DT (dead tree DTA, dead and damaged tree DTB). That is, the damaged tree search processing unit 13A searches for a damaged tree DT (dead tree DTA, dead and damaged tree DTB) contained in the i-th rectangular image RMi. Next, in step S13-4, the damaged tree detection processing unit 13 determines whether or not there are any damaged trees DT (dead trees DTA, dead and damaged trees DTB) included in the i-th rectangular image RMi. If there are any damaged trees DT (dead trees DTA, dead and damaged trees DTB) included in the i-th rectangular image RMi, the process proceeds to step S13B. On the other hand, if there are no damaged trees DT (dead trees DTA, dead and damaged trees DTB) included in the i-th rectangular image RMi, the process proceeds to step S13-6.

[0055] In step S13B, the damaged tree position calculation processing unit 13B calculates the position information of the damaged trees DT (dead trees DTA, dead and damaged trees DTB) included in the i-th rectangular image RMi. Next, in step S13C, the matching processing unit 13C matches the damaged tree DT (dead tree DTA, damaged tree DTB) based on the position information of the damaged tree DT (dead tree DTA, damaged tree DTB) calculated in step S13B. In detail, the matching processing unit 13C determines whether or not a damaged tree DT (dead tree DTA, damaged tree DTB) identical to the damaged tree DT (dead tree DTA, damaged tree DTB) whose position information was calculated in step S13B already exists in the damaged tree information list (see FIG. 9(A)). If a damaged tree DT (dead tree DTA, damaged tree DTB) identical to the damaged tree DT (dead tree DTA, damaged tree DTB) whose position information was calculated in step S13B does not already exist in the damaged tree information list (see FIG. 9(A)), the process proceeds to step S13D. On the other hand, if a damaged tree DT (dead tree DTA, dead and damaged tree DTB) identical to the damaged tree DT (dead tree DTA, dead and damaged tree DTB) whose location information was calculated in step S13B already exists in the damaged tree information list (see Figure 9(A)), proceed to step S13-6.

[0056] In step S13D, the noise removal processor 13D performs noise removal processing. Specifically, the noise removal processor 13D determines whether the reliability that the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is included in the i-th rectangular image RMi is equal to or less than a second threshold value (0.5). If the reliability that the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is included in the i-th rectangular image RMi is equal to or less than the second threshold value (0.5), the noise removal processor 13D determines that the damaged tree DT (dead tree DTA, dead and damaged tree DTB) included in the i-th rectangular image RMi is noise, removes the damaged tree DT (dead tree DTA, dead and damaged tree DTB) included in the i-th rectangular image RMi, and proceeds to step S13-6. On the other hand, if the reliability that the damaged tree DT (dead tree DTA, dead and damaged tree DTB) is included in the i-th rectangular image RMi is greater than the second threshold value ("0.5"), the noise removal processing unit 13D determines that the damaged tree DT (dead tree DTA, dead and damaged tree DTB) included in the i-th rectangular image RMi is not noise but actually exists, and proceeds to step S13-5.

[0057] In step S13-5, the damaged tree detection processing unit 13 performs a damaged tree information list update process to add the damaged trees DT (dead trees DTA, dead and damaged trees DTB) included in the i-th rectangular image RMi to the damaged tree information list (see FIG. 9(A)). Then, the process proceeds to step S13-6.

[0058] In step S13-6, the damaged tree detection processing unit 13 increments the number i of the rectangular image RMi of the detection target of the damaged tree DT (dead tree DTA, dead and damaged tree DTB) (i=i+1).

[0059] In step S14, the analysis result output processing unit 14 outputs a damaged tree information list (see Figure 9(A)) showing the analysis results by the damaged tree detection processing unit 13 (for example, the position of the damaged tree DT within the ``range AR where damaged tree detection is performed'' shown in Figure 2, and the identification result of whether the damaged tree DT is a dead tree DTA or a dead and damaged tree DTB). Next, in step S15, the ground height filter processing unit 15 performs ground height filtering. In detail, the ground height filter processing unit 15 determines whether the damaged trees DT (dead trees DTA, dead and damaged trees DTB) detected by the damaged tree detection processing unit 13 are erroneous detections of the ground surface, and performs processing to delete the erroneously detected damaged trees DT (dead trees DTA, dead and damaged trees DTB) from the damaged tree information list.

[0060] As described above, the first embodiment of the automatic forest damaged tree detection device 1 is an application of a series of analysis processes, and for example, local governments, forestry businesses, etc. can obtain the above-mentioned analysis results by inputting into the automatic forest damaged tree detection device 1 an orthoimage (aerial image AM) generated from an RGB color image in the visible light range taken using a convenient drone, and detection range designation information contained in the aerial image AM that designates the ``range AR in which damaged trees are detected.'' In other words, according to the first embodiment of the automatic forest damaged tree detection device 1, it is possible to determine, by a simple method, whether or not there are damaged trees DT (dead trees DTA, dead and damaged trees DTB) in the forest area being surveyed, and the locations of the damaged trees DT (dead trees DTA, dead and damaged trees DTB) present in the forest area being surveyed. For example, by including trees damaged by pine wilt disease as damaged tree DT in the training data aerial images TM, it is possible to identify trees damaged by pine wilt disease in a short time and with simple operations. Furthermore, by increasing the amount of training data used in supervised learning of the object detection model, it is possible to improve the accuracy of estimating damaged tree DT by the object detection model after supervised learning. Furthermore, by including trees damaged by oak wilt as damaged trees DT in the training data aerial images TM, it is possible to identify trees damaged by oak wilt in a short time and with simple operations. Similarly, by including trees damaged by bear stripping (i.e., trees damaged by pine wilt disease and trees damaged by oak wilt disease) as damaged trees DT in the aerial images TM for training data, it is possible to identify trees damaged by bear stripping (trees damaged by pine wilt disease and trees damaged by oak wilt disease) in a short time and with simple operations. It is also possible to estimate the volume of damaged trees by combining the analysis results output from the automatic damaged forest tree detection device 1 of the first embodiment with a sample survey of standing trees on-site. The program executed by the computer that constitutes the automatic forest damaged tree detection device 1 of the first embodiment may be provided as a web application service. That is, the automatic forest damaged tree detection device 1 of the first embodiment can be said to be a low-cost, highly versatile, and easily widespread technology.

[0061] Second Embodiment A second embodiment of the automatic forest damaged tree detection device, the automatic forest damaged tree detection method, and the program of the present invention will be described below. The automatic forest damaged tree detection device 1 of the second embodiment is configured similarly to the automatic forest damaged tree detection device 1 of the first embodiment described above, except for the points described below. Therefore, the automatic forest damaged tree detection device 1 of the second embodiment can achieve the same effects as the automatic forest damaged tree detection device 1 of the first embodiment described above, except for the points described below.

[0062] FIG. 14 is a diagram showing an example of an automatic damaged forest tree detection device 1 according to the second embodiment. 14, the automatic forest damaged tree detection device 1 of the second embodiment automatically detects damaged trees in a forest area to be surveyed. In detail, the automatic forest damaged tree detection device 1 of the second embodiment classifies (identifies) and detects damaged trees as infected trees, dead trees, and dead and damaged trees. The automatic forest damaged tree detection device 1 comprises a data input processing unit 11, an image clipping processing unit 12, a damaged tree detection processing unit 13, and an analysis result output processing unit . The data input processing unit 11 receives input of data of an aerial image AM of a forest area to be surveyed. The aerial image AM of the forest area to be surveyed is an orthoimage generated from a multispectral image taken by an artificial satellite or an aircraft. The data input processing unit 11 also receives input of detection range designation information, which is information that designates the "area AR within which damaged tree detection is carried out" included in the aerial photograph image AM. The image cutout processing unit 12 cuts out multiple rectangular images RMi (i = 1 to N) from the aerial image AM (orthoimage generated from the multispectral image) by generating a grid within the ``range AR where damaged tree detection is performed'' specified by the detection range designation information (polygon file).

[0063] The damaged tree detection processing unit 13 detects damaged trees DT (infected trees, dead trees, and dead and damaged trees) contained in each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. The damaged tree detection processing unit 13 includes a damaged tree search processing unit 13A, a damaged tree position calculation processing unit 13B, a matching processing unit 13C, and a noise removal processing unit 13D. The damaged tree search processing unit 13A performs supervised learning of an object detection model using training data, and then uses the object detection model after supervised learning to determine whether or not each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a damaged tree DT (infected tree, dead tree, or dead and damaged tree).

[0064] In the example shown in Figure 14, the training data used for supervised learning of the object detection model includes aerial images for training data including infected trees, dead trees, and dead and damaged trees, labeling "infected tree" applied to the infected tree circumscribing rectangle for training data (infected tree circumscribing rectangle for training data) that circumscribing the infected tree included in the aerial images for training data, labeling "dead tree" applied to the dead tree circumscribing rectangle for training data (damaged tree circumscribing rectangle for training data) that circumscribing the dead tree included in the aerial images for training data, and labeling "dead tree" applied to the dead and damaged tree circumscribing rectangle for training data (damaged tree circumscribing rectangle for training data) that circumscribing the dead and damaged tree included in the aerial images for training data. In another example, the training data used for supervised learning of the object detection model may include a training data aerial image containing only infected trees, with the label "infected tree" applied to the training data infected tree circumscribing rectangle (training data damaged tree circumscribing rectangle) circumscribing the infected tree included in the training data aerial image; a training data aerial image containing only dead trees, with the label "dead tree" applied to the training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle) circumscribing the dead tree included in the training data aerial image; and a training data aerial image containing only dead and damaged trees, with the label "dead and damaged tree" applied to the training data dead and damaged tree circumscribing rectangle (training data damaged tree circumscribing rectangle) circumscribing the dead and damaged tree included in the training data aerial image (i.e., one training data aerial image may contain only infected trees, dead trees, or damaged trees).

[0065] In the example shown in Figure 14, the damaged tree search processing unit 13A determines whether or not each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutting processing unit 12 contains an infected tree, determines whether or not each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutting processing unit 12 contains a dead tree, and determines whether or not each of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutting processing unit 12 contains a dead or damaged tree. When the victim tree search processing unit 13A determines that an infected tree is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the object detection model outputs a target infected tree circumscribing rectangle (target victim tree circumscribing rectangle), which is a rectangle circumscribing an infected tree included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. When the damaged tree search processing unit 13A determines that one of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a dead tree, the object detection model outputs a dead tree circumscribing rectangle to be detected (damaged tree circumscribing rectangle to be detected), which is a rectangle circumscribing a dead tree contained in one of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12. When the damaged tree search processing unit 13A determines that one of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a dead or damaged tree, the object detection model outputs a dead or damaged tree circumscribing rectangle to be detected (damaged tree circumscribing rectangle to be detected), which is a rectangle circumscribing a dead or damaged tree contained in one of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12.

[0066] In the example shown in Figure 14, when the damaged tree search processing unit 13A determines that any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a damaged tree (infected tree, dead tree, or damaged tree), it calculates the reliability that each rectangular image RMi (i = 1 to N) contains a damaged tree (infected tree, dead tree, or damaged tree) by using an object detection model after supervised learning. When the damaged tree search processing unit 13A determines that an infected tree is included in any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12, the damaged tree position calculation processing unit 13B calculates the position information of the infected tree detected by the damaged tree detection processing unit 13 based on the infected tree circumscribing rectangle of the detected target infected tree (the damaged tree circumscribing rectangle of the detected target damaged tree) output by the object detection model. In detail, the damaged tree position calculation processing unit 13B calculates the center positions in the vertical and horizontal directions of the infected tree circumscribing rectangle of the detected target infected tree (the damaged tree circumscribing rectangle of the detected target damaged tree) output by the object detection model as the position of the infected tree detected by the damaged tree detection processing unit 13.

[0067] When the damaged tree search processing unit 13A determines that any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a dead tree, the damaged tree position calculation processing unit 13B calculates the position information of the dead tree detected by the damaged tree detection processing unit 13 based on the detection target dead tree circumscribing rectangle (detection target damaged tree circumscribing rectangle) output by the object detection model. In detail, the damaged tree position calculation processing unit 13B calculates the center positions in the vertical and horizontal directions of the detection target dead tree circumscribing rectangle (detection target damaged tree circumscribing rectangle) output by the object detection model as the position of the dead tree detected by the damaged tree detection processing unit 13. When the damaged tree search processing unit 13A determines that any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a dead or damaged tree, the damaged tree position calculation processing unit 13B calculates the position information of the dead or damaged tree detected by the damaged tree detection processing unit 13 based on the circumscribing rectangle of the dead or damaged tree to be detected (the circumscribing rectangle of the damaged tree to be detected) output by the object detection model. In detail, the damaged tree position calculation processing unit 13B calculates the center positions in the vertical and horizontal directions of the circumscribing rectangle of the dead or damaged tree to be detected (the circumscribing rectangle of the damaged tree to be detected) output by the object detection model as the position of the dead or damaged tree detected by the damaged tree detection processing unit 13.

[0068] In the example shown in Figure 14, when the damaged tree search processing unit 13A determines that any of the multiple rectangular images RMi (i = 1 to N) cut out by the image cutout processing unit 12 contains a damaged tree (infected tree, dead tree, or damaged tree), the matching processing unit 13C matches the damaged tree (infected tree, dead tree, or damaged tree) based on the position information of the damaged tree (infected tree, dead tree, or damaged tree) calculated by the damaged tree position calculation processing unit 13B. The noise removal processing unit 13D performs processing to remove damaged trees (infected trees, dead trees, and dead and damaged trees) as noise based on the reliability calculated by the damaged tree search processing unit 13A. The analysis result output processing unit 14 outputs the analysis results from the damaged tree detection processing unit 13 (for example, the position of the damaged tree within the ``range AR where damaged tree detection is performed'' shown in Figure 2, and the identification result of whether the damaged tree is an infected tree, a dead tree, or a dead and damaged tree).

[0069] Since multispectral images taken by satellites or aircraft represent reflectance spectral characteristics that differ depending on the activity of plants, there is almost no risk that the damaged tree detection processing unit 13 will mistakenly detect the ground surface as an infected tree, or that the damaged tree detection processing unit 13 will mistakenly detect the ground surface as a dead tree, or that the damaged tree detection processing unit 13 will mistakenly detect the ground surface as a dead tree. In view of this, in the example shown in FIG. 14, the automatic forest damaged tree detection device 1 of the second embodiment does not have a function equivalent to the ground height filter processing unit 15 shown in FIG. According to the second embodiment of the automatic forest damaged tree detection device 1, by utilizing multispectral images taken by a satellite or an aircraft, it is possible to determine whether or not there are damaged trees (infected trees, dead trees, and damaged trees) in the forest area being surveyed, and the locations of the damaged trees (infected trees, dead trees, and damaged trees) present in the forest area being surveyed.

[0070] <Third embodiment> A third embodiment of the automatic forest damaged tree detection device, the automatic forest damaged tree detection method, and the program of the present invention will be described below. The automatic forest damaged tree detection device 1 of the third embodiment is configured similarly to the automatic forest damaged tree detection device 1 of the first embodiment described above, except for the points described below. Therefore, the automatic forest damaged tree detection device 1 of the third embodiment can achieve the same effects as the automatic forest damaged tree detection device 1 of the first embodiment described above, except for the points described below.

[0071] The automatic forest damaged tree detection device 1 of the third embodiment is configured similarly to the automatic forest damaged tree detection device 1 of the second embodiment shown in FIG. The automatic damaged forest tree detection device 1 of the third embodiment automatically detects damaged trees in a forest area to be surveyed. In detail, the automatic damaged forest tree detection device 1 of the third embodiment, like the automatic damaged forest tree detection device 1 of the first embodiment, classifies (identifies) and detects damaged trees as dead trees and dead and damaged trees. The forest damaged tree automatic detection device 1 (see FIG. 14) of the third embodiment includes a data input processing unit 11, an image clipping processing unit 12, a damaged tree detection processing unit 13, and an analysis result output processing unit . The data input processing unit 11 accepts input of aerial image data, including visible light range orthoimage data, which is an orthoimage generated from an RGB color image in the visible light range taken using a drone, and multispectral orthoimage data, which is an orthoimage generated from a multispectral image taken by a satellite or aircraft. The data input processing unit 11 also receives input of detection range designation information, which is information that designates the "area AR in which damaged tree detection is performed" included in the visible light range orthoimage and the multispectral orthoimage. The image cutout processing unit 12 cuts out a plurality of rectangular images from the visible light range orthoimage, and cuts out a plurality of rectangular images from the multispectral orthoimage.

[0072] The damaged tree detection processing unit 13 detects damaged trees (dead trees and dead and damaged trees) included in each of a plurality of rectangular images cut out from the visible light range orthoimage, and also detects damaged trees (dead trees and dead and damaged trees) included in each of a plurality of rectangular images cut out from the multispectral orthoimage. The damaged tree detection processing unit 13 includes a damaged tree search processing unit 13A, a damaged tree position calculation processing unit 13B, a matching processing unit 13C, and a noise removal processing unit 13D. The damaged tree search processing unit 13A performs supervised learning of a first object detection model using training data of visible light range orthoimages, and then, by using the first object detection model after supervised learning, determines whether or not each of the multiple rectangular images cut out from the visible light range orthoimages by the image cutout processing unit 12 contains a damaged tree (dead tree, dead tree). In addition, the damaged tree search processing unit 13A performs supervised learning of a second object detection model using training data of the multispectral orthoimage, and then, by using the second object detection model after supervised learning, determines whether or not each of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12 contains a damaged tree (dead tree, dead tree).

[0073] In one example of the third embodiment of the automatic forest damaged tree detection device 1, the training data used for supervised learning of the first object detection model includes a training data visible light range orthoimage including dead trees and dead and damaged trees, labeling ``dead tree'' applied to a training data dead tree circumscribing rectangle (training data damaged tree circumscribing rectangle) that circumscribing the dead tree included in the training data visible light range orthoimage, and labeling ``dead tree'' applied to a training data dead and damaged tree circumscribing rectangle (training data damaged tree circumscribing rectangle) that circumscribing the dead and damaged tree included in the training data visible light range orthoimage. The training data used for supervised learning of the second object detection model includes multispectral orthoimages for training data that include dead and damaged trees, labeling "dead tree" applied to the dead tree circumscribing rectangle for training data (damaged tree circumscribing rectangle for training data) that circumscribing the dead tree included in the multispectral orthoimages for training data, and labeling "dead tree" applied to the dead tree circumscribing rectangle for training data (damaged tree circumscribing rectangle for training data) that circumscribing the dead tree included in the visible light range orthoimages for training data.

[0074] The damaged tree search processing unit 13A determines whether or not each of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12 contains a dead tree, and determines whether or not each of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12 contains a dead or damaged tree. In addition, the damaged tree search processing unit 13A determines whether or not each of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12 contains a dead tree, and determines whether or not each of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12 contains a dead or damaged tree. When the damaged tree search processing unit 13A determines that a dead tree is included in any of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12, the first object detection model outputs a dead tree circumscribing rectangle to be detected (damaged tree circumscribing rectangle to be detected), which is a rectangle circumscribing a dead tree included in any of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12. When the damaged tree search processing unit 13A determines that a dead tree is included in any of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12, the first object detection model outputs a dead tree circumscribing rectangle to be detected (damaged tree circumscribing rectangle to be detected), which is a rectangle circumscribing a dead tree included in any of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12. When the damaged tree search processing unit 13A determines that a dead tree is included in any of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12, the second object detection model outputs a dead tree circumscribing rectangle to be detected (damaged tree circumscribing rectangle to be detected), which is a rectangle circumscribing a dead tree included in any of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12. When the damaged tree search processing unit 13A determines that a dead tree is included in any of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12, the second object detection model outputs a dead tree circumscribing rectangle to be detected (damaged tree circumscribing rectangle to be detected), which is a rectangle circumscribing a dead tree included in any of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12.

[0075] When the damaged tree search processing unit 13A determines that any of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12 contains a damaged tree (dead tree, dead or damaged tree), it calculates the reliability that each rectangular image contains a damaged tree (dead tree, dead or damaged tree) by using the first object detection model after supervised learning. In addition, when the damaged tree search processing unit 13A determines that any of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12 contains a damaged tree (dead tree, dead or damaged tree), it calculates the reliability that each rectangular image contains a damaged tree (dead tree, dead or damaged tree) by using the second object detection model after supervised learning. The damaged tree position calculation processing unit 13B calculates the position information of damaged trees (dead trees, dead and damaged trees) determined by the damaged tree search processing unit 13A to be included in any of multiple rectangular images cut out from the visible light range orthoimage, and the position information of damaged trees (dead trees, dead and damaged trees) determined by the damaged tree search processing unit 13A to be included in any of multiple rectangular images cut out from the multispectral orthoimage.

[0076] If the distance between the position of a damaged tree (dead tree, dead or damaged tree) included in any of the multiple rectangular images cut out from the visible light range orthoimage and the position of a damaged tree (dead tree, dead or damaged tree) included in any of the multiple rectangular images cut out from the multispectral orthoimage is within a first threshold (e.g., 3 m), the damaged tree position calculation processing unit 13B considers the damaged tree (dead tree, dead or damaged tree) included in any of the multiple rectangular images cut out from the visible light range orthoimage and the damaged tree (dead tree, dead or damaged tree) included in any of the multiple rectangular images cut out from the multispectral orthoimage to be the same damaged tree (dead tree, dead or damaged tree). In addition, the damaged tree position calculation processing unit 13B adopts either the position information of a damaged tree (dead tree, dead or damaged tree) contained in one of multiple rectangular images cut out from the visible light range orthoimage, or the position information of a damaged tree (dead tree, dead or damaged tree) contained in one of multiple rectangular images cut out from the multispectral orthoimage, whichever has the higher reliability calculated by the damaged tree search processing unit 13A. The noise removal processing unit 13D performs processing to remove, as noise, the location information of damaged trees (dead trees, dead and damaged trees) contained in one of multiple rectangular images extracted from a visible light range orthoimage, and the location information of damaged trees (dead trees, dead and damaged trees) contained in one of multiple rectangular images extracted from a multispectral orthoimage, whichever has a lower reliability calculated by the damaged tree search processing unit 13A.

[0077] When the damaged tree search processing unit 13A determines that any of the multiple rectangular images cut out from the visible light range orthoimage by the image cutout processing unit 12 contains a damaged tree (dead tree, dead or damaged tree), the matching processing unit 13C matches the damaged tree (dead tree, dead or damaged tree) based on the position information of the damaged tree (dead tree, dead or damaged tree) calculated by the damaged tree position calculation processing unit 13B. In addition, when the damaged tree search processing unit 13A determines that any of the multiple rectangular images cut out from the multispectral orthoimage by the image cutout processing unit 12 contains a damaged tree (dead tree, dead or damaged tree), the matching processing unit 13C matches the damaged tree (dead tree, dead or damaged tree) based on the position information of the damaged tree (dead tree, dead or damaged tree) calculated by the damaged tree position calculation processing unit 13B. The analysis result output processing unit 14 outputs the analysis results from the damaged tree detection processing unit 13 (for example, the position of the damaged tree within the ``range AR where damaged tree detection is performed'' shown in Figure 2, and the identification result of whether the damaged tree is a dead tree or a dead and damaged tree).

[0078] Although the present invention has been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. The configurations described in the above-described embodiments and examples may be combined.

[0079] In addition, all or part of the functions of each unit of the automatic forest damaged tree detection device 1 in the above-mentioned embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, and then loading and executing the program recorded on this recording medium into a computer system. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage units such as hard disks built into computer systems. Furthermore, "computer-readable recording media" may also include devices that dynamically store programs for a short period of time, such as communication lines when transmitting programs over networks like the Internet or communication lines like telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within computer systems that serve as servers or clients in such cases. Furthermore, the above-mentioned programs may be programs that realize some of the aforementioned functions, or may be programs that can realize the aforementioned functions in combination with programs already stored in the computer system. [Explanation of symbols]

[0080] 1...Automatic forest damaged tree detection device, 11...Data input processing unit, 12...Image clipping processing unit, 13...Damaged tree detection processing unit, 13A...Damaged tree search processing unit, 13B...Damaged tree position calculation processing unit, 13C...Matching processing unit, 13D...Noise removal processing unit, 14...Analysis result output processing unit, 15...Ground height filter processing unit, AM...Aerial image, AR...Range for damaged tree detection, DT...Damaged tree, DTA...Dead tree, DTB...Dead and damaged tree, DER...Circumscribed rectangle of damaged tree to be detected, TM...Aerial image for training data, TER...Circumscribed rectangle of damaged tree for training data

Claims

1. An automatic forest damaged tree detection device that automatically detects damaged trees in a surveyed forest area, a data input processing unit that receives input of aerial image data of the forest area to be surveyed and detection range designation information that designates the range within which damaged trees included in the aerial image are to be detected; an image cropping processor that crops out a plurality of rectangular images from the aerial image by generating a grid within the range where damaged trees are detected, which is specified by the detection range designation information; a damaged tree detection processing unit that detects damaged trees included in each of the plurality of rectangular images cut out by the image cutout processing unit, The damaged tree detection processing unit a damaged tree search processing unit that determines whether or not a damaged tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; a damaged tree position calculation processing unit that calculates the position information of the damaged tree when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a damaged tree, The damaged tree search processing unit Perform supervised learning of the object detection model using training data, Using the object detection model after supervised learning, it is determined whether or not each of the plurality of rectangular images cut out by the image cutout processing unit includes a damaged tree; The training data used in the supervised learning of the object detection model includes a training data aerial image including a damaged tree, and labeling applied to a training data damaged tree circumscribing rectangle, which is a rectangle circumscribing the damaged tree included in the training data aerial image, When the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a damaged tree, the object detection model outputs a target damaged tree circumscribing rectangle, which is a rectangle circumscribing a damaged tree included in any of the multiple rectangular images cut out by the image cutout processing unit; the damaged tree position calculation processing unit calculates position information of the damaged tree detected by the damaged tree detection processing unit based on the circumscribing rectangle of the detected damaged tree output by the object detection model, the damaged tree position calculation processing unit calculates the center positions in the vertical and horizontal directions of the circumscribing rectangle of the detected damaged tree output by the object detection model as the position of the damaged tree detected by the damaged tree detection processing unit; The image cropping processing unit crops out a plurality of rectangular images from the aerial image so that a part of one of two rectangular images adjacent in the vertical direction overlaps with a part of the other of the rectangular images, and a part of one of two rectangular images adjacent in the horizontal direction overlaps with a part of the other of the rectangular images. Automatic detection device for damaged forest trees.

2. The damaged tree detection processing unit a matching processing unit that performs matching of damaged trees based on the position information of the damaged trees calculated by the damaged tree position calculation processing unit when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a damaged tree; The matching processing unit If the distance between the position of a damaged tree included in one of two vertically adjacent rectangular images and the position of a damaged tree included in the other of the two vertically adjacent rectangular images is within a first threshold, the damaged tree included in one of the two vertically adjacent rectangular images and the damaged tree included in the other of the two vertically adjacent rectangular images are considered to be the same damaged tree, If the distance between the position of a damaged tree included in one of two horizontally adjacent rectangular images and the position of a damaged tree included in the other of the two horizontally adjacent rectangular images is within the first threshold value, the damaged tree included in one of the two horizontally adjacent rectangular images and the damaged tree included in the other of the two horizontally adjacent rectangular images are considered to be the same damaged tree. The automatic forest damaged tree detection device according to claim 1.

3. a vertical overlap width, which is an overlap width between a part of one of two rectangular images adjacent in the vertical direction and a part of the other, is greater than the first threshold value, and a horizontal overlap width, which is an overlap width between a part of one of two rectangular images adjacent in the horizontal direction and a part of the other, is greater than the first threshold value; 3. The automatic forest damaged tree detection device according to claim 2.

4. the damaged tree search processing unit calculates a reliability that a damaged tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit by using the object detection model after supervised learning; the damaged tree detection processing unit includes a noise removal processing unit, When the damaged tree search processing unit determines that a damaged tree is included in any of the multiple rectangular images cut out by the image cutout processing unit, and the reliability calculated by the damaged tree search processing unit is equal to or less than a second threshold, the noise removal processing unit removes the damaged tree having a reliability equal to or less than the second threshold as noise. The automatic forest damaged tree detection device according to claim 3.

5. The aerial image data input by the data input processing unit is orthoimage data generated from an RGB color image in the visible light range captured using a drone, the damaged tree search processing unit determines whether or not a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit, and determines whether or not a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; The damaged tree position calculation processing unit calculates location information of dead trees when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a dead tree; calculates location information of dead and damaged trees when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a dead and damaged tree; training data used for supervised learning of the object detection model includes the training data aerial images including dead and damaged trees, or the training data aerial images including dead trees and the training data aerial images including damaged trees, labeling applied to the training data dead tree circumscribing rectangle which is a rectangle circumscribing the dead tree included in the training data aerial images, and labeling applied to the training data dead tree circumscribing rectangle which is a rectangle circumscribing the dead tree included in the training data aerial images; When the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a dead tree and / or a damaged tree, the object detection model outputs a detection target dead tree circumscribing rectangle, which is a rectangle circumscribing a dead tree and / or a detection target dead and damaged tree circumscribing rectangle, which is a rectangle circumscribing a dead or damaged tree included in any of the multiple rectangular images cut out by the image cutout processing unit; The damaged tree position calculation processing unit Calculating position information of the dead tree detected by the damaged tree detection processing unit based on the circumscribing rectangle of the dead tree to be detected output by the object detection model; Calculating position information of the dead tree detected by the damaged tree detection processing unit based on the circumscribing rectangle of the dead tree to be detected output by the object detection model; a ground height filter processing unit that determines whether the detection result of the damaged tree detection processing unit is a false detection or not; The ground height filtering unit Acquire digital elevation model data of the forest area to be surveyed and digital surface model data of the forest area to be surveyed; determining whether the damaged tree detection processing unit has erroneously detected the ground surface as a dead tree and / or a dead tree based on the detection result of the dead tree and / or a dead tree by the damaged tree detection processing unit, the digital elevation model data, and the digital surface model data; 5. The automatic forest damaged tree detection device according to claim 4.

6. the aerial image data input by the data input processing unit is orthoimage data generated from multispectral images taken by an artificial satellite or an aircraft, the damaged tree search processing unit executes a determination as to whether or not an infected tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit, a determination as to whether or not a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit, and a determination as to whether or not a dead tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit, The damaged tree position calculation processing unit calculates location information of the infected tree when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes an infected tree; calculates location information of the dead tree when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a dead tree; calculates location information of the dead tree when the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes a dead or damaged tree; the training data used for supervised learning of the object detection model includes the training data aerial images including infected trees, dead trees, and dead or damaged trees, labeling applied to training data infected tree circumscribing rectangles which are rectangles circumscribing infected trees included in the training data aerial images, labeling applied to training data dead tree circumscribing rectangles which are rectangles circumscribing dead trees included in the training data aerial images, and labeling applied to training data dead tree circumscribing rectangles which are rectangles circumscribing dead trees included in the training data aerial images, When the damaged tree search processing unit determines that any of the multiple rectangular images cut out by the image cutout processing unit includes an infected tree, a dead tree, and / or a damaged tree, The object detection model outputs a detection target infected tree circumscribing rectangle, which is a rectangle circumscribing an infected tree included in any of the multiple rectangular images cut out by the image cutout processing unit, a detection target dead tree circumscribing rectangle, which is a rectangle circumscribing a dead tree, and / or a detection target dead tree circumscribing rectangle, which is a rectangle circumscribing a dead tree; The damaged tree position calculation processing unit Calculating position information of the infected tree detected by the damaged tree detection processing unit based on the circumscribing rectangle of the infected tree to be detected output by the object detection model; Calculating position information of the dead tree detected by the damaged tree detection processing unit based on the circumscribing rectangle of the dead tree to be detected output by the object detection model; Calculating position information of the dead tree detected by the damaged tree detection processing unit based on the circumscribing rectangle of the dead tree to be detected output by the object detection model.

5. The automatic forest damaged tree detection device according to claim 4.

7. The data input processing unit receives input of data of the aerial image, including data of a visible light range orthoimage, which is an orthoimage generated from an RGB color image in the visible light range photographed using a drone, and data of a multispectral orthoimage, which is an orthoimage generated from a multispectral image photographed by an artificial satellite or an aircraft; the image cutout processing unit cuts out a plurality of rectangular images from the visible light range orthoimage and cuts out a plurality of rectangular images from the multispectral orthoimage; The damaged tree search processing unit determining whether or not a damaged tree is included in each of a plurality of rectangular images extracted from the visible light range orthoimage, and calculating a reliability that a damaged tree is included in each of the plurality of rectangular images extracted from the visible light range orthoimage; determining whether or not a damaged tree is included in each of a plurality of rectangular images extracted from the multispectral orthoimage, and calculating a reliability that a damaged tree is included in each of the plurality of rectangular images extracted from the multispectral orthoimage; The damaged tree position calculation processing unit Calculating position information of damaged trees determined by the damaged tree search processing unit to be included in any of a plurality of rectangular images cut out from the visible light range orthoimage, and position information of damaged trees determined by the damaged tree search processing unit to be included in any of a plurality of rectangular images cut out from the multispectral orthoimage; If the distance between the position of the damaged tree included in one of the multiple rectangular images extracted from the visible light range orthoimage and the position of the damaged tree included in one of the multiple rectangular images extracted from the multispectral orthoimage is within the first threshold, the damaged tree included in one of the multiple rectangular images extracted from the visible light range orthoimage and the damaged tree included in one of the multiple rectangular images extracted from the multispectral orthoimage are considered to be the same damaged tree, and one of the position information of the damaged tree included in one of the multiple rectangular images extracted from the visible light range orthoimage and the position information of the damaged tree included in one of the multiple rectangular images extracted from the multispectral orthoimage, whichever has a higher reliability calculated by the damaged tree search processing unit, is adopted. The automatic forest damaged tree detection device according to claim 3.

8. A method for automatically detecting damaged trees in a forest area to be surveyed, comprising: a data input processing step of receiving input of aerial image data of the forest area to be surveyed and detection range designation information, which is information designating the range within which damaged trees included in the aerial image are to be detected; an image cropping process step of cropping a plurality of rectangular images from the aerial image by generating a grid within the range where damaged trees are detected, which is specified by the detection range designation information; a damaged tree detection processing step for detecting damaged trees included in each of the plurality of rectangular images extracted in the image extraction processing step, The damaged tree detection processing step includes: a damaged tree search processing step for determining whether or not a damaged tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; a damaged tree position calculation step for calculating position information of the damaged tree when it is determined in the damaged tree search step that any of the plurality of rectangular images cut out in the image cutout step is included, In the damaged tree search processing step, Supervised learning of the object detection model using training data is performed. Using the object detection model after supervised learning, it is determined whether or not each of the plurality of rectangular images extracted in the image extraction processing step includes a damaged tree; The training data used in the supervised learning of the object detection model includes a training data aerial image including a damaged tree, and labeling applied to a training data damaged tree circumscribing rectangle, which is a rectangle circumscribing the damaged tree included in the training data aerial image, When it is determined in the damaged tree search processing step that any of the plurality of rectangular images cut out in the image cutout processing step includes a damaged tree, the object detection model outputs a detection target damaged tree circumscribing rectangle, which is a rectangle circumscribing a damaged tree included in any of the multiple rectangular images extracted in the image extraction processing step; In the damaged tree position calculation processing step, position information of the damaged tree detected in the damaged tree detection processing step is calculated based on the circumscribing rectangle of the detected damaged tree output by the object detection model, In the damaged tree position calculation processing step, the center positions in the vertical and horizontal directions of the circumscribing rectangle of the detected damaged tree output by the object detection model are calculated as the position of the damaged tree detected in the damaged tree detection processing step; The image cropping process step crops a plurality of rectangular images from the aerial image such that a portion of one of two rectangular images adjacent in the vertical direction overlaps a portion of the other of the rectangular images, and a portion of one of two rectangular images adjacent in the horizontal direction overlaps a portion of the other of the rectangular images. A method for automatically detecting damaged trees in forests.

9. On the computer, a data input processing step of receiving input of aerial image data of the forest area to be surveyed and detection range designation information, which is information designating the range within which damaged trees included in the aerial image are to be detected; an image cropping process step of cropping a plurality of rectangular images from the aerial image by generating a grid within the range where damaged trees are detected, which is specified by the detection range designation information; a damaged tree detection processing step for detecting damaged trees included in each of the plurality of rectangular images extracted in the image extraction processing step, The damaged tree detection processing step includes: a damaged tree search processing step for determining whether or not a damaged tree is included in each of the plurality of rectangular images cut out by the image cutout processing unit; a damaged tree position calculation step for calculating position information of the damaged tree when it is determined in the damaged tree search step that any of the plurality of rectangular images cut out in the image cutout step is included, In the damaged tree search processing step, Supervised learning of the object detection model using training data is performed. Using the object detection model after supervised learning, it is determined whether or not each of the plurality of rectangular images extracted in the image extraction processing step includes a damaged tree; The training data used in the supervised learning of the object detection model includes a training data aerial image including a damaged tree, and labeling applied to a training data damaged tree circumscribing rectangle, which is a rectangle circumscribing the damaged tree included in the training data aerial image, When it is determined in the damaged tree search processing step that any of the plurality of rectangular images cut out in the image cutout processing step includes a damaged tree, the object detection model outputs a detection target damaged tree circumscribing rectangle, which is a rectangle circumscribing a damaged tree included in any of the multiple rectangular images extracted in the image extraction processing step; In the damaged tree position calculation processing step, position information of the damaged tree detected in the damaged tree detection processing step is calculated based on the circumscribing rectangle of the detected damaged tree output by the object detection model, In the damaged tree position calculation processing step, the center positions in the vertical and horizontal directions of the circumscribing rectangle of the detected damaged tree output by the object detection model are calculated as the position of the damaged tree detected in the damaged tree detection processing step; The image cropping process step crops a plurality of rectangular images from the aerial image such that a portion of one of two rectangular images adjacent in the vertical direction overlaps a portion of the other of the rectangular images, and a portion of one of two rectangular images adjacent in the horizontal direction overlaps a portion of the other of the rectangular images. program.

Images