Plant automatic detection device, plant automatic detection method, and program

Abstract

To automatically detect a planted tree with high accuracy.SOLUTION: A planted tree automatic detection device creates a preprocessed aerial photographed image data from aerial photographed color image data of a forest area, removes noise included in the preprocessed aerial photographed image data to create noise-removed aerial photographed image data, detects planted tree candidate points based on the preprocessed aerial photographed image data and the noise-removed aerial photographed image data to create planted tree candidate point detection image data including the planted tree candidate points, extracts a feature amount from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data to create planted tree detection data, and detects a planted tree included in the aerial photographed color image data based on the planted tree detection data. The planted tree automatic tree detection device divides the planted tree detection data into estimation data and training data, constructs an estimation model by supervised learning using the training data, evaluates the accuracy of the estimation model, and performs the estimation of the planted tree included in the estimation model using the estimation model selected based on an accuracy evaluation result of the estimation model.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a planted tree automatic detection device, a planted tree automatic detection method, and a program. Specifically, the present invention relates to a planted tree automatic detection device, a planted tree automatic detection method, and a program that highly accurately extract planted trees in a forest area to be surveyed based on image data taken from above by a drone, an aircraft, etc., or laser measurement data and image data (for example, ortho-image data) using artificial intelligence (AI).

Background Art

[0002] In forestry, the artificial forests of cedar and cypress planted after the war have reached the harvest period, and the number of reforested areas after harvest has increased rapidly throughout the country. Since the afforestation project is a subsidized project, it is necessary for people to investigate the location, number, and growth status of the planted trees based on the planting plan. Underbrush cutting, which is dangerous of being stung by bees in June and July, is carried out, and the planted area, number of trees, and withering are inspected. However, due to the rapid increase in area, the Forestry Agency, prefectures, municipalities, and forest cooperatives are short of manpower, which has become a major problem to be solved at the production site.

[0003] In the inspection of the planting work of such afforestation subsidy projects, the survival inspection (confirmation) of the planted trees is carried out in summer. It is confirmed whether the forestry (afforestation) workers planted along the planting plan. In the extraction inspection, multiple extraction inspection locations with an area of 0.02 ha are set for the target area, and the number of planted trees, the mortality rate, and the damage rate in the underbrush cutting work are investigated. Amid the danger of being stung by bees and the labor required during the hot summer season for preliminary investigations for confirmation and inspection, the burden on forest officers, local government employees, and forestry business employees at the site is heavy.

[0004] Although the image data obtained by aircraft or drone measurement has a high-definition ground resolution of several centimeters to several tens of centimeters, when viewed directly from above, the planted trees are small objects with a size of 10 centimeters to 20 centimeters, and it has been considered difficult to detect the planted trees in the image among the overgrown weeds.

[0005] In recent years, a technique has been proposed for creating forest resource information regarding a target forest area based on laser measurement data obtained by irradiating a region including a predetermined forest area (also referred to as the "target forest area") from above (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] Patent Document 1 describes a forest resource information calculation method for creating forest resource information regarding a target forest area based on laser measurement data obtained by irradiating a predetermined forest area with laser light, and a method for highly accurately calculating forest resource information such as the crown diameter, breast height diameter, and tree height of each tree, and the volume obtained from these tree heights and breast height diameters from three-dimensional laser point cloud data is described. However, in the technique described in Patent Document 1, although information on mature trees included in an aerial photograph can be calculated, it is not possible to detect extremely small objects surrounded by weeds, such as planted trees.

[0008] Patent Document 2 describes a method for growing rooted trees, in which a medium in which a tree is planted is placed in a plant cultivation facility capable of adjusting the temperature, the roots of the tree are immersed in a flowing culture solution, and dormancy release is performed on dormant trees by controlling the environmental temperature in the plant cultivation facility. In order to shorten the period of planted tree production, in the technique described in Patent Document 2, an artificial light illumination device for irradiating artificial light is provided in the plant cultivation facility, and during dormancy release, the time of the light period of the tree is controlled using the artificial light illumination device. By the way, the technology described in Patent Document 2 cannot be used for detecting planted trees in a forest or for growth diagnosis.

[0009] In view of the above problems, an object of the present invention is to provide a planted tree automatic detection device, a planted tree automatic detection method, and a program that can automatically detect planted trees with high accuracy in a predetermined forest area. Specifically, an object of the present invention is to provide a general-purpose planted tree automatic detection device, a planted tree automatic detection method, and a program that can scientifically and efficiently automatically detect planted trees from color image data taken from above by an aircraft, a drone, etc. in the entire area or an arbitrary range of a predetermined forest area using artificial intelligence (AI). That is, an object of the present invention is to provide a planted tree automatic detection device, a planted tree automatic detection method, and a program that can automatically perform the work of confirming planted trees that requires manual labor and cost, and can automatically confirm and diagnose the growth of planted trees from aerial photography color image data by a drone or the like.

Means for Solving the Problems

[0010] In this specification, a "planted tree" refers to a sapling of a tree grown from a seed and planted in a forest. Planted trees include, for example, tall trees such as cedar, cypress, and larch. The forest area to be surveyed is a forest area where a forest formed by these trees to be surveyed has a predetermined area. Since planted trees are suppressed by surrounding weeds and their growth is inhibited, it is necessary to mow the weeds by undercutting work every summer. The inspection period of the afforestation work is carried out together with the undercutting work for a period of approximately 7 to 10 years until the planted trees become taller than the weeds.

[0011] One aspect of the present invention is a planted tree automatic detection device that automatically detects planted trees in a predetermined forest area. The device includes an aerial image data preprocessing unit that creates preprocessed aerial image data by performing preprocessing on aerial color image data of the forest area, a noise removal processing unit that creates noise-removed aerial image data by removing noise included in the preprocessed aerial image data created by the aerial image data preprocessing unit, a planted tree candidate point detection processing unit that detects planted tree candidate points based on the preprocessed aerial image data and the noise-removed aerial image data, and creates planted tree candidate point detection image data that is aerial image data including the planted tree candidate points, a feature amount extraction processing unit that extracts feature amounts from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created by the planted tree candidate point detection processing unit, and creates data for planted tree detection, a planted tree detection processing unit that detects the planted trees included in the aerial color image data based on the data for planted tree detection created by the feature amount extraction processing unit, a planted tree information creation processing unit that creates information regarding the planted trees detected by the planted tree detection processing unit, and a planted tree information aggregation processing unit that aggregates information regarding all the planted trees included in the forest area, which is the information regarding the planted trees created by the planted tree information creation processing unit. The planted tree detection processing unit includes a data division processing unit that divides the data for planted tree detection created by the feature amount extraction processing unit into estimation data that is data used for estimating the planted trees and training data that is data used for constructing an estimation model of the planted trees, an estimation model construction processing unit that constructs the estimation model by performing supervised learning using the training data, an accuracy evaluation processing unit that performs an accuracy evaluation of the estimation model constructed by the estimation model construction processing unit, and a planted tree estimation processing unit that estimates the planted trees included in the estimation data by using a selected estimation model that is an estimation model selected based on the result of the accuracy evaluation of the estimation model performed by the accuracy evaluation processing unit. It is a planted tree automatic detection device.

[0012] In the planted tree automatic detection device according to one aspect of the present invention, the aerial image data preprocessing unit includes an aerial image data input processing unit that receives the input of the aerial color image data, a three-dimensional point cloud data creation processing unit that creates three-dimensional point cloud data from the aerial color image data received by the aerial image data input processing unit, and a tree crown height image data creation processing unit that creates the tree crown height image data as the preprocessed aerial image data from the three-dimensional point cloud data created by the three-dimensional point cloud data creation processing unit. The planted tree candidate point detection processing unit may detect the planted tree candidate points by performing a difference process between images in which the tree crown height image data as the preprocessed aerial image data and the aerial image data after noise removal are overlapped and differenced.

[0013] In the planted tree automatic detection device according to one aspect of the present invention, the feature amount extraction processing unit creates a circle with a radius of 10 centimeters centered on the planted tree candidate points included in the planted tree candidate point detection image data created by the planted tree candidate point detection processing unit as the predetermined region, and the feature amounts extracted from the predetermined region may include the sum, maximum value, minimum value, average value, range, median, minimum frequency value, standard deviation, variance, and statistical values of the manifold for each of the RGB values, normalized RGB values, and tree crown height values within the predetermined region.

[0014] In the planted tree automatic detection device according to one aspect of the present invention, the data division processing unit may divide 80% of the planted tree detection data created by the feature amount extraction processing unit into the estimation data and divide 20% of the planted tree detection data created by the feature amount extraction processing unit into the training data.

[0015] In the planted tree automatic detection device according to one aspect of the present invention, the estimation model construction processing unit may construct the estimation model by eliminating the feature amounts that do not contribute to the performance of the estimation model among the feature amounts included in the training data.

[0016] In the planted tree automatic detection device according to one aspect of the present invention, the accuracy evaluation processing unit divides the training data into k parts, uses (k - 1) / k of the training data as learning data, uses 1 / k of the training data as verification data, and may use the k-fold cross-validation method to perform the accuracy evaluation of the estimation model while changing the combination of the learning data and the verification data.

[0017] In the planted tree automatic detection device according to one aspect of the present invention, the planted tree estimation processing unit may estimate the planted trees included in the estimation data by using the selected post-estimation model, which is the estimation model that obtained the highest accuracy evaluation result in the accuracy evaluation of the estimation model executed by the accuracy evaluation processing unit.

[0018] In the planted tree automatic detection device according to one aspect of the present invention, the information on the planted trees created by the planted tree information creation processing unit may at least include the ID (identifier) of the planted trees detected by the planted tree detection processing unit, the information indicating the standing tree positions of the planted trees detected by the planted tree detection processing unit, and the information indicating the seedling heights of the planted trees detected by the planted tree detection processing unit.

[0019] In the planted tree automatic detection device according to one aspect of the present invention, the information on all the planted trees included in the forest area aggregated by the planted tree information aggregation processing unit includes the total number of all the planted trees included in the forest area and the number of planted trees per unit area of the forest area, and the planted tree information aggregation processing unit may have a function of registering the aggregated information in a database.

[0020] One aspect of the present invention is a method for automatically detecting planted trees in a predetermined forest area, comprising: an aerial image data preprocessing step of creating preprocessed aerial image data by performing preprocessing on aerial color image data of the forest area; a noise removal processing step of creating noise-removed aerial image data by removing noise included in the preprocessed aerial image data created in the aerial image data preprocessing step; a planted tree candidate point detection processing step of detecting planted tree candidate points based on the preprocessed aerial image data and the noise-removed aerial image data, and creating planted tree candidate point detection image data which is aerial image data including the planted tree candidate points; a feature amount extraction processing step of extracting feature amounts from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created in the planted tree candidate point detection processing step, and creating data for planted tree detection; a planted tree detection processing step of detecting the planted trees included in the aerial color image data based on the data for planted tree detection created in the feature amount extraction processing step; a planted tree information creation processing step of creating information regarding the planted trees detected in the planted tree detection processing step; and a planted tree information aggregation processing step of aggregating the information regarding the planted trees created in the planted tree information creation processing step, which is information regarding all the planted trees included in the forest area. The planted tree detection processing step includes: a data division processing step of dividing the data for planted tree detection created in the feature amount extraction processing step into estimation data which is data used for estimating the planted trees and training data which is data used for constructing an estimation model of the planted trees; an estimation model construction processing step of constructing the estimation model by performing supervised learning using the training data; an accuracy evaluation processing step of performing accuracy evaluation of the estimation model constructed in the estimation model construction processing step; and a planted tree estimation processing step of estimating the planted trees included in the estimation data by using a selected estimation model which is an estimation model selected based on the result of the accuracy evaluation of the estimation model performed in the accuracy evaluation processing step.

[0021] One aspect of the present invention is to cause a computer to execute an aerial image data preprocessing step of creating preprocessed aerial image data by preprocessing aerial color image data of a predetermined forest area, a noise removal processing step of creating noise-removed aerial image data by removing noise included in the preprocessed aerial image data created in the aerial image data preprocessing step, a planted tree candidate point detection processing step of detecting planted tree candidate points based on the preprocessed aerial image data and the noise-removed aerial image data and creating planted tree candidate point detection image data which is the aerial image data including the planted tree candidate points, a feature amount extraction processing step of extracting feature amounts from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created in the planted tree candidate point detection processing step and creating data for planted tree detection, a planted tree detection processing step of detecting planted trees included in the aerial color image data based on the data for planted tree detection created in the feature amount extraction processing step, a planted tree information creation processing step of creating information regarding the planted trees detected in the planted tree detection processing step, and a planted tree information aggregation processing step of aggregating information regarding all the planted trees included in the forest area, which is the information regarding the planted trees created in the planted tree information creation processing step, and is a program for causing the computer to execute the steps. The planted tree detection processing step includes a data division processing step of dividing the data for planted tree detection created in the feature amount extraction processing step into estimation data which is data used for estimating the planted trees and training data which is data used for constructing an estimation model of the planted trees, an estimation model construction processing step of constructing the estimation model by performing supervised learning using the training data, an accuracy evaluation processing step of performing accuracy evaluation of the estimation model constructed in the estimation model construction processing step, and a planted tree estimation processing step of estimating the planted trees included in the estimation data by using a selected estimation model which is an estimation model selected based on the result of the accuracy evaluation of the estimation model performed in the accuracy evaluation processing step.

Advantages of the Invention

[0022] According to the present invention, it is possible to provide a planted tree automatic detection device, a planted tree automatic detection method, and a program capable of accurately automatically detecting planted trees in a predetermined forest area. Specifically, according to the present invention, it is possible to scientifically and efficiently automatically detect planted trees from color image data taken from above by an aircraft, a drone, etc. in the entire area or an arbitrary range of a predetermined forest area using artificial intelligence (AI), and to provide a general-purpose planted tree automatic detection device, a planted tree automatic detection method, and a program. That is, according to the present invention, it is possible to automatically perform the work of confirming planted trees that requires manpower and cost, and for example, it is possible to provide a planted tree automatic detection device, a planted tree automatic detection method, and a program capable of automatically confirming planted trees and diagnosing growth from aerial photography color image data by a drone or the like.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0024] Hereinafter, embodiments of the planted tree automatic detection device, the planted tree automatic detection method, and the program of the present invention will be described.

[0025] <First Embodiment> FIG. 1 is a diagram showing an example of the planted tree automatic detection device 1 of the first embodiment. In the example shown in FIG. 1, the planted tree automatic detection device 1 automatically detects planted trees in a predetermined forest area. Specifically, the planted tree automatic detection device 1 automatically detects planted trees included in aerial color image data photographed from an aircraft or a drone so as to include the forest area. The planted tree automatic detection device 1 includes an aerial image data preprocessing unit 11, a noise removal processing unit 12, a planted tree candidate point detection processing unit 13, a feature amount extraction processing unit 14, a planted tree detection processing unit 15, a planted tree information creation processing unit 16, and a planted tree information aggregation processing unit 17.

[0026] The aerial image data preprocessing unit 11 creates preprocessed aerial image data by performing preprocessing on the aerial color image data of the forest area. The aerial image data preprocessing unit 11 includes an aerial image data input processing unit 11A, a three-dimensional point cloud data creation processing unit 11B, and a tree crown height image data creation processing unit 11C. The aerial image data input processing unit 11A receives the input of aerial color image data photographed from a drone or an aircraft so as to include the above-described forest area (survey target forest area).

[0027] FIG. 2 is a diagram for explaining an example of the aerial color image data received by the aerial image data input processing unit 11A. Specifically, FIG. 2 shows a state in which an aerial color image of the survey target forest area photographed from a drone is displayed on a display (not shown). In FIG. 2, the ranges surrounded by the frames indicated by reference numerals "AR1" and "AR2" indicate the ranges of the planted areas of the accuracy verification areas by on-site surveys. The strip-shaped portions (indicated by reference numeral "HW") within the frames indicated by reference numerals "AR1" and "AR2" are the locations (accumulation locations) where the forest residues of the logging traces were accumulated by a bulldozer, and broad-leaved trees are growing. The portion (indicated by reference numeral "PT") between the accumulation locations HW and the accumulation location HW within the frames indicated by reference numerals "AR1" and "AR2" indicates a planted area (a location where planted trees may exist).

[0028] In the example shown in FIG. 1, the three-dimensional point cloud data creation processing unit 11B creates three-dimensional point cloud data from the aerial color image data received by the aerial image data input processing unit 11A. As a technique for creating three-dimensional point cloud data from color image data, for example, a known technique corresponding to the following URL can be used. https: / / psgsv2.gsi.go.jp / koukyou / download / danmen_manual_190329.pdf https: / / www.hitachicm.com / global / jp / solution-linkage / about-ict / 3d-point-cloud / https: / / const.fukuicompu.co.jp / constmag / info / 54

[0029] The tree crown height image data creation processing unit 11C creates tree crown height image data as pre-processed aerial image data from the three-dimensional point cloud data created by the three-dimensional point cloud data creation processing unit 11B. Specifically, the tree crown height image data creation processing unit 11C creates meshed digital surface model data and digital elevation model data based on the three-dimensional point cloud data created by the three-dimensional point cloud data creation processing unit 11B, and takes the difference between the digital surface model data and the digital elevation model data for each mesh, thereby creating tree crown height image data in which the tree crown height is obtained for each mesh.

[0030] Alternatively, the tree crown height image data creation processing unit 11C may create tree crown height image data by substituting the 5m mesh (elevation) of the base map information (digital elevation model) measured by the airborne laser survey of the Geospatial Information Authority of Japan, performing a process to make the mesh size the same, and then taking the difference between the digital surface model data and the base map information (digital elevation model).

[0031] FIG. 3 is a diagram for explaining an example of the tree crown height image data created by the tree crown height image data creation processing unit 11C. Specifically, FIG. 3 shows a state where the tree crown height image is displayed on a display (not shown). The planting area PT shown in FIG. 3 corresponds to the planting area PT shown in FIG. 2. In the example shown in FIG. 3, the tree crown height image data creation processing unit 11C creates meshed digital surface model data and digital elevation model data for every 50 cm pixel, and by taking the difference between the digital surface model data and the digital elevation model data for each mesh, tree crown height image data in which the tree crown height is obtained for every 50 cm × 50 cm mesh is created.

[0032] In the example shown in FIG. 1, the noise removal processing unit 12 creates post-noise-removal aerial image data by removing the noise included in the pre-processed aerial image data (tree crown height image data) created by the aerial image data pre-processing unit 11. The noise removal processing unit 12 removes the noise included in the tree crown height image data by executing an averaging filter process that smoothes the image to remove the noise.

[0033] The planted tree candidate detection processing unit 13 detects planted tree candidate points based on the pre-processed aerial image data (tree crown height image data) created by the aerial image data pre-processing unit 11 and the post-noise-removal aerial image data created by the noise removal processing unit 12, and creates planted tree candidate point detection image data which is aerial image data including the planted tree candidate points. Specifically, the planted tree candidate detection processing unit 13 detects planted tree candidate points by executing an inter-image difference process of overlapping and differentiating the tree crown height image data as the pre-processed aerial image data and the post-noise-removal aerial image data. By displaying the differentiated image (planted tree candidate detection image) on the display, the changed locations are mechanically detected, the vertices of the planted trees become zero, and the planted tree candidate points can be detected. On the other hand, the planted tree candidates also include broad-leaved trees growing in the planted forest land and large-sized herbaceous weeds such as pampas grass.

[0034] FIG. 4 is a diagram for explaining an example of the processing by the planted tree candidate detection processing unit 13. Specifically, FIG. 4(A) shows a state in which the aerial image data (tree crown height image data) after preprocessing and the aerial image data after noise removal are superimposed. In FIG. 4(A), (1) shows the tree crown height model of the planted tree as the tree crown height image data, and (2) shows the tree crown height correction model obtained by performing the averaging filter processing of noise removal as the aerial image data after noise removal. FIG. 4(B) shows the difference file (3) created by calculating the difference between the tree crown height model and the tree crown height correction model. FIG. 4(C) shows points where the value of the difference file (3) becomes zero (the vertex candidate points of the planted tree), etc. As shown in FIG. 4, the planted tree candidate detection processing unit 13 performs a difference process between images by superimposing the tree crown height image data and the aerial image data after noise removal and taking the difference, so that the changed portions are mechanically detected, and the difference becomes zero at the vertices of the planted trees, and the planted tree candidate points can be detected.

[0035] FIG. 5 is a diagram for explaining an example of the planted tree candidate detection image data, which is the aerial image data including the planted tree candidate points created by the planted tree candidate detection processing unit 13. Specifically, FIG. 5 shows the aerial image data including the planted tree candidate points created by the planted tree candidate detection processing unit 13 based on the tree crown height image data shown in FIG. 3. That is, FIG. 5 is substantially the aerial color image shown in FIG. 2 with the planted tree candidate points plotted thereon. In the example shown in FIG. 5, the detection processing image of the planted tree candidates in the forest area to be surveyed (the one with the planted tree candidate points plotted on the aerial color image) is displayed on the display.

[0036] FIG. 6 is an enlarged view of the detection processing image of the planted tree candidates shown in FIG. 5 (the one with the planted tree candidate points plotted on the aerial color image). Specifically, FIG. 6 shows the enlarged detection processing image of the planted tree candidates displayed on the display. In the example shown in FIG. 6, as indicated by the three dashed circles, the planted tree candidate points include broad-leaved trees growing in the planted forest land and large-sized herbs such as pampas grass. That is, in the example shown in FIG. 6, the planting tree candidate points within the three dashed circles do not indicate planting trees, but rather broad-leaved trees or large-sized herbs. That is, it can be seen that there is a possibility that broad-leaved trees and large-sized herbs such as pampas grass growing in the forest land may be erroneously detected (erroneously estimated) as planting trees.

[0037] In the example shown in FIG. 1, the feature amount extraction processing unit 14 extracts feature amounts from a predetermined region including the planting tree candidate points (points with zero difference) in the planting tree candidate point detection image data created by the planting tree candidate point detection processing unit 13, and creates data for planting tree detection. Specifically, the feature amount extraction processing unit 14 extracts feature amounts from a circle with a radius of 10 centimeters centered on the planting tree candidate points (points with zero difference) included in the planting tree candidate point detection image data created by the planting tree candidate point detection processing unit 13, which is the above-mentioned predetermined region. The feature amounts extracted from the predetermined region include the sum, maximum value, minimum value, average value, range, median, minimum frequency value, standard deviation, variance, and statistical values of the manifold for each of the RGB values, normalized RGB values, and crown height values within the predetermined region. That is, the feature amounts extracted from the predetermined region include the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the R value within the predetermined region, the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the G value within the predetermined region, the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the B value within the predetermined region, the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the normalized R value within the predetermined region, the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the normalized G value within the predetermined region, the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the normalized B value within the predetermined region, and the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and statistical values of the manifold (10 values) of the pixels for the tree crown height value within the predetermined region.

[0038] FIG. 7 is a diagram for explaining a predetermined region where feature amount extraction is performed and an example of a feature amount. As shown in FIG. 7, in the process executed by the feature amount extraction processing unit 14, for the detection processing image of the planted tree candidate (where the planted tree candidate points are plotted on the aerial color image), a circle with a radius of 10 cm is created for each planted tree candidate point. For each of the seven files of the measured values of red, green, and blue (RGB), the normalized index values of RGB, and the tree crown height value (for example, the value calculated by the tree crown height image data creation processing unit 11C) included in the circle of the photographic image (aerial color image), the sum, maximum value, minimum value, average value, range, median value, minimum frequency value, standard deviation, variance, and 10 statistical values of the manifold are calculated.

[0039] In the example shown in FIG. 1, the planted tree detection processing unit 15 detects (estimates) the planted trees included in the aerial color image data based on the planted tree detection data created by the feature amount extraction processing unit 14. The planted tree detection processing unit 15 includes a data division processing unit 15A, an estimation model construction processing unit 15B, an accuracy evaluation processing unit 15C, and a planted tree estimation processing unit 15D. The data division processing unit 15A divides the planted tree detection data created by the feature amount extraction processing unit 14 into estimation data, which is data used for estimating the planted trees, and training data, which is data used for constructing the estimation model of the planted trees. Specifically, the data division processing unit 15A divides 80% of the planted tree detection data created by the feature amount extraction processing unit 14 into estimation data, and divides 20% of the planted tree detection data created by the feature amount extraction processing unit 14 into training data. Thereby, the training data can be used as teacher - supervised learning data for model construction in an AI program. Also, the estimation data can be used for accuracy verification (performance index) of the constructed estimation model.

[0040] The estimation model construction processing unit 15B constructs an estimation model by performing teacher - supervised learning using the training data. Specifically, the estimation model construction processing unit 15B constructs an estimation model by eliminating, one by one, the feature amounts that do not contribute to the performance of the estimation model among the feature amounts included in the training data. In the example shown in FIG. 1, a binary classification is performed on two items: the planted trees included in the detection process of the planted tree candidates and other than the planted trees. For example, using the algorithm of Recursive Feature Elimination from 70 feature amounts, a total of 65 Support Vector Machine models with the number of feature amounts from 69 to 5 are constructed as the estimation model. In other examples, the estimation model construction processing unit 15B may construct a model other than the Support Vector Machine model as the estimation model.

[0041] In the example shown in FIG. 1, the accuracy evaluation processing unit 15C executes the accuracy evaluation of the estimation model constructed by the estimation model construction processing unit 15B. Specifically, the accuracy evaluation processing unit 15C divides the training data into k parts, uses (k - 1) / k of the training data as learning data, and uses 1 / k of the training data as verification data. Also, the accuracy evaluation processing unit 15C uses the k-fold cross-validation method in which the accuracy evaluation of the estimation model is executed while changing the combination of the learning data and the verification data.

[0042] FIG. 8 is a diagram for explaining an example in which the accuracy evaluation processing unit 15C uses the 5-fold cross-validation method. In the example shown in FIG. 8, the accuracy evaluation processing unit 15C divides the training data into 5 parts, performs repeated learning while changing the combination of the learning data and the verification data, and verifies the accuracy of the estimation model (machine learning model). By using the 5-fold cross-validation method, the accuracy evaluation processing unit 15C evaluates the accuracy of 65 support vector machine models.

[0043] For the in-situ accuracy verification of the estimation model, a sample area is randomly sampled on the image, and it is visually inspected or investigated in the field to confirm whether the planted trees in the area are correctly determined. The number of in-situ investigations and visual inspections is defined as the number of correct answers, the number of planted trees detected by the learning model is defined as the number of model detections, and the number of planted trees whose positions match between the number of correct answers and the number of model detections is defined as the number of matches. That is, in the supervised learning described above, a pair of aerial image data and the in-situ investigation results of the planted trees included in the aerial image data (results indicating where the planted trees exist in the aerial image, etc.) is used as teacher data (training data).

[0044] FIG. 9 is a diagram showing an example of an equation for calculating the coincidence rate and the false detection rate for verifying an estimation model with high accuracy in the accuracy evaluation of the estimation model executed by the accuracy evaluation processing unit 15C. In the formula shown in FIG. 9, the number of correct answers is the number of planted trees visually interpreted and confirmed on-site within the sample area. The number of detected trees is the number of planted trees detected by the learning model (estimation model). The number of matching trees is the number of trees where the positions of the planted trees in the number of correct answers and the number of detected trees match.

[0045] In the example shown in FIG. 1, the planted tree estimation processing unit 15D estimates (detects) the planted trees included in the estimation data (aerial image data) by using the selected post-estimation model, which is the estimation model selected based on the result of the accuracy evaluation of the estimation model executed by the accuracy evaluation processing unit 15C. Specifically, the planted tree estimation processing unit 15D estimates (detects) the planted trees included in the estimation data (aerial image data) by using the selected post-estimation model, which is the estimation model that obtained the highest accuracy evaluation result in the accuracy evaluation of the estimation model executed by the accuracy evaluation processing unit 15C.

[0046] FIGS. 10 and 11 are diagrams showing an example of planted trees and the like estimated (detected) by the planted tree estimation processing unit 15D. Specifically, FIG. 10 is a diagram showing an example of aerial image data including the planted trees estimated (detected) by the planted tree estimation processing unit 15D. That is, FIG. 10 is substantially a plot of points indicating the planted trees estimated (detected) by the planted tree estimation processing unit 15D on the aerial color image shown in FIG. 2. FIG. 11(A) shows an enlarged part of the aerial image data including the planted tree candidate points shown in FIG. 5, and FIG. 11(B) shows an enlarged part of the aerial image data including the points indicating the planted trees estimated (detected) by the planted tree estimation processing unit 15D at the position corresponding to FIG. 11(A).

[0047] In the example shown in FIG. 10, the detection processing image of the planted trees in the forest area to be surveyed (a plot of points indicating the planted trees on the aerial color image) is displayed on the display. Therefore, the detection results of the planted trees throughout the survey area can be confirmed. In the example shown in FIG. 11(A), as indicated by the three dashed circles, the tree planting candidate points include broad-leaved trees growing in the tree planting area and large-sized herbs such as miscanthus. That is, in the example shown in FIG. 11(A), the tree planting candidate points within the three dashed circles do not indicate tree plantings, but rather broad-leaved trees or large-sized herbs. On the other hand, in the example shown in FIG. 11(B), the tree planting estimation processing unit 15D estimates that there are no tree plantings within the three dashed circles. That is, the tree planting estimation processing unit 15D estimates that the tree planting candidate points within the three dashed circles shown in FIG. 11(A) do not indicate tree plantings, but rather broad-leaved trees or large-sized herbs.

[0048] That is, in the example shown in FIG. 11(B), it is possible to distinguish between tree plantings and non-tree plantings in the forest area to be surveyed, and only tree plantings can be automatically detected and mapped. The tree planting estimation processing unit 15D (the learning model of the AI program) can estimate that the broad-leaved trees and large-sized herbs included in the tree planting candidate points are not tree plantings, and can automatically detect only tree plantings.

[0049] In the example shown in FIG. 1, the tree planting information creation processing unit 16 creates information (such as information on ID (identifier), standing tree position, seedling height, etc.) regarding the tree plantings detected by the tree planting detection processing unit 15. Therefore, in the example shown in FIG. 1, the tree planting automatic detection device 1 can create a database of tree plantings. By displaying the information regarding tree plantings as, for example, a list on a display, an overview of the resources of the tree plantings in the forest area to be surveyed can be grasped.

[0050] FIG. 12 is a diagram showing an example of information regarding tree plantings created by the tree planting information creation processing unit 16. In the example shown in FIG. 12, the information regarding tree plantings created by the tree planting information creation processing unit 16 includes the ID of the tree plantings detected (estimated) by the tree planting detection processing unit 15, information (X coordinate and Y coordinate) indicating the standing tree position of the tree plantings detected (estimated) by the tree planting detection processing unit 15, and information indicating the seedling height of the tree plantings detected by the tree planting detection processing unit 15. For example, for the planted tree with ID "556", the X coordinate of the planted tree is "-29598.1", the Y coordinate of the planted tree is "83922.83", and the seedling height of the planted tree is "1.2702". For example, for the planted tree with ID "716", the X coordinate of the planted tree is "-29593.2", the Y coordinate of the planted tree is "83920.23", and the seedling height of the planted tree is "0.47168". For example, for the planted tree with ID "827", the X coordinate of the planted tree is "-29607.7", the Y coordinate of the planted tree is "83918.13", and the seedling height of the planted tree is "1.28333". The planted tree information creation processing unit 16 can use known spreadsheet software to create information about planted trees as shown in, for example, FIG. 12, and output the information about planted trees as, for example, GIS data.

[0051] In the example shown in FIG. 1, the planted tree information aggregation processing unit 17 aggregates information about planted trees created by the planted tree information creation processing unit 16, which is information about all planted trees included in the forest area to be surveyed. The information about all planted trees included in the forest area to be surveyed aggregated by the planted tree information aggregation processing unit 17 includes the total number of all planted trees included in the forest area to be surveyed and the number of planted trees per unit area (for example, 1 hectare) of the forest area to be surveyed. The planted tree information aggregation processing unit 17 has a function of registering the aggregated information in a database.

[0052] FIG. 13 is a diagram showing an example of information about all planted trees included in the forest area to be surveyed aggregated by the planted tree information aggregation processing unit 17. In the example shown in FIG. 13, the information about all planted trees included in the forest area to be surveyed aggregated by the planted tree information aggregation processing unit 17 includes the area of the forest area to be surveyed, the total number of all planted trees included in the forest area to be surveyed, and the number of planted trees per unit area (1 hectare) of the forest area to be surveyed. Specifically, in the example shown in FIG. 13, information on all planted trees included in the target forest area aggregated by the planted tree information aggregation processing unit 17 is displayed on the display. The planted tree information aggregation processing unit 17 can aggregate information on all planted trees in the target forest area, calculate resource information such as planted area, number of trees, number of trees per hectare, and average seedling height, and register it in the database.

[0053] As shown in the example of FIG. 13, by displaying information on all planted trees in the target forest area, such as the standing tree positions of each planted tree, on the display, the number of planted trees indicating how many were planted in the target forest area and the planting positions can be accurately grasped. Based on this, the form (area, number of planted trees, and planting interval) of the planting plan can be determined.

[0054] FIG. 14 is a flowchart for explaining an example of the process executed in the planted tree automatic detection device 1 of the first embodiment. FIG. 15 is a flowchart for explaining in detail an example of the process executed in step S50 of FIG. 14. In the examples shown in FIGS. 14 and 15, in step S10, the aerial image data preprocessing unit 11 executes preprocessing of the aerial color image data of the forest area. Specifically, in step S11, the aerial image data input processing unit 11A receives the input of aerial color image data taken from an aircraft or drone so as to include the target forest area. Next, in step S12, the three-dimensional point cloud data creation processing unit 11B creates three-dimensional point cloud data from the aerial color image data received in step S11. Next, in step S13, the crown height image data creation processing unit 11C creates crown height image data as preprocessed aerial image data from the three-dimensional point cloud data created in step S12.

[0055] Next, in step S20, the noise removal processing unit 12 creates post-noise-removal aerial image data by removing noise included in the pre-processed aerial image data (the tree crown height image data created in step S13) created in step S10. Next, in step S30, the planted tree candidate detection processing unit 13 detects planted tree candidate points based on the pre-processed aerial image data (the tree crown height image data created in step S13) created in step S10 and the post-noise-removal aerial image data created in step S20, and creates planted tree candidate point detection image data which is aerial image data including the planted tree candidate points. Next, in step S40, the feature quantity extraction processing unit 14 extracts feature quantities from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created in step S30, and creates data for planted tree detection.

[0056] Next, in step S50, the planted tree detection processing unit 15 detects (estimates) the planted trees included in the aerial color image data based on the data for planted tree detection created in step S40. Specifically, in step S51, the data division processing unit 15A divides the data for planted tree detection created in step S40 into estimation data which is data used for estimating planted trees and training data which is data used for constructing an estimation model of planted trees. Next, in step S52, the estimation model construction processing unit 15B constructs an estimation model by performing supervised learning using the training data. Next, in step S53, the accuracy evaluation processing unit 15C performs an accuracy evaluation of the estimation model constructed in step S52. Next, in step S54, the planted tree estimation processing unit 15D estimates (detects) the planted trees included in the estimation data (aerial image data) by using the selected estimation model which is the estimation model selected based on the result of the accuracy evaluation of the estimation model performed in step S53.

[0057] Next, in step S60, the planted tree information creation processing unit 16 creates information regarding the planted trees detected in step S50 (for example, information such as ID, standing tree position, and seedling height). Next, in step S70, the planted tree information aggregation processing unit 17 aggregates information regarding the planted trees created in step S60, which is information regarding all the planted trees included in the forest area to be surveyed.

[0058] <Second Embodiment> Hereinafter, a second embodiment of the planted tree automatic detection device, the planted tree automatic detection method, and the program of the present invention will be described. The planted tree automatic detection device 1 of the second embodiment is configured in the same manner as the planted tree automatic detection device 1 of the first embodiment described above, except for the points described later. Therefore, according to the planted tree automatic detection device 1 of the second embodiment, the same effects as those of the planted tree automatic detection device 1 of the first embodiment described above can be achieved, except for the points described later.

[0059] As described above, in the planted tree automatic detection device 1 of the first embodiment, the aerial image data input processing unit 11A receives the input of aerial color image data taken from a drone or an aircraft so as to include the forest area to be surveyed. On the other hand, in the planted tree automatic detection device 1 of the second embodiment, the aerial image data input processing unit 11A may receive the input of aerial color image data (for example, aerial color image data used to create an orthoimage) taken from above by a drone or an aircraft so as to include the forest area to be surveyed during laser measurement.

[0060] As described above, the embodiments for implementing the present invention have been described using the embodiments. However, the present invention is not limited to such embodiments, and various modifications and substitutions can be made without departing from the gist of the present invention. The configurations described in the above-described embodiments and each example may be combined.

[0061] Note that, the whole or part of the functions of each part included in the planted tree automatic detection device 1 in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, reading the program recorded on this recording medium into a computer system, and executing it. Here, the "computer system" is assumed to include hardware such as an OS and peripheral devices. Also, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, etc., and a storage unit such as a hard disk built in a computer system. Further, the "computer-readable recording medium" also includes those that dynamically hold a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and those that hold a program for a certain time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Also, the above program may be for realizing a part of the aforementioned functions, and may further be realizable in combination with a program already recorded in the computer system for realizing the aforementioned functions.

Explanation of Signs

[0062] 1... Planted tree automatic detection device, 11... Aerial image data preprocessing unit, 11A... Aerial image data input processing unit, 11B... Three-dimensional point cloud data creation processing unit, 11C... Crown height image data creation processing unit, 12... Noise removal processing unit, 13... Planted tree candidate point detection processing unit, 14... Feature amount extraction processing unit, 15... Planted tree detection processing unit, 15A... Data division processing unit, 15B... Estimation model construction processing unit, 15C... Accuracy evaluation processing unit, 15D... Planted tree estimation processing unit, 16... Planted tree information creation processing unit, 17... Planted tree information aggregation processing unit

Claims

1. A planted tree automatic detection device for automatically detecting planted trees in a specified forest area, an aerial image data preprocessing unit that creates preprocessed aerial image data by performing preprocessing on the aerial color image data of the forest area; a noise removal processing unit that creates noise-removed aerial image data by removing noise included in the preprocessed aerial image data created by the aerial image data preprocessing unit; a planted tree candidate point detection processing unit that detects planted tree candidate points based on the preprocessed aerial image data and the noise-removed aerial image data, and creates planted tree candidate point detection image data that is aerial image data including the planted tree candidate points; a feature amount extraction processing unit that extracts feature amounts from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created by the planted tree candidate point detection processing unit, and creates data for planted tree detection; a planted tree detection processing unit that detects the planted trees included in the aerial color image data based on the data for planted tree detection created by the feature amount extraction processing unit; a planted tree information creation processing unit that creates information regarding the planted trees detected by the planted tree detection processing unit; comprising a planted tree information aggregation processing unit that aggregates information regarding all the planted trees included in the forest area, which is the information regarding the planted trees created by the planted tree information creation processing unit, wherein the planted tree detection processing unit comprises a data division processing unit that divides the data for planted tree detection created by the feature amount extraction processing unit into estimation data that is data used for estimating the planted trees and training data that is data used for constructing the estimation model of the planted trees; an estimation model construction processing unit that constructs the estimation model by performing supervised learning using the training data; an accuracy evaluation processing unit that performs accuracy evaluation of the estimation model constructed by the estimation model construction processing unit; and a planted tree estimation processing unit that estimates the planted trees included in the estimation data by using a selected estimation model, which is an estimation model selected based on the result of the accuracy evaluation of the estimation model performed by the accuracy evaluation processing unit, wherein the aerial image data preprocessing unit comprises an aerial image data input processing unit that receives input of the aerial color image data A three-dimensional point cloud data creation processing unit that creates three-dimensional point cloud data from the aerial color image data received by the aerial image data input processing unit; A tree crown height image data creation processing unit that creates tree crown height image data as the pre-processed aerial image data from the three-dimensional point cloud data created by the three-dimensional point cloud data creation processing unit; The planted tree candidate detection processing unit detects the planted tree candidate points by performing a difference process between images that overlaps and differentiates the tree crown height image data as the pre-processed aerial image data and the aerial image data after noise removal. An automatic planted tree detection device.

2. The feature amount extraction processing unit: Creates a circle with a radius of 10 centimeters centered on the planted tree candidate points included in the planted tree candidate point detection image data created by the planted tree candidate detection processing unit as the predetermined area; The feature amounts extracted from the predetermined area include: The sum, maximum value, minimum value, average value, range, median, minimum frequency value, standard deviation, variance, and statistical values of the manifold for each of the RGB values, normalized RGB values, and tree crown height values within the predetermined area. The automatic planted tree detection device according to claim 1.

3. The data division processing unit: Divides 80% of the planted tree detection data created by the feature amount extraction processing unit into the estimation data; Divides 20% of the planted tree detection data created by the feature amount extraction processing unit into the training data. The automatic planted tree detection device according to claim 1 or 2.

4. The estimation model construction processing unit: Constructs the estimation model by eliminating the feature amounts that do not contribute to the performance of the estimation model among the feature amounts included in the training data. The automatic planted tree detection device according to any one of claims 1 to 3.

5. The accuracy evaluation processing unit: Uses the k-fold cross-validation method of dividing the training data into k parts, using (k - 1) / k of the training data as the learning data, using 1 / k of the training data as the verification data, and performing accuracy evaluation of the estimation model while changing the combination of the learning data and the verification data. The automatic planted tree detection device according to any one of claims 1 to 4.

6. The planted tree estimation processing unit: By using the selected post-estimation model, which is the estimation model that obtained the highest accuracy evaluation result in the accuracy evaluation of the estimation model executed by the accuracy evaluation processing unit, the planted trees included in the estimation data are estimated. The planted tree automatic detection device according to any one of claims 1 to 5.

7. The information on the planted trees created by the planted tree information creation processing unit includes the ID (identifier) of the planted trees detected by the planted tree detection processing unit, information indicating the standing tree positions of the planted trees detected by the planted tree detection processing unit, and information indicating the seedling heights of the planted trees detected by the planted tree detection processing unit, and at least includes them. The planted tree automatic detection device according to any one of claims 1 to 6.

8. The information on all the planted trees included in the forest area aggregated by the planted tree information aggregation processing unit includes the total number of all the planted trees included in the forest area, and the number of planted trees per unit area of the forest area, and The planted tree information aggregation processing unit has a function of registering the aggregated information in a database. The planted tree automatic detection device according to any one of claims 1 to 7.

9. A planted tree automatic detection method for automatically detecting planted trees in a predetermined forest area, comprising an aerial image data preprocessing step of creating preprocessed aerial image data by executing preprocessing of the aerial color image data of the forest area; a noise removal processing step of creating noise-removed aerial image data by removing noise included in the preprocessed aerial image data created in the aerial image data preprocessing step; a planted tree candidate point detection processing step of detecting planted tree candidate points based on the preprocessed aerial image data and the noise-removed aerial image data, and creating planted tree candidate point detection image data, which is the aerial image data including the planted tree candidate points; a feature amount extraction processing step of extracting feature amounts from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created in the planted tree candidate point detection processing step, and creating data for planted tree detection. Based on the tree-planting detection data created in the feature quantity extraction processing step, a tree-planting detection processing step for detecting the tree-plantings included in the aerial color image data, and a tree-planting information creation processing step for creating information regarding the tree-plantings detected in the tree-planting detection processing step, A tree-planting information aggregation processing step for aggregating information regarding the tree-plantings created in the tree-planting information creation processing step, which is information regarding all the tree-plantings included in the forest area, The tree-planting detection processing step includes A data division processing step of dividing the tree-planting detection data created in the feature quantity extraction processing step into estimation data which is data used for estimating the tree-plantings and training data which is data used for constructing the estimation model of the tree-plantings, and an estimation model construction processing step of constructing the estimation model by performing supervised learning using the training data, An accuracy evaluation processing step of performing an accuracy evaluation of the estimation model constructed in the estimation model construction processing step, A tree-planting estimation processing step of estimating the tree-plantings included in the estimation data by using the selected post-estimation model which is the estimation model selected based on the result of the accuracy evaluation of the estimation model performed in the accuracy evaluation processing step, The aerial image data preprocessing step includes An aerial image data input processing step of receiving the input of the aerial color image data, A three-dimensional point cloud data creation processing step of creating three-dimensional point cloud data from the aerial color image data received in the aerial image data input processing step, A crown height image data creation processing step of creating crown height image data as the preprocessed aerial image data from the three-dimensional point cloud data created in the three-dimensional point cloud data creation processing step, The tree-planting candidate point detection processing step is a step of detecting the tree-planting candidate points by performing a difference processing between images of superimposing and differentiating the crown height image data as the preprocessed aerial image data and the aerial image data after noise removal. Automatic tree-planting detection method.

10. On a computer, An aerial image data preprocessing step of creating preprocessed aerial image data by performing preprocessing of aerial color image data of a predetermined forest area, A noise removal processing step of creating post-noise-removal aerial image data by removing noise included in the pre-processed aerial image data created in the aerial image data pre-processing step; A planted tree candidate point detection processing step of detecting planted tree candidate points based on the pre-processed aerial image data and the post-noise-removal aerial image data, and creating planted tree candidate point detection image data which is aerial image data including the planted tree candidate points; A feature amount extraction processing step of extracting feature amounts from a predetermined area including the planted tree candidate points in the planted tree candidate point detection image data created in the planted tree candidate point detection processing step, and creating data for planted tree detection; A planted tree detection processing step of detecting planted trees included in the aerial color image data based on the data for planted tree detection created in the feature amount extraction processing step; A planted tree information creation processing step of creating information regarding the planted trees detected in the planted tree detection processing step; A program for causing execution of a planted tree information aggregation processing step of aggregating information regarding all the planted trees included in the forest area, the information regarding the planted trees being the information created in the planted tree information creation processing step; In the planted tree detection processing step, A data division processing step of dividing the data for planted tree detection created in the feature amount extraction processing step into estimation data which is data used for estimation of the planted trees and training data which is data used for construction of an estimation model of the planted trees; an estimation model construction processing step of constructing the estimation model by performing supervised learning using the training data; An accuracy evaluation processing step of performing accuracy evaluation of the estimation model constructed in the estimation model construction processing step; A planted tree estimation processing step of estimating the planted trees included in the estimation data by using a selected estimation model which is an estimation model selected based on the result of the accuracy evaluation of the estimation model performed in the accuracy evaluation processing step; The aerial image data pre-processing step includes An aerial image data input processing step of receiving input of the aerial color image data; A three-dimensional point cloud data creation processing step of creating three-dimensional point cloud data from the aerial color image data received in the aerial image data input processing step; A tree crown height image data creation processing step of creating tree crown height image data as the pre-processed aerial image data from the three-dimensional point group data created by the three-dimensional point group data creation processing step, The planted tree candidate detection processing step is a step of detecting the planted tree candidate points by performing a difference process between images that overlaps and differentiates the tree crown height image data as the pre-processed aerial image data and the aerial image data after noise removal. Program.

Images