Image processing device and image processing program

Abstract

This invention provides an image processing device capable of appropriately generating 3D data of forests. [Solution] The image processing device 10 includes a CPU 10a as at least one processing unit, and processes image data in order to generate 3D data of a forest from image data of a forest that has been photographed. The CPU 10a is configured to identify trees T1 to T9 that are unsuitable for generating 3D data from among the multiple columnar shapes of trees T1 to T9 that appear in the image data, and to generate 3D data from the portion of the image data from which the excluded trees have been removed.

Description

Technical Field

[0001] The disclosure according to this specification relates to a technique for generating three-dimensional data of a forest.

Background Art

[0002] Conventionally, a technique for generating three-dimensional data from image data has been known. When generating three-dimensional data, it is necessary to remove the background from an image in which both the object for generating the three-dimensional data and the background are reflected. Patent Document 1 discloses a technique for recognizing the edges of a person in an image in which both the person and the background are reflected, separating the person and the background, and removing the background.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Now, the inventors conceived of generating three-dimensional data of a forest from image data of the forest. However, when attempting to separate the object for generating the three-dimensional data (for example, trees, ground) and the background in order to generate three-dimensional data of the forest, it was discovered that there are problems specific to forests. For example, in a forest, since trees grow thickly, there are trees with the same shape at both long distances and short distances, and it is difficult to separate such long-distance areas and remove them as the background. Also, for example, thin branches such as young branches become noise because they change due to sunlight and wind, and thus need to be removed.

[0005] One of the purposes according to the disclosure of this specification is to provide an image processing apparatus and an image processing program capable of appropriately generating three-dimensional data of a forest.

Means for Solving the Problems

[0006] One aspect disclosed herein is an image processing apparatus comprising at least one processing unit (10a) for processing image data to generate three-dimensional data of a forest from image data of a forest, At least one processing unit, Among the multiple column shapes (T1-T9) captured in the image data, those column shapes unsuitable for generating 3D data are identified and excluded. It is configured to generate 3D data from the portion of the image data that has been excluded.

[0007] Another aspect of the disclosed embodiment is an image processing program that processes image data to generate three-dimensional data of a forest from image data of a forest, At least one processing unit (10a) Among the multiple column shapes (T1-T9) captured in the image data, those column shapes unsuitable for generating 3D data are identified and excluded. The process involves generating 3D data from the portion of the image data that has been excluded.

[0008] According to these methods, multiple columnar shapes appearing in the image data are identified in order to process the trees growing in the forest. Columnar shapes unsuitable for generating 3D data are then excluded. Since the 3D data is generated from the parts excluding the excluded elements, it becomes possible to appropriately generate 3D data of the forest.

[0009] The symbols in parentheses included in the claims, etc., are illustrative examples illustrating the correspondence with the embodiments described later, and are not intended to limit the technical scope. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing the schematic configuration of an image processing device. [Figure 2]A schematic diagram illustrating the process of photographing trees with a camera and their subsequent removal. [Figure 3] A diagram showing the functional configuration of an image processing device. [Figure 4] A graph to explain the sensitivity coefficient. [Figure 5] A diagram to explain the definition of angle α. [Figure 6] A flowchart illustrating an example of processing by an image processing device. [Figure 7] A flowchart showing an example of how to process tree trunks. [Figure 8] A flowchart illustrating an example of ground treatment. [Figure 9] A diagram illustrating the bark feature matrix. [Figure 10] A flowchart illustrating an example of image recognition processing. [Figure 11] A flowchart illustrating an example of depth estimation processing. [Modes for carrying out the invention]

[0011] In this disclosure or claims, the term "processor" means one or more hardware processors configured to execute processing defined by computer program code (i.e., one or more instructions of a computer program) contained in a computer program by reading the code each time. In other words, a "processor" is a hardware device that executes one or more programmed processes. Therefore, computer program code can also be considered software that can define the processing of the processor according to its content. For example, a "processor" may be a general-purpose or specific-purpose processor and may be, but is not limited to, a CPU, microprocessor, GPU, and DFP (Data Flow Processor).

[0012] In the present disclosure or claims, the term "memory" refers to a non-transitory physical recording medium configured to record computer program code and / or data in an accessible manner from a processor. "Memory" can be implemented by memory technologies such as SRAM, SDRAM, non-volatile flash type memory, or other types of memory. The computer program code constituting the program is recorded on the memory and executed by the processor, enabling the processor to realize the various functions described above.

[0013] In the present disclosure or claims, the term "circuit" refers to a single or multiple hardware logic circuits configured to perform specific processing defined based on a pre-designed circuit configuration. In other words (and in contrast to a "processor"), the "circuit" in the present disclosure or claims does not refer to something whose processing is defined by software such as the above computer program code, but rather refers to a hardware device that executes specific processing based on a circuit configuration. For example, "circuit" may include custom ICs such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array) designed by a hardware description language (HDL: Hardware Description Language). That is, the "circuit" in the present disclosure or claims includes all hardware circuits except the above processor that executes processing by reading computer program code.

[0014] In the present disclosure or claims, the expression "at least one of a circuit and a processor" should be interpreted as a disjunction (logical OR), and should not be interpreted as at least one circuit and at least one processor.

[0015] In the present disclosure or claims, the term "processing unit" means a hardware device that executes processing by a "processor", "circuit", or a combination thereof. When its function is interpreted as being impossible to be realized by a "circuit" and possible to be realized by a "processor", it may mean the "processor" itself.

[0016] Hereinafter, a plurality of embodiments will be described based on the drawings. In each embodiment, the same reference numerals may be assigned to corresponding components, and redundant descriptions may be omitted. When only a part of the configuration is described in each embodiment, the configuration of other embodiments described previously can be applied to other parts of the configuration. Also, not only the combinations of configurations explicitly shown in the description of each embodiment, but also the configurations of a plurality of embodiments can be partially combined with each other as long as there is no problem with the combination.

[0017] (First Embodiment) The image processing apparatus 10 according to the first embodiment shown in FIG. 1 is used, for example, in a forest management service. The forest management service is a service for visualizing forest information to assist in the efficiency improvement of forestry and the proper management of old-growth forests and the like. By properly managing the forest, the forest can exhibit a high absorption capacity for greenhouse gases such as carbon dioxide. Also, the forest management service guarantees the quality of carbon credits through regular forest measurements.

[0018] Specifically, the forest management service generates 3D data of the forest from the image data of the photographed forest and provides the 3D data of the forest as visualized forest information. The image processing apparatus 10 processes the image data in order to generate 3D data of the forest.

[0019] Image data is captured using camera 30 as shown in Figure 2. Alternatively, an aircraft such as a drone equipped with camera 30 may search the forest and capture images at multiple locations within the forest. Instead of a drone, image data may be captured by a robot capable of moving through the forest, or by a human photographer walking around the forest. The photography route within the forest can be set based on a route plan.

[0020] The 3D data is a 3D representation of the forest's shape, and is represented, for example, by a point cloud. Here, the forest's shape may include the shape of the ground. The forest's shape may include the position and shape of trees growing from the ground. In this case, the tree's shape only needs to include the shape of the trunk, which corresponds to the main body of the tree, and does not need to include the shapes of the branches and leaves. The trunk's shape includes its thickness. The trunk's shape may or may not include its height.

[0021] As shown in Figure 1, the image processing device 10 is configured primarily around, for example, at least one computer. The computer comprising the image processing device 10 has at least one CPU (Central Processing Unit) 10a and at least one memory 10b. The memory 10b may be a non-transitory tangible storage medium that non-temporarily stores computer programs, data, artificial intelligence models, etc., that can be read by the CPU 10a. Furthermore, the computer may have a rewritable volatile storage medium such as RAM (Random Access Memory) 10c. The CPU 10a can execute various processes according to the computer programs stored in the memory 10b.

[0022] Furthermore, the computer may have an interface 10d for exchanging data with the outside world. Interface 10d may include a communication interface for connecting the computer to the internet or external devices. Interface 10d may include an operation interface such as a keyboard or mouse for accepting human input.

[0023] The computer may also have a display 10e. The display 10e may be a liquid crystal display or an OLED display that displays the results of processing by the computer as an image. The display 10e may also be a projection-type display such as a projector.

[0024] Furthermore, the image processing device 10 or its computer may have a database 10f for storing image data of forests and processing data obtained by processing this data. The database 10f is configured to include at least one type of non-transitional physical storage medium, such as semiconductor memory, magnetic media, and optical media. Multiple databases 10f may be provided depending on the type of data.

[0025] As shown in Figure 3, the image processing device 10 has a remote data exclusion unit F1, a tree trunk separation unit F2, a tree trunk processing unit F3, and a ground processing unit F4 as functional units whose functions are realized when the CPU 10a executes a computer program (image processing program) stored in the memory 10b.

[0026] The distant data exclusion unit F1 excludes distant regions that capture distant objects from the area recorded in the image data, and excludes these regions from the generation of the tree point cloud. Here, the region may be composed of multiple pixels.

[0027] Specifically, the remote data exclusion unit F1 acquires image data to be processed from the database 10f. The remote data exclusion unit F1 then recognizes the depth of each tree captured in the image. Here, depth refers to the distance from the point of capture by the camera 30. Depth may also be called depth distance DD. Depth estimation may be performed by the remote data exclusion unit F1 recognizing pixels in the image data in which tree bark is captured, and recognizing the depth of those pixels in which the tree bark is captured. Depth recognition at the pixel can employ various methods, such as a depth estimation method using parallax, or a method called monocular depth estimation. Depth recognition may also be implemented using an artificial intelligence model.

[0028] Next, the far-field data exclusion unit F1 calculates the depth limit distance DL. The depth limit distance DL is the critical distance for excluding the far-field region from the image data. The depth limit distance DL is set based on the performance of the camera 30, etc., so as to achieve the target resolution TR, which can be expressed in units of pixels per meter [PX / m]. Information regarding the performance of the camera 30 can be obtained from camera model information stored in memory 10b or camera model information attached to the image data. For example, the depth limit distance DL can be expressed as shown in Equation 1 below. (Formula 1) CR[PX] / (2×TR[PX / m]×tan(AOV[°])) =DL[m]

[0029] Here, CR is the resolution of camera 30, and AOV is the field of view of camera 30. For example, for the trunk of an upright tree, CR should be the horizontal resolution and AOV should be the horizontal field of view.

[0030] Then, the far-field data exclusion unit F1 determines that the recognized depth of the image data is greater than the depth limit distance DL to be a far-field region and excludes it.

[0031] The tree trunk separation unit F2 recognizes the pixels of the tree trunks that are the target of 3D data generation from the near region, which is the data after the far region has been excluded, i.e., the data in which the target resolution TR can be achieved. The tree trunk separation unit F2 separates the pixels of the columnar tree trunks that can be the target of 3D data generation from the other pixels. This separation may involve actually separating the image data into two parts: the tree portion and the non-tree portion, or it may involve marking each pixel so that it can be separated and processed later.

[0032] The tree trunk processing unit F3 processes the pixels of the tree trunks separated by the tree trunk separation unit F2. Specifically, for each tree trunk forming a column shape, the tree trunk processing unit F3 extracts both ends of the same tree trunk and estimates the distance between the two ends. The distance between the two ends corresponds to the diameter of the trunk. Trunks that are too small to extract both ends are difficult to estimate in terms of diameter, making it difficult to obtain highly accurate 3D data. For this reason, small trunks may be excluded from processing at this stage. At this time, the tree trunk processing unit F3 may calculate the shooting distortion of the camera 30 based on the camera model information, correct the shooting distortion, and then process the pixels of the tree trunks.

[0033] Next, the tree trunk processing unit F3 sets the sensitivity coefficient ζ. The sensitivity coefficient ζ is set in the range of 0 < ζ < 1 and is a coefficient given by d (trunk diameter) / d (diameter error). The sensitivity coefficient ζ is a coefficient that represents how much the measured diameter is affected by the error.

[0034] The sensitivity coefficient ζ may be set based on experimental data obtained in advance, for example. For example, as shown in Figure 4, for multiple trees, the error is calculated from the difference between the tree diameter estimated from image data and the measured diameter of the tree (also called the measured diameter), and a straight line is approximated from the relationship between the measured diameter and the error using the least squares method or the like. The sensitivity coefficient ζ is then obtained as the coefficient of the approximated straight line.

[0035] Next, the tree trunk processing unit F3 determines the angle α of the tree trunk to be used for generating 3D data, based on equation 2. (Formula 2)DL[m]×ζ=DD[m] / cos(α[°])

[0036] As shown in Figure 5, angle α is the angle between a line segment extending from camera 30 to an arbitrary point on the surface of the tree and the perpendicular line to that arbitrary point on the surface. Points with a large angle α indicate low accuracy. In other words, unless there is a certain amount of region with a small angle α, the accuracy of the 3D data modeling cannot be ensured. That is, calculating angle α is essentially an estimation of the expected modeling accuracy for each tree trunk, and angle α can be said to be a parameter for estimating accuracy.

[0037] The tree trunk processing unit F3 excludes a tree trunk from generating 3D data if its trunk angle α is smaller than a preset threshold based on the required accuracy. In other words, only trunks for which high accuracy can be guaranteed are selected.

[0038] The tree trunk processing unit F3 constructs a point cloud of higher quality than the ground area, which will be described later, based on the selected tree trunk region from the image data. Here, "quality" can refer to the resolution of the point cloud.

[0039] The ground processing unit F4 processes the remaining pixels from the image data, excluding the pixels that were excluded as distant regions and the pixels of tree trunks separated by the tree trunk separation unit F2. In a forest, the remaining pixels are almost entirely composed of the ground, so the ground processing unit F4 processes the remaining pixels as the ground region.

[0040] The ground processing unit F4 downgrades the pixels of the ground. Downgrading here may involve, for example, merging multiple pixels to increase the size of the pixels. Based on the ground region of the image data, the ground processing unit F4 constructs a point cloud of lower quality than the tree trunk region. In this way, by making the ground region, which has a less complex shape than the trees, into a lower-quality point cloud, the overall amount of data in the 3D data can be reduced, or the amount of data in the ground region can be used to improve the quality of the tree point cloud.

[0041] Here, using the example in Figure 2, we will explain how trees T1 to T9, captured by camera 30, are handled. Tree T1 is located farther away than the depth limit distance DL relative to camera 30. Therefore, tree T1 is excluded from the generation of 3D data based on the processing of the distant data exclusion unit F1.

[0042] Although trees T2 and T3 are located closer than the depth limit distance DL, their angles α are smaller than those of trees T4-T9 (40° and 70° respectively). Therefore, based on the processing of the tree trunk processing unit F3, trees T2 and T3 are excluded from the generation of 3D data.

[0043] Trees T4 and T5 have trunks that are too small in diameter to be modeled with high accuracy. Therefore, based on the processing of the tree trunk processing unit F3, trees T4 and T5 are excluded from the generation of 3D data.

[0044] Because trees T7, T8, and T9 have smaller diameters than tree T6, errors are more likely to affect the tree structure reproduced in 3D data. Therefore, whether or not to include them in the 3D data generation is determined based on the setting of the sensitivity coefficient ζ.

[0045] Next, an example of how the image processing device 10 processes image data will be explained using the flowcharts in Figures 6-8. These flowcharts may be implemented by the CPU 10a executing a computer program stored in memory 10b.

[0046] In the flowchart of Figure 6, steps S10 to S60 show the overall process. In the first step, S10, the far-field data exclusion unit F1 recognizes bark pixels from the image data. In S20, after processing in S10, the far-field data exclusion unit F1 calculates depth data for the recognized bark pixels. In S30, after processing in S20, the far-field data exclusion unit F1 calculates the depth limit distance DL and excludes the far-field region based on the depth limit distance DL.

[0047] In S40, following the processing in S30, the tree trunk separation unit F2 recognizes the pixels of the tree trunk and separates the region of the tree trunk.

[0048] In S50, following the processing in S40, the tree trunk processing unit F3 processes the tree trunk region that was separated in S40.

[0049] In S60, following the processing in S50, the ground processing unit F4 processes the remaining area (ground area) after it was separated in S40. The series of processes ends with S60.

[0050] Next, the specific processing related to tree trunks in S50 will be explained using steps S51 to S55 in the flowchart of Figure 7. In the first step, S51, the tree trunk processing unit F3 estimates the diameter of each tree trunk. In S52, after processing in S51, the tree trunk processing unit F3 sets the sensitivity coefficient ζ. In S53, after processing in S52, the tree trunk processing unit F3 calculates the angle α of each tree trunk. In S54, after processing in S53, the tree trunk processing unit F3 selects the tree trunks to be used for point cloud generation (target for 3D data generation) based on the depth distance DD and angle α. In S55, after processing in S54, the tree trunk processing unit F3 constructs a high-quality point cloud. The series of processes ends with S55.

[0051] Next, the specific processing related to the ground in S60 will be explained using steps S61-62 in the flowchart of Figure 8. In the first step, S61, the ground processing unit F4 downgrades the pixels of the ground. In S62, following the processing in S61, the ground processing unit F4 constructs a low-quality point cloud. The series of processes ends with S62.

[0052] According to the first embodiment described above, in order to process the trees growing in the forest, trees T1 to T9 are identified as multiple columnar shapes that appear in the image data. Columnar shapes unsuitable for generating 3D data are then excluded. Since the 3D data is generated from the parts excluding the excluded elements, it becomes possible to appropriately generate 3D data of the forest.

[0053] Furthermore, according to the first embodiment, the depth distance DD of multiple column shapes is estimated. In addition, the depth limit distance DL is set based on the performance of the camera 30 that captured the image data. Then, by comparing the depth distance DD of each column shape with the depth limit distance DL, column shapes estimated to be farther away than the depth limit distance DL are excluded. In this method, in a forest where trees with similar column shapes grow endlessly, it is possible to estimate the depth distance DD of each column shape and separate and exclude column shapes located at far distances that are judged to have low accuracy based on the performance of the camera 30. Therefore, trees that can be modeled with high accuracy can be selected as the target for 3D data generation.

[0054] Furthermore, according to the first embodiment, a sensitivity coefficient ζ is set, which is a coefficient that represents the extent to which the measured diameter of the column shape is affected by error. Then, based on the depth distance DD, the depth limit distance DL, and the sensitivity coefficient ζ, an angle α is calculated as a parameter for estimating the expected accuracy for each column shape. If the angle α falls within a range that indicates lower accuracy than the preset accuracy, that column shape is excluded. In this way, even trees located at relatively close distances are excluded if they have diameters that are susceptible to error. Therefore, trees that can be modeled with high accuracy can be selected as targets for 3D data generation.

[0055] Furthermore, according to the first embodiment, the area of ​​trees as columnar shapes reflected in the image data is separated from the area of ​​the ground. As a result, 3D data is generated in which the area of ​​trees has a higher resolution than the area of ​​the ground. By differentiating the resolution according to the required accuracy, high-quality 3D data can be obtained with an appropriate amount of data.

[0056] Furthermore, according to the first embodiment, the area of ​​the tree as a columnar shape reflected in the image data is separated from other areas. The resolution of other areas in the image data is then degraded, and 3D data is generated based on this degradation. By degrading areas of relatively low necessity, it is possible to obtain high-quality 3D data with an appropriate amount of data.

[0057] (Second Embodiment) As shown in Figures 9 and 10, the second embodiment is a modified version of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

[0058] In the second embodiment, the recognition of bark pixels for depth estimation will be described in detail. The far-field data exclusion unit F1 uses machine learning to recognize bark pixels from image data. Specifically, the bark feature matrix, which is a matrix representing the characteristics of the bark, is inspected using an artificial intelligence model that has been pre-trained by machine learning. As a result of this inspection, pixels that capture bark are labeled. As shown in Figure 8, the bark feature matrix is ​​a matrix defined, for example, by primary color values, i.e., RGB values, for representing the color of the pixel, and the direction of their gradient. The bark features can also be described as the texture of the tree bark.

[0059] The artificial intelligence model is, for example, a model trained using deep learning on images of the bark of various types of trees. The tree types here include cedar, cypress, larch, etc. The tree types may also include age, as bark characteristics differ depending on age. Learning involves training the AI ​​model with defined parameters using tree images and their types as training data.

[0060] The image processing apparatus 10 of the second embodiment includes a bark feature database (hereinafter referred to as the bark feature DB) 10g for utilizing this artificial intelligence model. The bark feature DB 10g is configured to include at least one type of non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, and an optical medium. The bark feature DB 10g stores the parameters defined in the artificial intelligence model.

[0061] Next, using the flowchart in Figure 10, an example of a method for processing image data using the image processing device 10, particularly for recognizing pixels of tree bark, will be explained. Steps S211 to S212 of this flowchart may be implemented by the CPU 10a executing a computer program stored in memory 10b.

[0062] In the first step, S211, the remote data exclusion unit F1, after acquiring image data, examines the bark feature matrix using parameters defined in an artificial intelligence model trained from the bark feature DB10g. That is, it inputs the image data into the artificial intelligence model. In S212, following the processing in S211, the remote data exclusion unit F1 labels the pixels. For example, if the features of a certain pixel are the closest to the bark feature matrix of a 30-year-old cedar tree, the label "30-year-old cedar tree" is added to that pixel as data. The series of processes ends with S212.

[0063] According to the second embodiment described above, a bark feature DB10g is further provided as a database that stores bark features for identifying tree species. Image data is examined based on bark features and labeled with tree species. The depth distance DD of the column shape is estimated using the label. Then, column shapes unsuitable for generating 3D data are identified using the depth distance DD. Since the depth distance DD is estimated using bark features, trees that should be removed in a forest where trees with similar column shapes grow endlessly can be identified, making it possible to appropriately generate 3D data of the forest.

[0064] (Third embodiment) As shown in Figure 11, the third embodiment is a modified version of the second embodiment. The third embodiment will be described focusing on the differences from the second embodiment.

[0065] In the third embodiment, the process of estimating depth and determining the area in which trees are captured will be described in detail. The distant data exclusion unit F1 uses machine learning to estimate depth from image data. Specifically, the depth of the trees is estimated using an artificial intelligence model that has been pre-trained by machine learning using statistical data of trees.

[0066] More specifically, the tree statistics include data on tree dimensions for each tree species, particularly trunk diameter. The trunk diameter data may represent the trunk diameter distribution of that tree species, or it may represent at least one of the mean and median diameters. Here, diameter may be associated with tree age. Furthermore, the statistics may include data representing the age distribution of that tree, or it may include at least one of the mean and median ages.

[0067] The learning process involves learning width regression data to estimate the depth of trees captured in image data, using data on trunk width and trunk diameter in image data labeled with tree species.

[0068] The image processing apparatus 10 of the third embodiment further includes a tree statistics database (hereinafter referred to as the tree statistics DB) 10h in order to utilize this artificial intelligence model. The tree statistics DB 10h is configured to include at least one type of non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, and an optical medium. The tree statistics DB 10h stores statistical data for each type of tree.

[0069] The remote data exclusion unit F1 may refer to the labels assigned to the pixels, obtain statistical data of the type corresponding to the labels from the tree statistical information DB10h, learn width regression data, and estimate the depth. Alternatively, the remote data exclusion unit F1 may refer to the labels assigned to the pixels, obtain width regression data of the type corresponding to the labels from the tree statistical information DB10h, and estimate the depth from said width regression data.

[0070] Next, using the flowchart in Figure 11, an example of a method for processing image data using the image processing device 10, specifically a method for estimating the depth of a tree after recognizing pixels of bark, will be explained. Step S311 of this flowchart may be realized by the CPU 10a executing a computer program stored in memory 10b.

[0071] In S311, the far-field data exclusion unit F1 acquires pixel information labeled with the tree type. Then, according to the machine learning specifications, the far-field data exclusion unit F1 estimates the depth at that pixel from the width regression data.

[0072] According to the third embodiment described above, a tree statistics information DB10h is further provided as a database that stores statistical data regarding the diameter of tree trunks. The depth distance DD of the column shape is estimated using the width of the column shape in the image data and the statistical data. The accuracy of the estimation of the depth distance DD can be improved by using the statistical data regarding the diameter of tree trunks. By using the depth distance DD thus estimated, column shapes unsuitable for generating 3D data can be identified, making it possible to appropriately generate 3D data of the forest.

[0073] (Other embodiments) Although several embodiments have been described above, this disclosure is not limited to those embodiments and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0074] In another embodiment, the point cloud may be constructed with similar quality in the tree trunk region and the ground region. That is, the tree trunk processing unit F3 may downgrade the pixels of the tree trunk, while the ground processing unit F4 may not downgrade the pixels of the ground.

[0075] In another embodiment, the 3D data for modeling a forest may consist of representation methods other than point clouds. For example, the 3D data may consist of lines, curved surfaces, three-dimensional objects, and combinations thereof.

[0076] In another embodiment, at least one of the database 10f, bark feature DB 10g, and tree statistics DB 10h may be located outside the image processing device 10. In this case, the image processing device 10 can access at least one of the database 10f, bark feature DB 10g, and tree statistics DB 10h via wired or wireless communication and perform processing.

[0077] In another embodiment, the computer comprising the image processing apparatus 10 may include another type of processor (e.g., a GPU) in place of, or together with, the CPU 10a.

[0078] In another embodiment, the computer constituting the image processing device 10 may incorporate circuits such as FPGAs, and a portion of the processing of the image processing device 10 may be realized by the processor controlling the circuits based on a computer program, or by the circuits operating independently. [Explanation of symbols]

[0079] 10: Image processing unit, 10a: CPU (processing unit), T1~T9: Tree (columnar shape)

Claims

1. An image processing apparatus comprising at least one processing unit (10a), which processes image data to generate three-dimensional data of a forest from image data of a forest, The at least one processing unit is, Among the multiple column shapes (T1 to T9) captured in the aforementioned image data, the column shapes unsuitable for generating the three-dimensional data are identified as targets for exclusion. An image processing apparatus configured to perform the following: generate the three-dimensional data from the portion of the image data from which the exclusion target has been removed.

2. The aforementioned determination means that To estimate the depth distance (DD) of multiple column shapes, The depth limit distance (DL) is set based on the performance of the camera (30) that captured the aforementioned image data, The image processing apparatus according to claim 1, comprising: comparing the depth distance of each column shape with the depth limit distance, and identifying the column shapes that are estimated to be farther than the depth limit distance as targets for exclusion.

3. The aforementioned exclusion means A sensitivity coefficient (ζ) is set, which is a coefficient that represents the extent to which the measured diameter of the column shape is affected by error, The image processing apparatus according to claim 2, comprising: calculating a parameter (α) for estimating the expected accuracy for each column shape based on the depth distance, the depth limit distance, and the sensitivity coefficient; and identifying the column shape as an exclusion target if the parameter falls within a range indicating that the accuracy is lower than a preset accuracy.

4. The generation of the aforementioned three-dimensional data is To separate the area of ​​the tree as a column shape that appears in the aforementioned image data from the area of ​​the ground, The image processing apparatus according to claim 1, comprising generating three-dimensional data in which the area of ​​the trees has a higher resolution than the area of ​​the ground.

5. The generation of the aforementioned three-dimensional data is To separate the area of ​​the tree as a column shape that appears in the aforementioned image data from other areas, The image processing apparatus according to claim 1, comprising degrading the resolution of the other regions of the image data to generate the three-dimensional data.

6. It also includes a database (10g) that stores bark characteristics for identifying tree species, The aforementioned determination means that The image data is examined based on the characteristics of the bark, and the type of tree is labeled on the image data. Using the aforementioned labels, the depth distance of the column shape is estimated, The image processing apparatus according to claim 1, comprising using the depth distance to determine the column shape unsuitable for generating the three-dimensional data.

7. It also includes a database (10h) that stores statistical data on the diameter of tree trunks, The aforementioned determination means that The depth distance of the column shape is estimated using the width of the column shape in the image data and the statistical data. The image processing apparatus according to claim 1, comprising using the depth distance to determine the column shape unsuitable for generating the three-dimensional data.

8. An image processing program for processing image data to generate three-dimensional data of a forest from image data of a forest, At least one processing unit (10a) Among the multiple column shapes (T1 to T9) captured in the aforementioned image data, the column shapes unsuitable for generating the three-dimensional data are identified as targets for exclusion. An image processing program that generates the three-dimensional data from the portion of the aforementioned image data from which the exclusion target has been removed.

Images