Tree management system, control program for tree management system, and tree management method

Abstract

To provide a tree management technique capable of simplifying management including reduction in labor and time.SOLUTION: A tree management system includes: a stereo camera 11; position acquisition means 13 for acquiring a position of the stereo camera; direction acquisition means 14 for acquiring a photographic direction of the stereo camera; display means 15 for displaying a photographic image by the stereo camera; and control means 16 for input / outputting information to / from the stereo camera, the position acquisition means, the direction acquisition means and the display means, and performing information processing. The tree management system photographs a tree as a target by the stereo camera 11, displays the image of the tree as the target on the display means 15, sets a reference point in the image, estimates a photographic image between the place of the tree corresponding to the reference point and the stereo camera, estimates a tree position on the basis of the position of the stereo camera, the photographic direction of the stereo camera, and the photographic distance, estimates a dimension of the tree on the basis of the photographic distance and the image of the tree, and records the tree position and the dimension of the tree.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for managing trees, and more particularly to a technique for managing trees on streets, in parks or in gardens. [Background technology]

[0002] Urban greenery trees, such as street trees, park trees, and garden trees, are important elements that form the urban landscape, just like buildings and bridges. Urban greenery trees are also important elements that make up the urban environment, providing shade and suppressing the heat island effect. On the other hand, while buildings and bridges change little over time, trees grow. Therefore, long-term management is important.

[0003] If the trees are not growing fast enough, it is necessary to take measures to promote their growth, and if they are growing too fast, they may become an obstacle to vehicle traffic, for example. Therefore, it is common to manage the condition of trees over time (for example, every three years) and record it in a ledger. This ledger is also useful in urban redevelopment and surrounding construction projects.

[0004] Meanwhile, trees in forests are also managed. For example, by estimating the total number of trees in a forest, it is possible to estimate the amount of carbon dioxide absorbed by the trees. By mounting a laser scanning device on an aircraft and acquiring reflections, it is possible to perform three-dimensional measurements and estimate the shape of trees (see, for example, Patent Document 1). Alternatively, by placing multiple laser scanning devices on the ground and acquiring reflections, it is possible to perform three-dimensional measurements and estimate the shape of trees (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-111725 [Patent Document 2] Patent No. 5844438 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the purposes for planting trees for urban greening and forest trees are different, and the objectives for managing them are somewhat different. Specifically, measurements in forest tree management are aimed at obtaining information on timber production volume as part of forestry, trunk diameter as an indicator of timber volume, the environment for thickness, and the appropriate spacing between trees for light absorption. In contrast, the purpose of urban greening trees is to obtain information on tree shape (height, spread, diameter), as well as their placement and location, to show that they have a neat and stable tree shape. Therefore, it is not advisable to apply tree management methods used in forests as is.

[0007] In particular, urban greening trees are part of the city, and it is preferable to manage the trees simply so as not to interfere with urban activities such as traffic.

[0008] Conventionally, multiple workers measure dimensions using ropes, poles, height meters, etc., and record the measurement results in a ledger. Tree locations are then checked against map information separately. This series of tasks is cumbersome and requires a great deal of labor. For example, two people are required for the measurement work and one person for the recording work. Therefore, there is a demand for simplification, such as reducing the labor and time required.

[0009] The present invention is intended to solve the above problems, and aims to provide a simple technique for managing urban greening trees and the like. [Means for solving the problem]

[0010] The present invention, which solves the above problem, is a tree management system that includes a stereo camera, a position acquisition means for acquiring the position of the stereo camera, a direction acquisition means for acquiring the shooting direction of the stereo camera, a display means for displaying images captured by the stereo camera, and a control means for inputting and outputting information to and from the stereo camera, the position acquisition means, the direction acquisition means, and the display means, and for processing the information.

[0011] By photographing the target tree with a stereo camera, the tree's position and shape can be estimated. This is simpler than conventional management methods.

[0012] In the above invention, preferably, the control means includes a display processing unit that displays an image of the target tree on the display means, a reference point setting processing unit that sets a reference point in the image, a shooting distance estimation processing unit that estimates the shooting distance between the point on the tree corresponding to the reference point and the stereo camera, a tree position estimation processing unit that estimates the tree position based on the position of the stereo camera, the shooting direction of the stereo camera, and the shooting distance, and a tree dimension estimation processing unit that estimates the tree dimensions based on the shooting distance and the image of the tree.

[0013] This allows the tree position and shape to be estimated based on the shooting distance.

[0014] In the above invention, preferably, the control means has a shape extraction processing unit that extracts tree shapes by dividing the image into tree-corresponding areas and background areas, and the tree dimension estimation processing unit sets measurement points based on the extracted tree shapes.

[0015] This allows the measurement points to be set automatically.

[0016] In the above invention, preferably, there is further provided a pixel designation processing section for designating an arbitrary pixel in the image, and the tree size estimation processing section sets a measurement point based on the designated pixel.

[0017] This allows you to manually set measurement points if automatic setting is not desirable.

[0018] In the above invention, preferably, the control means has a measurement point correction unit that moves the measurement points slightly in the image, and the tree dimension estimation processing unit estimates the dimensions of the tree based on the corrected measurement points.

[0019] This allows you to set measurement points even if an unexpected malfunction occurs.

[0020] In the above invention, preferably, the control means has a date and time acquisition section and a recording section, and the recording section records the management date and time and tree dimensions for each tree position.

[0021] This allows a ledger to be created by photographing target trees with a stereo camera.

[0022] The present invention, which solves the above-mentioned problems, provides a control program for a tree management system. The tree management system includes a stereo camera, a position acquisition means for acquiring the position of the stereo camera, a direction acquisition means for acquiring the imaging direction of the stereo camera, and a display means for displaying an image captured by the stereo camera. The control program executes a display process for displaying an image of a target tree on the display means, a reference point setting process for setting a reference point in the image, a shooting distance estimation process for estimating the shooting distance between the stereo camera and a point on the tree corresponding to the reference point, a tree position estimation process for estimating the position of the tree based on the position of the stereo camera, the imaging direction of the stereo camera, and the shooting distance, and a tree dimension estimation process for estimating the dimensions of the tree based on the shooting distance and the image of the tree.

[0023] By photographing the target tree with a stereo camera, the tree's position and shape can be estimated. This is simpler than conventional management methods.

[0024] The present invention, which solves the above-mentioned problems, provides a tree management method using a tree management system. The tree management system includes a stereo camera, a position acquisition means for acquiring the position of the stereo camera, a direction acquisition means for acquiring the imaging direction of the stereo camera, a display means for displaying images captured by the stereo camera, and a control means for inputting and outputting information to and from the stereo camera, the position acquisition means, the direction acquisition means, and the display means, and for information processing. The method includes: capturing images of a target tree using the stereo camera; displaying the image of the target tree on the display means; setting a reference point in the image; estimating the imaging distance between the stereo camera and a point on the tree corresponding to the reference point; estimating the position of the tree based on the position of the stereo camera, the imaging direction of the stereo camera, and the imaging distance; and estimating the tree dimensions based on the imaging distance and the image of the tree.

[0025] By photographing the target tree with a stereo camera, the tree's position and shape can be estimated. This is simpler than conventional management methods. [Effects of the Invention]

[0026] The technology according to the present invention can simplify management by reducing the amount of labor and time required. [Brief explanation of the drawings]

[0027] [Figure 1] System configuration diagram [Figure 2] Functional Block Diagram [Figure 3] Control Flow Diagram [Figure 4] Image of practical example [Figure 5] Display screen example image [Figure 6] Virtual plane setting image [Figure 7] Measurement point setting image [Figure 8] Image of estimated horizontal width [Figure 9] Image of measurement point correction example [Figure 10] Image of measurement point correction example DETAILED DESCRIPTION OF THE INVENTION

[0028] ~Basic configuration~ 1 is a schematic diagram of the system according to this embodiment. The system is made up of a stereo camera 11 and a mobile terminal 12 such as a smartphone, a tablet terminal, or a notebook computer.

[0029] Commercially available mobile terminals 12 are equipped as standard with a GPS sensor 13, a magnetic sensor 14, a display 15, a calculation function such as a CPU 16, a memory function (recording function) 17, a communication function 18, and a date and time acquisition function (not shown).

[0030] A commercially available mobile terminal 12 has the above functions 13 to 18 in one device, which is inexpensive, practical and suitable, but the above functions 13 to 18 may also be separate and independent devices.

[0031] The stereo camera 11 is integrated by connecting it to an input / output port of the mobile terminal 12. At the time of filing, there are few commercially available mobile terminals equipped with the stereo camera 11 to be connected, but the stereo camera 11 may be equipped in the mobile terminal 12.

[0032] The stereo camera 11 can obtain depth information (photographing distance) of the subject based on parallax information from the two cameras arranged side by side.

[0033] The GPS sensor 13 receives radio waves transmitted from an artificial satellite (GPS satellite) and identifies the camera position (latitude and longitude). The magnetic sensor 14 detects magnetic force and identifies the shooting direction (orientation).

[0034] Additionally, the mobile terminal 12 is equipped with a gyro sensor, which may be used to identify the shooting direction (elevation angle). For the sake of simplicity, it is assumed that there is no shooting elevation angle.

[0035] The display 15 displays a composite image from the stereo camera 11, an operation panel, etc. It also functions as an input device by displaying a touch panel, a cursor, etc. The input device may be provided separately.

[0036] The calculation function 16 inputs and outputs information to and processes information from the stereo camera 11, GPS sensor 13, magnetic sensor 14, display 15, memory function 17, and communication function 18. In other words, it is a control device.

[0037] The recording function 17 records the results of predetermined calculations performed by the calculation function 16. The processing of the calculation function 16 is executed by a program stored in the recording function 17.

[0038] A part of the calculation function 16 and the recording function 17 may be located in the server 19 via the communication function 18 or the network. Processing may also be performed only by the mobile terminal 12 without communication.

[0039] ~Operation and control overview~ Fig. 2 is a functional block diagram of the control device 16 according to this embodiment, and Fig. 3 is a control flow diagram of each process executed by the arithmetic function 16. The operation of this invention will be explained together with the control.

[0040] The control device 16 has a display processing unit 21, a reference point setting processing unit 22, a shooting distance estimation processing unit 23, a tree position estimation processing unit 24, a tree dimension estimation processing unit 25, and a recording unit 26.

[0041] Fig. 4 is an image diagram of an example of a practical application of the system. Fig. 5 is an image diagram of an example of a display screen on the display 15. The stereo camera 11 is placed in a position where it can photograph a target tree, and the tree is photographed (step S1). The image photographed by the stereo camera 11 is displayed on the display 15 by the display processing unit 21 (step S2). The photographing position is adjusted so that the entire target tree is included, but is not excessively small.

[0042] FIG. 6 is an image diagram relating to the setting of the virtual plane.

[0043] Incidentally, urban green trees such as street trees, park trees, and garden trees have a more consistent shape than trees in forests, etc., and their trunks can often be seen with the naked eye at breast height (approximately 1.2 m above ground level). As an example, this position is used as the reference point in this system.

[0044] Specifically, a cursor is displayed on the display screen, and the stereo camera 11 is adjusted so that the cursor position coincides with the reference point. Alternatively, the cursor may be moved to the reference point. The reference point setting processor 22 associates the reference point of the tree with the reference point on the image (step S3). A virtual plane is set that includes the reference point and is perpendicular to the shooting direction.

[0045] The reference point may be reset to the breast height position obtained by estimating tree dimensions (described later), or may be reset taking into account the trunk radius obtained by estimating tree dimensions.

[0046] The shooting distance estimation processing unit 23 estimates the shooting distance between the reference point and the stereo camera based on the principle of the stereo camera 11 (step S4).

[0047] Tree position estimation processing unit 24 receives camera position information via GPS sensor 13 (step S5) and shooting direction information via magnetic sensor 14 (step S6). Furthermore, it receives the shooting distance from shooting distance estimation processing unit 23 and estimates the tree position (e.g., latitude and longitude) based on the camera position, shooting direction, and shooting distance (step S7).

[0048] The tree size estimation processing unit 25 estimates the size of the tree based on the set measurement points (step S8) and the images captured by the stereo camera 11 (step S9), as will be described in detail later.

[0049] The recording unit 26 receives the position information from the tree position estimation processing unit 24 and the shape information from the tree dimension estimation processing unit 25, and outputs them to the recording device 17 (step S10).

[0050] ~Tree size estimation~ Figure 7 shows an image of the measurement point settings. In conventional registers, tree shape is managed by tree height, branch spread, height to bottom of branches, and trunk circumference (or diameter at breast height). The system of this application manages tree shape in the same way as conventional registers.

[0051] The image information from the stereo camera 11 can be used to extract a target tree region from a background image based on semantic segmentation using differences in shooting distance. A target tree region generally consists of a relatively thin lower branch (trunk) and an approximately triangular upper crown. Therefore, extraction is relatively easy. The target tree region can also be extracted by adapting existing technology for extracting subjects such as people from a background image.

[0052] The tree dimension estimation processing unit 25 has a measurement point setting unit. The top of the crown and the bottom of the trunk are set as measurement points, and the distance between the measurement points is set as tree height. Measurement points on the left and right where the crown is at its maximum horizontal width are set, and the distance between the measurement points is set as branch spread. Branch spread can be in the east-west direction or the north-south direction, but at least one of these directions must be selected. The bottom of the crown and the bottom of the trunk are set as measurement points, and the distance between the measurement points is set as branch height. Measurement points on the left and right where the trunk is at its horizontal width at breast height are set, and the distance between the measurement points is set as breast height diameter. Furthermore, the breast height diameter is multiplied by pi to obtain the trunk circumference.

[0053] Measurement points may be set automatically by extraction from the target tree area, or they may be set manually via the display 15, for example, using a touch panel. The pixel corresponding to the point specified via the touch panel is set as the measurement point.

[0054] Furthermore, the tree dimension estimation processing unit 25 may have a measurement point correction unit. Specifically, a measurement point to be corrected is selected and slightly moved using an operation panel (e.g., a <> icon) displayed on the display 15 (details will be described later). Automatically extracted measurement points may also be slightly moved.

[0055] Figure 8 is an image diagram for estimating branch spread (horizontal width). The left and right measurement points for branch spread are assumed to be on an imaginary plane.

[0056] The tree dimension estimation processing unit 25 acquires the angle made by the line extending from the camera 11 to the left measurement point and the angle made by the line extending from the camera 11 to the right measurement point. Furthermore, the shooting distance is input from the shooting distance estimation processing unit 23, and the distance between the left and right measurement points is estimated from the geometric relationship.

[0057] Alternatively, similar to the processing by the tree position estimation processing unit 24 (step S7), the positions (for example, latitude and longitude) of the left and right measurement points may be estimated, and the distance between the measurement points may be estimated from the relative positions of the left and right measurement points.

[0058] The tree size estimation processor 25 associates the branch spread dimension with the number of pixels in the image information, and estimates the tree height from the number of pixels corresponding to the tree height. Similarly, it estimates the clear branch height from the number of pixels corresponding to the clear branch height.

[0059] The tree size estimation processing unit 25 may estimate the diameter at breast height from a geometric relationship, as in estimating the branch spread, or may estimate the diameter at breast height based on the correspondence with the number of pixels, as in estimating the tree height and the height to the lowest branch. Furthermore, the trunk circumference is estimated by multiplying the diameter at breast height by pi.

[0060] Furthermore, roadside trees, park trees, or garden trees are artificial, and compared to natural trees, their shapes are not complicated, and the distance between trees is maintained, so that the tree dimensions can be easily estimated as described above.

[0061] ~Measurement point correction~ Figure 9 is an image diagram of an example of measurement point correction in the tree size estimation processing unit 25. In general, coniferous trees tend to grow straight and have a conical tree shape, while broad-leaved trees tend to branch and have a rounded, bulging tree shape. As a result, while the top of the crown of a coniferous tree can be seen with the naked eye, the top of the crown of a broad-leaved tree may not be visible.

[0062] In this system, tree height is estimated by setting a virtual plane, so if the top of the target tree area in the image is used as the measurement point, there is a risk of overestimation. The upper measurement point is slightly moved downward based on the measurer's estimation. This prevents overestimation.

[0063] FIG. 10 is an image diagram of another example of measurement point correction in the tree size estimation processing unit 25.

[0064] As a general rule, when photographing trees, care should be taken to position the camera so that the entire tree can be seen. In particular, if there are shrubs around, the camera should be positioned so that the bottom of the trunk can be seen. However, there may be circumstances in which the bottom of the trunk cannot be seen.

[0065] In this system, tree height is estimated by setting a virtual plane, so if the measurement point is set at the bottom of the target tree area in the image, there is a risk of underestimation. The lower measurement point is moved slightly downward based on the measurement person's estimation. This prevents underestimation.

[0066] The fact that the measurement point has been corrected is recorded in the recording device 17, and a warning is issued the next time the measurement point is corrected. Furthermore, the amount of slight movement of the previous measurement point is also notified. This makes it possible to reduce variations due to differences in the person making the measurement.

[0067] Alternatively, another correction means may be provided. For example, the display processing unit 21 may be video compatible. In other words, multiple captured images can be acquired per unit time. When a tree vibrates, the image showing the highest point of the tree crown is selected from the multiple captured images, and the tree height is estimated based on that image. This improves the estimation accuracy. At this time, the tree may be vibrated by another worker.

[0068] ~Ledger~ When workers take photos with their cameras, the date and time of care, tree location information, shape information, and captured images are automatically recorded. A tree registration number is assigned to each location. Workers can also write notes on whether the tree is diseased, its growth status, whether pruning has been performed, etc.

[0069] The actual work is simply for the worker to take the photos. The worker determines the best shooting position, and after photographing one tree, he or she repeats the process, photographing the next tree, until multiple street trees, park trees, or garden trees are photographed. At this time, a separate map database may be linked to compare the tree location information with map information.

[0070] By the above series of operations, a ledger is automatically formed in the recording device 17. The management ledger may be displayed as a list or as a map.

[0071] The above usage example assumes that workers will photograph each tree, but if possible, the system may also be mounted on an autonomous vehicle or an autonomous drone.

[0072] By managing the data over time (for example, every three years), it is possible to compare the shape information and photographed images from each management time and check the changes in the trees over time. If the capacity of the recording device 17 is low, the previously managed data may be overwritten with new data.

[0073] Incidentally, among street trees, the uniform beauty of the trees lining the streets is particularly important. According to the original register, the position, height, height below the branches, and branch spread of each tree can be reproduced. This makes it possible to confirm the uniformity of the trees lining the streets. Furthermore, the spacing between trees in the streets is set to ensure an appropriate amount of greenery. If trees grow more than expected, it will actually spoil the urban landscape. According to the original register, it is possible to reproduce the road landscape and the trees lining the streets together, making it possible to confirm whether a harmonious urban landscape has been formed.

[0074] ~Effects~ While conventional work involves multiple tasks such as measuring tree positions, measuring tree dimensions, taking photographs, and recording in a ledger, this system estimates tree positions, tree dimensions, and creates a ledger using only camera photography. It has been proven that the accuracy of estimating tree positions and tree dimensions is on the same level as conventional measurements. In other words, it has sufficient accuracy to achieve the specified objectives in tree management.

[0075] As a result, this system can simplify operations by reducing labor and time. For example, it can currently be operated by a minimum of one person. In the future, it may be possible to make it unmanned. [Explanation of symbols]

[0076] 11 Stereo Camera 12 Mobile devices 13 GPS sensor 14 Magnetic Sensor 15 Display 16 Calculation function (control device) 17 Memory function (recording device) 18 Communication Functions 19 External Servers 21 Display processing section 22 Reference point setting processing section 23 Shooting distance estimation processing unit 24 Tree position estimation processing unit 25 Tree size estimation processing unit 26 Recording Section

Claims

1. A stereo camera and a position acquisition means for acquiring a position of the stereo camera; a direction acquisition means for acquiring a photographing direction of the stereo camera; a display means for displaying an image captured by the stereo camera; A control means for inputting and outputting information to and from the stereo camera, the position acquisition means, the direction acquisition means, and the display means, and for processing the information. and The control means a display processing unit that displays an image of the target tree on the display means; a reference point setting processing unit that sets a reference point in the image; a shooting distance estimation processing unit that estimates a shooting distance between a location of the tree corresponding to the reference point and the stereo camera; a tree position estimation processing unit that estimates tree positions based on the positions of the stereo cameras and the imaging directions and imaging distances of the stereo cameras; a tree dimension estimation processing unit that sets two ends of a tree in the vertical or horizontal direction as two measurement points, and estimates the distance between the measurement points in the vertical or horizontal direction as the dimension of the tree based on the angle formed by a line from the stereo camera to one measurement point and the angle formed by a line from the stereo camera to the other measurement point, and the shooting distance. A tree management system equipped with:

2. the control means has a shape extraction processing unit that separates the image into a tree-corresponding portion and a background portion and extracts tree shapes; The tree size estimation processing unit sets measurement points based on the extracted tree shape. The tree management system of claim 1.

3. the control means has a pixel designation processing unit that designates an arbitrary pixel in the image, The tree size estimation processing unit sets measurement points based on designated pixels. The tree management system of claim 1.

4. the control means has a measurement point correction unit that slightly moves the measurement point on the image, The tree size estimation processing unit estimates the size of the tree based on the corrected measurement points. The tree management system according to claim 2 or 3.

5. the control means has a date and time acquisition unit and a recording unit, The recording unit records the management date and time and tree dimensions for each tree position. A tree management system according to any one of claims 1 to 4.

6. The tree is a street tree, a park tree, or a garden tree. A tree management system according to any one of claims 1 to 5.

7. A stereo camera and a position acquisition means for acquiring a position of the stereo camera; a direction acquisition means for acquiring a photographing direction of the stereo camera; a display means for displaying an image captured by the stereo camera; A control means for inputting and outputting information to and from the stereo camera, the position acquisition means, the direction acquisition means, and the display means and processing the information. A program for controlling a tree management system comprising: a display process for displaying an image of the target tree on the display means; a reference point setting process for setting a reference point in the image; a photographing distance estimation process for estimating a photographing distance between a location of the tree corresponding to the reference point and the stereo camera; a tree position estimation process for estimating tree positions based on the positions of the stereo cameras and the imaging directions and imaging distances of the stereo cameras; A tree dimension estimation process in which two ends of a tree in the vertical or horizontal direction are set as two measurement points, and the distance between the measurement points in the vertical or horizontal direction is estimated as the tree dimension based on the angle formed by a line from the stereo camera to one measurement point and the angle formed by a line from the stereo camera to the other measurement point, and the shooting distance. A control program that causes a computer to execute the above.

8. A stereo camera and a position acquisition means for acquiring a position of the stereo camera; a direction acquisition means for acquiring a photographing direction of the stereo camera; a display means for displaying an image captured by the stereo camera; A control means for inputting and outputting information to and from the stereo camera, the position acquisition means, the direction acquisition means, and the display means and processing the information. With a tree management system that includes displaying an image of a target tree on the display means; setting a reference point in the image; Estimating a shooting distance between a location of the tree corresponding to the reference point and a stereo camera; estimating tree positions based on the positions of the stereo cameras and the imaging directions and imaging distances of the stereo cameras; Two ends of the tree in the vertical or horizontal direction are set as two measurement points, and the distance between the measurement points in the vertical or horizontal direction is estimated as the dimension of the tree based on the angle formed by a line from the stereo camera to one measurement point and the angle formed by a line from the stereo camera to the other measurement point, and the shooting distance. Tree management methods.

Images