Tree Management System

Abstract

This facilitates the correspondence between each displayed tree and each tree in the actual location. [Solution] The system includes a 3D imaging unit (3D scanner) 10 that acquires a 3D point cloud of a target area where multiple trees exist by scanning the area from multiple directions; a tree recognition unit 24-1 that performs point cloud recognition processing on the acquired 3D point cloud to recognize trees; a skeleton recognition unit 24-2 that recognizes the skeleton corresponding to the trunk and thick branches and the branches and leaves separately based on the thickness of each part of the recognized tree; a 3D storage unit 34 that stores the 3D point cloud of the skeleton in association with each tree in the target area; and a display unit 44 that displays the 3D point cloud of the target area in 3D, emphasizing the recognized skeleton.

Description

Technical Field

[0001] The present disclosure relates to a tree management system for managing trees in parks and the like.

Background Art

[0002] Conventionally, in parks and the like, a management ledger has been created to manage trees. For each tree within the management area, its diagnosis results and the like are recorded to oh understand the condition of the tree and formulate plans for preservation and the like.

[0003] Here, even in such tree management, the use of information processing devices such as computers has been progressing, and it has also been proposed to include data including images of each tree in the management ledger. In addition, it has also been proposed to make the management ledger accessible on a portable terminal so that the condition of the tree can be input or an image of the tree can be displayed thereon.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, in parks and the like, there are often many trees planted. In such a case, even if it is possible to display data for each tree described in the management ledger on a floor plan on a terminal device, it is often difficult to determine which tree on the management ledger corresponds to which actual tree.

Means for Solving the Problems

[0006] The tree management system described herein includes: a 3D imaging unit that acquires 3D point clouds of a target area where multiple trees exist by photographing the area from multiple directions; a tree recognition unit that performs point cloud recognition processing on the acquired 3D point clouds to recognize trees; a skeleton recognition unit that recognizes the skeletons corresponding to the trunk and thick branches and the branches and leaves separately based on the thickness of each part of the recognized tree; a 3D storage unit that stores the 3D point clouds of the skeletons in association with each tree in the target area; and a display unit that displays the 3D point cloud of the target area in 3D, emphasizing the recognized skeletons.

[0007] It is recommended to determine the latitude and longitude of the tree's location by comparing it with separately prepared map data.

[0008] It would be ideal if the skeleton and the branches and leaves could be displayed on separate layers and processed independently.

[0009] The target area should be part or all of the park. [Effects of the Invention]

[0010] According to the tree management system described in this disclosure, the skeletal structure of each tree is emphasized in the display, making it easy to associate each displayed tree with each tree in the field. [Brief explanation of the drawing]

[0011] [Figure 1] This block diagram shows the schematic configuration of a tree management system according to an embodiment. [Figure 2] This shows an example of 3D point cloud recognition of a target area. [Figure 3] This is a point cloud that highlights the skeletal structure of recognized trees, showing the skeletal structure and branches / leaves in different colors. [Figure 4] This diagram shows only the framework, with branches, leaves, and other geographical features removed, using the current plan view as a reference. [Figure 5] This is an example image showing a display for a single tree. [Figure 6] This is a flowchart of the skeletal structure recognition process. [Modes for carrying out the invention]

[0012] The embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are not limiting to this disclosure, and configurations formed by selectively combining multiple examples are also included in this disclosure.

[0013] "System Configuration" Figure 1 is a block diagram showing the schematic configuration of a tree management system according to an embodiment.

[0014] First, a 3D scanner 10, which is a 3D imaging unit, is provided as an input device. The 3D scanner 10 measures the shape of an object in three dimensions, and the 3D measurement data is acquired as a collection of points with coordinate values ​​called a point cloud. The 3D scanner 10 also has a camera, and it is preferable to use a scanner that acquires a point cloud with color information added from image information captured simultaneously with the point cloud.

[0015] In this embodiment, the target area is part or all of a park with trees. For example, a worker can actually go to the target park and perform a 3D scan on-site. In this embodiment, in order to improve the accuracy of analyzing the skeleton of trees, such as trunks and thick branches, it is advisable to (i) ensure that the skeleton of the trees is captured in the image, (ii) scan the entire park with minimal blind spots, and (iii) capture images up to the top of the tree canopy. Furthermore, 3D scanning can be performed by various means, such as from the air, using robots, or using vehicles.

[0016] The measurement data from the 3D scanner 10 is supplied to the server computer 20. The server computer 20 has a communication unit 22 and a point cloud processing unit 24. The communication unit 22 controls information exchange with the outside. Also, the point cloud processing unit 24 performs various point cloud processes on the measurement data from the 3D scanner 10, such as individually recognizing trees, specifying the positions of the recognized trees, and processing the data for display. Note that map data is also supplied to the server computer 20, and the map data can be used for position recognition, combination of point clouds of facilities and roads, etc. Also, the image information acquired by the above-described camera can be used.

[0017] The communication unit 22 of the server computer 20 has various communication functions. The measurement data from the 3D scanner 10 may be supplied via a communication network such as the Internet, or may be supplied directly via a local data line.

[0018] The point cloud processing unit 24 of the server computer 20 has, as functional blocks, a tree recognition unit 24-1, a skeleton recognition unit 24-2, a display point cloud generation unit 24-3, and a tree position recognition unit 24-4. Note that these functional blocks are for convenience of separating the functions of the server computer 20, and may be one computer, or may be configured by a plurality of computers.

[0019] The tree recognition unit 24-1 performs point cloud recognition processing on the measurement data from the 3D scanner 10 to individually recognize trees. This can be done by using a conventionally known point cloud recognition technique, for example, by comparing the features of the stored trees with the point cloud to determine if there are matching features. Also, the recognition of trees can be performed using a point cloud recognition model configured by deep learning by AI.

[0020] The skeleton recognition unit 24-2 recognizes skeletons corresponding to the trunk and thick branches for the parts recognized as trees, at least according to the thickness for each part. Here, the skeleton of a tree is defined as thick branches and trunks of a certain thickness. In other words, the skeleton refers to the main trunks and thick branches with a strong functional aspect for supporting the tree body and determining the tree shape, and the branches and leaves refer to small branches and leaves collectively.

[0021] The recognition of the tree skeleton is also point cloud recognition similar to tree recognition, but by considering the following points, it can be analyzed with good cost performance in a short time. (i) Analyze by layer according to the height of the tree. That is, due to the height of the tree, there are characteristics such as layers that are easily blocked by the branches and leaves of ground cover and understory trees, and upper layers where the laser is difficult to reach, and each is processed separately. (ii) Analyze according to the scale sense of the tree. That is, analyze separately for each scale such as leaf unit, branch unit, trunk unit, individual unit, etc. (iii) Analyze according to the characteristics of the shape (planar or linear) of the parts of the tree (root, trunk, thick branch, small branch, leaf). (iv) Analyze according to the characteristics of the color of the parts of the tree and the reflection intensity of the laser. For example, using lasers of multiple wavelengths, point cloud analysis can be performed separately for each wavelength. (v) Analyze according to the integrity and connection of the tree as an individual. One individual grows continuously from the root to the trunk, thick branches, and branches and leaves. The characteristics of the connection method as that individual can be used for recognition. (vi) By the above combination methods, even complex and diverse trunk skeleton shapes can be extracted, and the recognition accuracy of the tree skeleton can be improved.

[0022] The display point cloud generation unit 24-3 creates display data based on the information recognized by the tree recognition unit 24-1 and the skeleton recognition unit 24-2.

[0023] In this embodiment, both 3D and 2D representations are used. The 3D representation is used to visually confirm the current shape in three dimensions, while the 2D representation is used for actual information manipulation. This is because 3D screens are difficult to operate, and using a 2D representation makes touch-based operation easier. Furthermore, adding a 2D representation to the 3D representation improves visibility. In this way, both representations are used simultaneously, each with its own main role. Additionally, using a 2D representation on the operation screen allows for smoother operation.

[0024] <3D representation> a. Create layers so that visibility can be toggled. For example, the skeleton and branches / leaves can be displayed on separate layers, allowing the display of branches / leaves to be toggled independently of the skeleton's display.

[0025] b. Emphasize and display the unique skeletal structure of local trees. For example, use color and shadow techniques to create a visually appealing effect. Also, highlight and represent features such as inclination, thickness, branching patterns, and curvature.

[0026] c. The average coordinates of the point cloud at the base of each tree (individual) are used as the position (2D coordinates) of that tree and plotted.

[0027] d. To emphasize the skeletal structure of the tree, minimize the depiction of branches and leaves, which tend to be distracting. However, it is good to make the general spatial extent of the branches and leaves clear.

[0028] e. Display other features as well. It would be good to represent the location of trees in a way that allows them to be inferred from their positional relationship with surrounding features. It would be good to use a height of about chest height as a reference point and display only the point cloud of features below that height.

[0029] f. For ground surface data, it is advisable to use not only point cloud data, but also a representation obtained in the 2D representation described in the next section, where the current plan view is superimposed onto the ground surface according to the terrain.

[0030] g. Allow major structures with height and presence to be displayed on separate layers.

[0031] <2D representation> An orthographic projection of the point cloud data of the ground and features up to breast height is created as a top-down view, and this is used as the base map (existing plan view). By utilizing the principle that point clouds of three-dimensional objects with height appear darker in color because the points overlap when viewed from above, the existing plan view can have a resolution that satisfies the following criteria.

[0032] a. The cross-sectional shape of the tree trunk is represented.

[0033] b. The outlines of shrubs and trimmed bushes are depicted.

[0034] c. The three-dimensional outlines of structures and other objects, which are necessary for understanding their relative positions, are represented.

[0035] d. Fine details such as curbs and steps that form the linear shape of the garden path are also rendered in a rich, detailed color.

[0036] <Screenshot of the screen> Figures 2-5 show examples of 3D point cloud recognition (screenshots). Note that although the images are in black and white, the actual data is in color, and color is an important factor in point cloud recognition.

[0037] Figure 2 shows the point cloud of raw data obtained from the 3D scanner 10, where trees and other features are not very clearly identifiable. Figure 3 shows the point cloud with recognized trees highlighted and the trunk and branches / leaves color-coded. Because the branches and leaves are displayed, the skeletal structure is not very clear.

[0038] Figure 4 shows the tree with the branches and leaves removed, making the trunk and other structural elements easier to see.

[0039] Figure 5 shows an example image of the display for a single tree.

[0040] Get with a scanner and othersFor point clouds, images obtained from cameras and map data, and videos of these, the following screen display is available: Possible Let's assume and good. (i) A screen that simultaneously displays existing outline diagrams or CAD drawings showing the layout of paths and green spaces, along with a 3D representation on two screens. (ii) A screen where the arrangement of trees is clearly visible in 3D representation. (iii) A screen that allows for the identification of trees by combining a 2D representation (current floor plan) and a 3D representation.

[0041] The tree position recognition unit 24-4 recognizes the 2D position of each tree by referring to separately supplied map data. The position information of the 3D scanner 10 is obtained from GPS, etc. Location detection function The system then uses GPS to determine the location of each tree, and the average coordinates of the base of the skeleton obtained from the skeleton recognition unit 24-2 are recognized as the position of each tree. Position at If location information is unavailable, the position of each tree can be determined by recognizing the skeleton and landmark locations from the 3D point cloud and comparing them with map data. Furthermore, for trees already recorded in the management ledger 30 (described later), the location data in the management ledger 30 can be referenced. If the management ledger 30 contains the identification number (ID) for each tree, the ID can also be identified during location determination.

[0042] Figure 6 is a flowchart of the skeleton recognition process in the skeleton recognition unit 24-2. First, information such as measurement data from the 3D scanner 10, map data, and captured images is taken in (S11). Next, the taken-in data is divided into hierarchical structures of vegetation (S12). As described above, the point cloud information is divided into lower layers that are easily hidden by shrubs and branches, middle layers where skeletons such as trunks and large branches are easily obtained, and upper layers where the laser has difficulty reaching.

[0043] Partial analysis is performed on the hierarchically divided data (S13). Point cloud data of trees is analyzed to recognize "branches and leaves," "twigs," "large branches," and "trunks." This partial analysis is performed based on pre-stored feature items for each part (scale (size), shape, color, reflectivity). As mentioned above, partial analysis using AI can also be performed as appropriate.

[0044] In S13, if part recognition is performed, an integration process is carried out to create a single tree (individual) (S14). A single tree is something that grows upward from the ground surface, and the recognized parts are integrated for each individual, and parts are recognized for a single tree.

[0045] In this way, we can recognize the skeletal structure of a single tree.

[0046] The point cloud data, after various data processing steps have been performed on the server computer 20, is supplied to the management ledger 30. The server computer 20 also possesses image data, video data, and location data obtained from cameras, and this information can also be supplied to the management ledger 30.

[0047] The management ledger 30 can be accessed by many computers connected via a communication line, and these computers can view the management ledger 30. Content updates are preferably performed by the server computer 20. Furthermore, other management computers may be used for inputting various data other than 3D point clouds.

[0048] The management ledger 30 assigns an identification number (ID) to each tree in the park, and stores attributes of each tree (size, age, location, etc.) associated with that ID. The 3D point cloud of each tree is also stored, associated with its ID. Furthermore, in this embodiment, the 3D point cloud of each tree obtained by the point cloud processing unit 24 is also stored in the terminal device 40 to enable high-speed display.

[0049] The management ledger 30 has a communication unit 32 and is responsible for exchanging information with the outside. And in this embodiment It has a 3D memory unit 34, where the 3D point cloud of the entire park is stored. In particular, As mentioned above, 3D point clouds can highlight the skeletal structure of each tree, consisting of its trunk and thick branches. This is how it works. This 3D point cloud is 3D Digital twin As such, on various terminals, It can also be made accessible.

[0050] A terminal device 40 is connected to the management ledger 30. The terminal device 40 is a portable device, such as a tablet PC. A smartphone is also acceptable, but one with a reasonably large screen is preferable. The terminal device 40 has a communication unit 42, a display unit 44, and an operation unit 46. By operating the operation unit 46, the user can access the management ledger 30, download various data, and display it on the display unit 44. The communication unit 42 is responsible for these communications.

[0051] In particular, in this embodiment, the 3D storage unit 34 of the management ledger 30 stores data that allows for the display of 3D point clouds of all trees in the park, associated with their locations. Therefore, workers can easily identify individual trees by looking at the display unit 44 of the terminal device 40 and observing the shape of each tree's trunk. For example, it is preferable that the terminal device 40 has a location detection function such as GPS, and that it can display multiple trees corresponding to its field of view.

[0052] In particular, the display shown in Figure 3 allows users to grasp the current arrangement, shape, and size of trees in a spatial manner by looking at the point cloud, and makes it easier to pinpoint locations more precisely by relying on the unique skeletal images of the trees. [Explanation of Symbols]

[0053] 10 3D scanner (3D imaging unit), 20 server computer, 22, 32, 42 communication unit, 24 point cloud processing unit, 24-1 tree recognition unit, 24-2 skeleton recognition unit, 24-3 display point cloud generation unit, 24-4 tree position recognition unit, 30 management ledger, 34 3D storage unit, 40 terminal device, 44 display unit, 46 operation unit.

Claims

1. A 3D imaging unit acquires a 3D point cloud of a target area where multiple trees exist by scanning the area from multiple directions, The acquired 3D point cloud is subjected to point cloud recognition processing and a tree recognition unit recognizes trees, Based on the thickness of each part of the recognized tree, a skeletal recognition unit recognizes the skeleton corresponding to the trunk and thick branches, and the branches and leaves separately. A 3D memory unit that stores a 3D point cloud of the skeleton associated with each tree in the target area, A 3D point cloud of the target area, with a display unit that emphasizes the recognized skeleton and displays it in 3D, including, Tree management system.

2. A tree management system according to claim 1, By comparing it with separately prepared map data, the location information of trees is recognized. Tree management system.

3. A tree management system according to claim 1, The skeleton and the branches and leaves can be displayed as separate layers and processed independently. Tree management system.

4. A tree management system according to any one of claims 1 to 3, The target area is part or all of the park. Tree management system.

Images