Plant shape measurement device and measurement method

Abstract

To provide a plant configuration measurement device and measurement method capable of streamlining creation work of a 3D model with fewer omissions for a large-scale plant.SOLUTION: A plant configuration measurement device comprises a plant configuration measurement sensor 111 that acquires data for generating a point cloud representing a detailed configuration of a plant; a voxel map generation unit 113 that generates a voxel map for a plant 200; and a 3D-CAD model generation unit 146 that generates point cloud data based on information from the plant configuration measurement sensor 111, and generates a 3D-CAD model of the plant 200. The plant configuration measurement device classifies the voxel map into voxels that include objects, voxels that do not include objects, and voxels that are not measured by the sensor based on the information from the plant configuration measurement sensor 111, presents the classification results to workers, and measures a plant configuration.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a plant shape measurement device and a measurement method for determining the position and orientation of a measuring device when measuring the shape of an object to be measured using, for example, a camera or a LIDAR (Light Detection and Ranging) measuring device. [Background technology]

[0002] When carrying out maintenance or demolition work on large-scale structures such as power plants or chemical plants, it is important to understand the condition of the target structure in advance in order to carry out the work efficiently and without incurring unnecessary costs. To achieve this, a method is sometimes used in which a 3D model is generated using 3D measurements using cameras or LIDAR, and this is then used to compare the on-site situation with the design drawings.

[0003] These plant facilities are three-dimensional structures made up of numerous parts with complex shapes tightly assembled. Therefore, when viewing the facility from a certain point around it, there are many cases where the rear part of the facility is hidden by the front part and cannot be observed (occlusion). Operators determine the placement positions of multiple different measuring instruments so that they can measure the rear part of such facilities and measure all relevant parts without missing anything. Determining the location of the measuring device is not an easy task, and requires a great deal of skill and experience to perform it quickly and efficiently. If the worker does not have sufficient skill or experience, the efficiency of the measurement work and 3D model generation work may deteriorate.

[0004] For example, if there are not enough measurement instrument locations, sufficient measurement data cannot be obtained in one measurement operation, which may result in the measurement operation having to be performed again, resulting in a decrease in the efficiency of the measurement operation.Also, if there are too many measurement instrument locations, the amount of measurement data obtained from the measurement will be enormous, which may result in a longer time required for the 3D model generation process and a decrease in the efficiency of the 3D model generation operation. As an invention that prevents such a decrease in the efficiency of the work of generating 3D models, for example, Patent Document 1 describes an optimal layout planning method for measuring equipment, in which an SfM (Structure from Motion) model of equipment, etc. is generated before measuring the equipment, etc. with a measuring instrument, and the placement position of the measuring instrument is almost automatically determined based on this SfM model. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6535779 Summary of the Invention [Problem to be solved by the invention]

[0006] However, as the scale and complexity of the target plant increases, the SfM model may contain omissions or errors, such as objects appearing where there really are none. Repeated remeasurements to correct these errors ultimately result in a decrease in work efficiency.

[0007] Therefore, the present invention provides a plant shape measuring device and a measuring method that can improve the efficiency of the work of generating a 3D model with few omissions for a large-scale plant. [Means for solving the problem]

[0008] In order to solve the above problems, a plant shape measurement device according to the present invention includes: a plant shape measurement sensor that acquires data for generating a point cloud that represents a detailed shape of a plant; a voxel map generation sensor that acquires data for generating a voxel map that manages omissions in observations by the plant shape measurement sensor; and based on information from the voxel map generation sensor, a voxel map generation unit that generates a voxel map for the plant; and a 3D-CAD model generation unit that generates point cloud data and generates a 3D-CAD model of the plant based on information from the plant shape measurement sensor, wherein, based on the information from the plant shape measurement sensor, voxels in the voxel map that include an object and voxels that do not include an object are generated; Plant shape measurementThe voxels that are not measured by the sensor are classified, and the classification results are presented to the worker while the plant shape is measured. and a display unit that derives an observation trajectory, which is a combination of the observation position and observation direction of the plant shape measurement sensor, and displays the observation trajectory on a display screen. It is characterized by:

[0009] Further, a plant shape measurement method according to the present invention includes acquiring data for generating a point cloud representing a detailed shape of a plant by a plant shape measurement sensor, and a voxel map generation unit: a voxel map generation sensor that acquires data for generating a voxel map that manages omissions in observations by the plant shape measurement sensor; and based on information from the voxel map generation sensor, A voxel map for the plant is generated, and a 3D-CAD model generation unit generates point cloud data based on information from the plant shape measurement sensor, and generates a 3D-CAD model of the plant. Based on the information from the plant shape measurement sensor, voxels in the voxel map that include objects and voxels that do not include objects are identified. Plant shape measurement The system classifies voxels into those that are not measured by the sensor, and presents the classification results to the worker while measuring the plant shape. An observation trajectory, which is a combination of the observation position and observation direction of the plant shape measurement sensor, is derived, and the observation trajectory is displayed on the display screen of a display unit. It is characterized by: [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a plant shape measuring device and a measuring method that can improve the efficiency of the work of generating a 3D model with few omissions for a large-scale plant. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a functional block diagram showing the overall configuration of a plant shape measuring apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is an external perspective view of a data collection device that constitutes the plant shape measuring device shown in FIG. [Figure 3] 2 is a diagram showing an example of a measurement range of a data collection device that configures the plant shape measuring apparatus shown in FIG. 1. FIG. [Figure 4] 2 is a functional block diagram showing the overall configuration of a three-dimensional position recognition device that constitutes the data collection device shown in FIG. 1. FIG. [Figure 5] 2 is a diagram showing an example of data acquisition by a data collection device constituting the plant shape measuring device shown in FIG. 1. FIG. [Figure 6] FIG. 1 is a diagram illustrating an example of a measurement target space according to an embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of voxel division of a measurement target space according to an embodiment of the present invention. [Figure 8] 2 is a diagram showing an example of measurement by a voxel map generating sensor that constitutes the data collection device shown in FIG. 1. FIG. [Figure 9] 2 is a diagram showing an example of a measurement result obtained by a voxel map generating sensor that constitutes the data collection device shown in FIG. 1. FIG. [Figure 10] FIG. 10 is a diagram showing an example of attribute transition of a voxel map according to an embodiment of the present invention. [Figure 11] FIG. 2 is a diagram showing an example of an observation permission area according to an embodiment of the present invention. [Figure 12] 10A and 10B are diagrams illustrating an example of an observation position and direction indication according to an embodiment of the present invention. [Figure 13] 4 is a flowchart showing a processing procedure executed by an observation position / direction determining device and an observation position / direction presenting device according to an embodiment of the present invention. [Figure 14] 1A and 1B are diagrams showing examples of displaying a current position, a target position, and the like using AR according to an embodiment of the present invention. [Figure 15] 1 is an external perspective view showing an example of a data collection device according to an embodiment of the present invention; [Figure 16] 1 is a diagram illustrating a data collection device and AR glasses according to an embodiment of the present invention. [Figure 17] FIG. 10 is a diagram showing an example of an image displayed by AR glasses as a display according to an embodiment of the present invention. [Figure 18] FIG. 10 is a diagram showing an example of an image displayed by AR glasses as a display according to an embodiment of the present invention. [Figure 19] FIG. 1 is a diagram illustrating a relationship between AR glasses as a display and a plant according to an embodiment of the present invention. [Figure 20]2 is a flowchart showing an example of a flow for generating a 3D-CAD model of a plant by the CAD model generating device shown in FIG. 1. [Figure 21] 2 is a flowchart showing an example of a flow for estimating a three-dimensional position and orientation of the data collecting device shown in FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Example]

[0013] 1 is a functional block diagram showing the overall configuration of a plant shape measurement apparatus according to an embodiment of the present invention. As shown in FIG. 1, the plant shape measurement apparatus 100 according to this embodiment includes a data collection device 110, an observation position / direction determination device 120, an observation position / direction determination device 130, a data acquisition device 140, a data acquisition device 150, a data acquisition device 160, a data acquisition device 170, a data acquisition device 180 presentation The system is composed of a device 130 and a CAD model generating device 140. The plant shape measuring device 100 that can create a 3D model with few omissions and errors in a short period of time will be described in detail below.

[0014] As shown in Fig. 1, the data collection device 110 is a device that collects data on plant shape information in order to generate a 3D model of the plant, and is composed of a plant shape measurement sensor 111, a 3D position recognition device 112 that estimates the 3D position of the data collection device 110, and a voxel map generation sensor 113. Fig. 2 is a perspective view of the exterior of the data collection device that constitutes the plant shape measurement device shown in Fig. 1. For example, as shown in Fig. 2, the data collection device 110 is constructed by combining a GNSS (Global Navigation Satellite System) receiver 301, a high-resolution camera 302, a LIDAR 303, and an IMU (Inertial Measurement Unit) 304.

[0015] The plant shape measurement sensor 111 is a sensor, such as a high-resolution camera or LIDAR, that acquires data for generating a point cloud that represents the detailed shape of the plant. The point cloud acquired by the plant shape measurement sensor 111 is converted into a point cloud defined in an external coordinate system by a CAD model generation device 140 based on the position and orientation of the data collection device 110 estimated by a 3D position recognition device 112, and then accumulated in a sensor data accumulation unit 141, which will be described in detail later. Note that the plant shape measurement sensor 111 may also be configured to measure the plant shape using a sensor such as an image sensor and / or LIDAR.

[0016] On the other hand, the voxel map generation sensor 113 is a sensor that acquires data to generate a voxel map that manages omissions in observations made by the plant shape measurement sensor 111. The data acquired by the voxel map generation sensor 113 is converted into information defined in an external coordinate system by the observation position and direction determination device 120 based on the position and orientation of the data collection device 110 estimated by the 3D position recognition device 112, and then a voxel map with attributes is generated.

[0017] As shown in FIG. 3, the voxel map generation sensor 113 is preferably a sensor that requires a short time to generate a voxel map, such as a LIDAR, and is also preferably a sensor with a wider detection range and a higher sampling period than the plant shape measurement sensor 111.

[0018] FIG. 4 is a functional block diagram showing the overall configuration of a three-dimensional position recognition device that constitutes the data collection device shown in FIG. 4, for example, the three-dimensional position recognition device 112 includes a GNSS receiver 114, an IMU 115, an external environment recognition sensor 116, plant design data 117, a position and attitude estimation and map update unit 118, and a surrounding environment map 119. The three-dimensional position recognition device 112 estimates the three-dimensional position and attitude of the data collection device 110 with sufficient accuracy and frequency to generate the above-mentioned CAD model and voxel map. Here, the position and attitude estimation and map update unit 118 is realized by, for example, a processor such as a CPU (not shown), a ROM for storing various programs, a RAM for temporarily storing data in the calculation process, and a storage device such as an external storage device, and the processor such as the CPU reads and executes the various programs stored in the ROM and stores the execution results, which are the calculation results, in the RAM or the external storage device.

[0019] The position and attitude estimation and map update unit 118 constituting the three-dimensional position recognition device 112 acquires data from the GNSS receiver 114 and the IMU 115, and estimates the three-dimensional position and attitude in the external coordinate system of the data collection device 110 at a high sampling period using a Kalman filter or the like, and also estimates the three-dimensional position and attitude with high accuracy, albeit at a low sampling period, by scan matching between the surrounding environment observation data acquired by the external environment recognition sensor 116 and the surrounding environment map 119 defined in the external coordinate system while referring to the plant design data 117, and combines this with the estimated values ​​of the three-dimensional position and attitude.

[0020] FIG. 5 is a diagram showing an example of data acquisition by a data collection device constituting the plant shape measurement apparatus shown in FIG. 1. As shown in FIG. 5, the observation position / direction determination device 120 is a device that calculates the observation position 203 and observation direction 204 of the data collection device 110, which are necessary to create a three-dimensional model with few omissions and errors. The observation position / direction determination device 120 is composed of an attributed voxel map update unit 121, an attributed voxel map 122, and an unmeasured portion observable position / direction calculation unit 123. Here, the attributed voxel map update unit 121 and the unmeasured portion observable position / direction calculation unit 123 are realized by, for example, a processor such as a CPU (not shown), a ROM for storing various programs, a RAM for temporarily storing data in the calculation process, and a storage device such as an external storage device. The processor such as the CPU reads and executes the various programs stored in the ROM and stores the execution results in the RAM or the external storage device.

[0021] In order to generate a voxel map that manages omissions in observations by the plant shape measurement sensor 111, the attributed voxel map update unit 121 converts the data acquired by the voxel map generation sensor 113 into information defined in an external coordinate system based on the position and orientation of the data collection device 110 estimated by the 3D position recognition device 112, and generates an attributed voxel map. Fig. 6 is a diagram showing an example of a measurement target space according to an embodiment of the present invention. The attributed voxel map 122 is a map database in which a measurement target space 201 including a plant for which a 3D-CAD model as shown in Fig. 6 is to be created is divided into predetermined unit voxels 202 as shown in Fig. 7, and the divided voxels are assigned five attributes ((1) object present and measurement present, (2) object present and measurement absent, (3) no object present and measurement present, (4) no object present and measurement absent, (5) unknown).

[0022] Fig. 8 is a diagram showing an example of measurement by the voxel map generation sensor constituting the data collection device shown in Fig. 1. For example, when a LIDAR is used as the voxel map generation sensor 113, as shown in Fig. 8, a laser 311 is emitted at regular intervals to acquire the distance to the reflection point, and the surface of the plant 200 in the measurement target space 201 is scanned, thereby updating the attributes of each voxel in the voxel map 122. Here, when the device is started up, the attributes of all voxels are initialized to (5) unknown, and then, for all voxels through which the laser 311 passes, the attribute-attached voxel map update unit 121 sets the attribute to object present if the voxel contains a reflection point 311p of the laser 311 irradiated by the LIDAR, as shown in Fig. 9, or sets the attribute to object absent if no object is present and the laser 311 passes through the voxel. Depending on this and whether the voxel is within the measurement range of the plant shape measurement sensor 111, one of the following measurement results is obtained: (1) object present and within measurement range, (2) object present and outside measurement range, (3) no object present and within measurement range, or (4) no object present and outside measurement range, and the respective attributes are updated according to the attribute transition table shown in Fig. 10 depending on the current attribute of each voxel. In the attribute transition table shown in Figure 10, the current attributes on the horizontal axis represent the attributes before measurement, and if something has already been measured, it goes from unknown on the right end to "object present / measured" on the left end. Also, all attributes become "measured" as you move from the right end to the left end. For example, if the shape of the plant 200 is such that an object is in an area surrounded by a rectangular-shaped obstruction as viewed from the information, it is expected that it will be difficult to measure the distance to the object regardless of the elevation angle. In this case, the object can be marked as "unknown" as shown in FIG. 10, or measurement can be performed using an aerial work platform, a drone, or the like. In this case, the attributes can be identified by referring to the measurement results and design data.

[0023] The unmeasured portion observable position / direction calculation unit 123 calculates the position and observation direction of the data collecting device 110 that can observe the unmeasured voxels in the voxel map.

[0024] The observation position / direction presentation device 130 is a device that presents to the data collection worker the observation position / direction of the data collection device 110 necessary to create a three-dimensional model with few omissions and errors, and is composed of an observation position / direction instruction processing unit 131, a display 132, and a speaker 133. FIG. 11 is a diagram illustrating an example of an observation permission area according to an embodiment of the present invention. The observation position / direction instruction processing unit 131 derives an observation trajectory 206, which is a combination of observation positions and observation directions that minimizes the measurement time, as shown in FIG. 12, from the positions and observation directions of the data collecting device 110 that are located within the specified observation permission area 205 shown in FIG. 11 and that can observe unmeasured voxels in the voxel map ((2) object present / not measured, (3) object absent / not measured, (5) unknown) calculated by the unmeasured part observable position / direction calculation unit 123. The observation trajectory 206 is presented to the operator via the display 132 and speaker 133. Here, when two towers constituting the plant 200 overlap as shown in FIG. 8 above, it is difficult for the data collecting device 110 to measure the distance to the tower at the back of the overlapping portion with the front tower. However, by measuring the distance using the observation trajectory 206 shown in FIG. 12 (described later), it is possible to measure the distance to the tower at the back as well. Here, the observation position / direction instruction processing unit 131 is realized by, for example, a processor such as a CPU (not shown), a ROM for storing various programs, a RAM for temporarily storing data in the calculation process, and a storage device such as an external storage device, and the processor such as a CPU reads and executes the various programs stored in the ROM, and stores the calculation results that are the execution results in the RAM or the external storage device.

[0025] The CAD model generation device 140 is a device that generates a 3D-CAD model from an acquired point cloud that represents the plant shape, and is configured with a sensor data accumulation unit 141, a sensor database 142, a high-density point cloud generation unit 143, a point cloud database 144, a design information database 145, a 3D-CAD model generation unit 146, and a 3D-CAD model 147. Here, the high-density point cloud generation unit 143 and the 3D-CAD model generation unit 146 are realized by, for example, a processor such as a CPU (not shown), a ROM that stores various programs, a RAM that temporarily stores data in the calculation process, and a storage device such as an external storage device, and the processor such as the CPU reads and executes the various programs stored in the ROM and stores the execution results, which are the calculation results, in the RAM or the external storage device.

[0026] The sensor data accumulation unit 141 associates the data acquired from the plant shape measuring sensor 113 with the self-position and orientation acquired in step S101 (FIG. 13) described later, and stores the data in the sensor database 142. The high-density point cloud generator 143 generates a point cloud representing the detailed shape of the plant defined in an external coordinate system based on the sensor data acquired by the plant shape measurement sensor 111 and the position and orientation of the data collection device 110 estimated by the 3D position recognition device 112, and stores the point cloud in a point cloud database 144.

[0027] The 3D-CAD model generation unit 146 estimates the arrangement of each partial CAD model by matching the point cloud representing the shape of the plant stored in the point cloud database 144 with the partial CAD models of the plant's piping, etc. recorded in the design information database 145, and generates a 3D-CAD model of the entire plant.

[0028] The above is a description of the configuration and components of the plant shape measuring device 100.

[0029] Next, a description will be given of the operations of the observation position / direction determining device 120 and the observation position / direction presenting device 130. Fig. 13 is a flowchart showing the processing steps executed by the observation position / direction determining device 120 and the observation position / direction presenting device 130. Steps S101 to S104 explain a method for calculating and presenting the observation position and direction of the data collecting device 110 required to create a 3D model with few omissions and errors by the plant shape measuring device 100. This process is executed for each measurement cycle of the voxel map generating sensor 113.

[0030] In step S 101 , the position and orientation of the data collecting device 110 are acquired from the three-dimensional position recognition device 112 . In step S102, the data acquired from the voxel map generation sensor 113 is converted into information defined in an external coordinate system based on the position and orientation of the data collection device 110 estimated in step S101, and an attributed voxel map 122 is generated with five attributes ((1) object present, measurement performed, (2) object present, no measurement, (3) no object, measurement performed, (4) no object, no measurement, (5) unknown).

[0031] In step S103, the unmeasured part observable position / direction calculation unit 123 calculates the position and observation direction of the data collection device 110 that can observe (2) object present / not measured, (3) object absent / not measured, and (5) unknown in the voxel map based on the data of the attributed voxel map 122.

[0032] In step S104, based on the observable position and direction calculated in step S103, the observation position and direction of the data collection device 110 required to create a 3D model with few omissions and errors is calculated by the observation position and direction presentation device 130, and the calculated observation position and direction are indicated by the display 132 and speaker 133. Here, by making the viewing angle of the plant shape measuring sensor 111 variable and calculating an appropriate viewing angle and presenting it to the operator, it is possible to further shorten the time required to acquire data.

[0033] Furthermore, as shown in Fig. 14, by using AR (augmented reality), the observation position, observation direction, and viewing angle can be displayed on the surrounding image, thereby making it possible to improve the efficiency of data acquisition by workers. In this case, it is desirable to use AR glasses as the display 132. In Fig. 14, a current position display point 401, a target position display point 402, a current attitude display coordinate system 403, and a target attitude display coordinate system 404 are displayed within the environmental image of the plant.

[0034] Furthermore, as shown in FIG. 15, by using a gimbal 405 to automate the adjustment of the orientation of the data collection device 110 and the adjustment of the viewing angle of the plant shape measurement sensor 111, the worker who acquires the data only needs to move, thereby reducing the burden on the worker. The above is an explanation of an example of the flow of operations of the observation position / direction determination device 120 and the observation position / direction presentation device 130.

[0035] Here, information provided to the worker when the AR glasses 500 are worn as the display 132 constituting the observation position / direction determining device 130 will be described. 16 is a diagram showing a data collection device and AR glasses according to one embodiment of the present invention. As shown in FIG. 16, an operator wears AR glasses 500 and operates a camera dolly 501 with a handle to adjust the viewing angle or elevation angle of the data collection device.

[0036] 17 is a diagram showing an example of an image displayed by AR glasses as a display according to one embodiment of the present invention. As shown in Fig. 17, the plant environment is displayed as is in an image 502 of the AR glasses, along with the current sensor and camera dolly 503a. Furthermore, a virtual model 503b representing the target position and orientation for the next data acquisition is superimposed on the image 502 of the AR glasses.

[0037] Fig. 18 is a diagram showing an example of an image displayed by AR glasses as a display according to one embodiment of the present invention. In the display shown in Fig. 17 above, the camera dolly 501 with the handle is moved or operated so that the current sensor and camera dolly 503a overlap with the virtual model 503b representing the target position and orientation for the next data acquisition. A display is provided so that the worker can easily recognize that they have been able to move to the target position and orientation. In Fig. 18, "OK" is displayed as an example.

[0038] 19 is a diagram showing the relationship between a plant and AR glasses as a display according to an embodiment of the present invention. When acquiring data within a plant in accordance with instructions from AR glasses 500, a worker simply moves or operates camera dolly 501 with a handle in accordance with instructions from AR glasses 500.

[0039] Next, a description will be given of the operation of the CAD model generation device 140. Fig. 20 is a flowchart showing an example of a flow for generating a 3D-CAD model of a plant by the CAD model generation device shown in Fig. 1. This process is executed for each measurement cycle of the plant shape measuring sensor 111.

[0040] In steps S201 to S204, an example of a flow for generating a 3D-CAD model of a plant from sensor data representing the shape of the plant acquired by the plant shape measurement sensor 111 will be described. In step S 201 , the position and orientation of the data collecting device 110 are acquired from the three-dimensional position recognition device 112 and stored in the sensor data storage unit 141 that constitutes the CAD model generating device 140 .

[0041] In step S202, the data acquired from the plant shape measuring sensor 113 and the self-position and orientation acquired in step S101 are associated with each other and stored in the sensor database 142. Regarding the association with the self-position and posture, for example, when a camera is used as the plant shape measurement sensor 111, the captured image may be recorded as a geotag.

[0042] In step S203, when the 3D-CAD conversion of the plant is started, the process proceeds to step S204 and subsequent steps for the 3D-CAD conversion process.

[0043] In step S204, a high-density point cloud representing the shape of the target plant is generated from the sensor data stored in the sensor database 142 and stored in the point cloud database 144. In step S205, a 3D CAD model of the entire plant is generated and recorded based on the point cloud representing the shape of the plant recorded in the point cloud database 144 and the design information of the plant acquired from the design information database 145. The above is a description of an example of the flow of the operation of the CAD model generating device 140.

[0044] Next, a description will be given of the operation of the three-dimensional position recognition device 112 that constitutes the data collection device 110. Fig. 20 is a flowchart showing an example of a three-dimensional position and orientation estimation flow of the data collection device shown in Fig. 1. This process is executed for each measurement cycle of the three-dimensional position recognition device 112.

[0045] In steps S301 to S305, an example of a flow for estimating the three-dimensional position and orientation of the three-dimensional position recognition device 112 constituting the data collection device 110 will be described. In step S301, the position and orientation of the three-dimensional position recognition device 112 are updated based on the inertial momentum acquired by the IMU 115. In step S302, if the GNSS receiver 114 acquires positioning information, in step S304 the position and attitude updated in step S301 are corrected using the positioning information acquired in step S302.

[0046] In step S303, when data is acquired from the external environment recognition sensor 116, the position and orientation of the three-dimensional position recognition device 112 and the surrounding environment map 119 are updated by matching the estimated values ​​of the position and orientation of the three-dimensional position recognition device 112 with the data acquired in step S303 and the surrounding environment map 119. This completes the description of an example of the operation flow of the three-dimensional position recognition device 112.

[0047] As described above, according to this embodiment, it is possible to provide a plant shape measuring device and a measuring method that can improve the efficiency of the work of generating a 3D model with few omissions for a large-scale plant. Furthermore, an observation trajectory 206, which is a combination of observation positions and observation directions that will shorten the measurement time as much as possible, is derived and presented to the worker via the display 132 and speaker 133, thereby improving the efficiency of the work.

[0048] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with the configuration of another embodiment. [Explanation of symbols]

[0049] 100...Plant shape measurement device 110...Data collection device 111...Plant shape measurement sensor 112…3D position recognition device 113...Voxel map generation sensor 114...GNSS receiver 115...IMU 116...External recognition sensor 117...Plant design data 118...Position and orientation estimation / map update unit 119...Surrounding Area Map 120...Observation position and direction determination device 121...Attributed voxel map update unit 122...Voxel map with attributes 123...Unmeasured part observable position and direction calculation unit 130...Observation position and direction determination device 131...Observation position / direction indication processing unit 132...Display 133...Speaker 140...CAD model generation device 141...Sensor data storage unit 142...Sensor data database 143…High-density point cloud generator 144...Point cloud database 145...Design information database 146...3D-CAD model generation unit 147...3D-CAD model 200...Plant 201...Measurement target space 202...Voxel 203...Observation location 204...Observation direction 205...Observation permission area 206...Observation orbit 301...GNSS receiver 302...High-resolution camera 303…LIDAR 304...IMU 311...Laser 311p...laser reflection point 312...Objectless voxel 313...Voxel with object 314...Unknown voxel 401...Current location display point 402...Target position display point 403...Current attitude display coordinate system 404…Target attitude display coordinate system 405...Gimbal 500…AR glasses 501...Camera dolly with handle 502…Image of AR glasses 503a...Current sensor and camera dolly 503b...Virtual model representing the target position and orientation for the next data acquisition

Claims

1. A measurement device that measures the shape of a plant, a plant shape measurement sensor that acquires data for generating a point cloud that represents a detailed shape of a plant; a voxel map generation sensor that acquires data for generating a voxel map that manages omissions in observations made by the plant shape measurement sensor; a voxel map generation unit that generates a voxel map for the plant based on information from the voxel map generation sensor; and a 3D-CAD model generation unit that generates point cloud data and generates a 3D-CAD model of the plant based on information from the plant shape measurement sensor, classifying the voxel map into voxels that include an object, voxels that do not include an object, and voxels that have not been measured by the plant shape measurement sensor based on information from the plant shape measurement sensor, and presenting the classification result to an operator while measuring the plant shape; The plant shape measurement device further comprises a display unit that derives an observation trajectory, which is a combination of the observation position and observation direction of the plant shape measurement sensor, and displays the observation trajectory on a display screen.

2. 2. The plant shape measuring apparatus according to claim 1, A plant shape measurement device characterized by calculating the position of the plant shape measurement sensor from which data can be acquired for voxels that are blocked by an obstruction or voxels that are not measured by the plant shape measurement sensor, and outputting the calculation results to a user.

3. 3. The plant shape measuring apparatus according to claim 2, the plant shape measurement sensor is a sensor that acquires high-resolution data, The plant shape measuring device is characterized in that the voxel map generating sensor has a wider detection range and a higher sampling period than the plant shape measuring sensor.

4. 2. The plant shape measuring apparatus according to claim 1, A plant shape measurement device characterized in that it calculates a position and a field of view at which data can be acquired for a voxel not measured by the plant shape measurement sensor, and outputs the calculation result to a user.

5. 5. The plant shape measuring apparatus according to claim 4, A plant shape measuring device characterized in that the current position and observation position, current attitude and observation direction, and field of view of the plant shape measuring sensor are displayed on a display screen on a voxel map and provided to an operator.

6. 5. The plant shape measuring apparatus according to claim 4, A plant shape measurement device characterized in that the current position and observation position, current attitude and observation direction, and field of view of the plant shape measurement sensor are displayed in augmented reality using AR glasses on a voxel map and provided to workers.

7. A plant shape measurement method for measuring the shape of a plant, comprising: a plant shape measurement sensor acquires data for generating a point cloud representing a detailed shape of a plant; a voxel map generation unit acquires data for generating a voxel map for managing omissions in observations by the plant shape measurement sensor, and generates a voxel map for the plant based on information from the voxel map generation sensor; a 3D-CAD model generation unit generates point cloud data based on information from the plant shape measurement sensor and generates a 3D-CAD model of the plant; classifying the voxel map into voxels that include an object, voxels that do not include an object, and voxels that have not been measured by the plant shape measurement sensor based on information from the plant shape measurement sensor, and presenting the classification result to an operator while measuring the plant shape; a plant shape measuring method comprising: deriving an observation trajectory, which is a combination of the observation position and observation direction of the plant shape measuring sensor; and displaying the observation trajectory on a display screen of a display unit;

8. In the plant shape measurement method according to claim 7, A plant shape measurement method comprising: calculating a position and a field of view at which data can be acquired for a voxel not measured by the plant shape measurement sensor; and outputting the calculation results to a user.

9. 9. The plant shape measurement method according to claim 8, A plant shape measurement method characterized in that the current position and observation position, current attitude and observation direction, and field of view of the plant shape measurement sensor are displayed on a display screen on a voxel map and provided to an operator.

10. 9. The plant shape measurement method according to claim 8, A plant shape measurement method characterized in that the current position and observation position, current attitude and observation direction, and field of view of the plant shape measurement sensor are displayed in augmented reality using AR glasses on a voxel map and provided to workers.

Images