Measurement device, and measurement method

Abstract

To provide a measurement device that can simply acquire point group data on an absolute coordinate system.SOLUTION: A measurement device has: a first acquisition unit that acquires a position of a receiver in an absolute coordinate system on the basis of a satellite signal the receiver receives; a second acquisition unit that on the basis of image data on images a sensor photographs, acquires a position of the sensor and point group data in a sensor coordinate system; a calculation unit that calculates conversion data for converting the point group data to the absolute coordinate system from the sensor coordinate system on the basis of the position of the receiver, and the position of the sensor; and a conversion unit that converts the point group data in the sensor coordinate system to the absolute coordinate system, using the conversion data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a measurement device and a measurement method. [Background technology]

[0002] There are measuring devices that acquire point cloud data of objects such as buildings and terrain. The acquired point cloud data is represented in a local coordinate system and is not linked to map information in an absolute coordinate system such as latitude, longitude, and altitude.

[0003] A method for converting point cloud data in this local coordinate system into an absolute coordinate system (latitude, longitude, and altitude) is disclosed, for example, in the "Public Survey Manual Using Terrestrial Laser Scanners" by the Geospatial Information Authority of Japan.

[0004] For example, control points (markers) are placed around the object for which point cloud data is to be acquired. The absolute coordinates of the control points are acquired, for example, by GNSS (Global Navigation Satellite System) positioning. The object is photographed so that the control points are included, and point cloud data (local coordinates) is acquired. The coordinates (local coordinates) of the control points included in the acquired point cloud data are compared with the absolute coordinates of the control points acquired by GNSS positioning, etc. to obtain a 3D Helmert transformation matrix. The point cloud data is transformed from the local coordinate system to the absolute coordinate system using the transformation matrix obtained. This obtains the absolute coordinates of the object. In principle, the transformation matrix can be obtained with three control points, but four points are recommended to ensure redundancy. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2019 / 193654 [Non-patent literature]

[0006] [Non-Patent Document 1] R. Mur-Artal, et al., "ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras", IEEE Transactions on Robotics, Volume: 33, Issue: 5, Oct. 2017 Summary of the Invention [Problem to be solved by the invention]

[0007] To obtain point cloud data in an absolute coordinate system, as described above, control points are set around the object from which point cloud data is to be obtained, and the absolute coordinates of the control points are obtained by GNSS positioning or the like. Then, the object is photographed so as to include the control points. Therefore, the conventional method has a problem in that the work is complicated.

[0008] Non-limiting examples of the present disclosure contribute to providing a measurement device and a measurement method that can easily acquire point cloud data in an absolute coordinate system. [Means for solving the problem]

[0009] A measurement device according to one embodiment of the present disclosure includes a first acquisition unit that acquires the position of the receiver in an absolute coordinate system based on a satellite signal received by the receiver, a second acquisition unit that acquires the position of the sensor and point cloud data in a sensor coordinate system based on image data of an image captured by the sensor, a calculation unit that calculates conversion data for converting the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor, and a conversion unit that converts the point cloud data in the sensor coordinate system to the absolute coordinate system using the conversion data.

[0010] A measurement method according to one embodiment of the present disclosure acquires the position of the receiver in an absolute coordinate system based on a satellite signal received by the receiver, acquires the position of the sensor and point cloud data in a sensor coordinate system based on image data of an image captured by a sensor, calculates transformation data for transforming the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor, and transforms the point cloud data in the sensor coordinate system to the absolute coordinate system using the transformation data.

[0011] In addition, these comprehensive or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0012] According to an embodiment of the present disclosure, point cloud data in an absolute coordinate system can be easily acquired.

[0013] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings, in which such advantages and / or benefits are provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of which are provided to obtain one or more identical features. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a measurement device according to a first embodiment; [Diagram 2] A diagram showing an example of how the measuring device is used [Diagram 3] FIG. 1 is a diagram showing an example of a block configuration of a computing device. [Figure 4] A diagram explaining an example of generating conversion data [Diagram 5] 1 is a flowchart showing an example of the operation of a measuring device. [Figure 6] A diagram explaining the horizontal misalignment between a satellite signal receiver and an image sensor. [Figure 7] FIG. 13 is a diagram for explaining correction according to the second embodiment; [Figure 8] 1 is a flowchart showing an example of the operation of a measuring device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

[0016] It should be noted that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0017] [First embodiment] <Example of measurement device configuration> 1 is a diagram showing an example of the configuration of a measuring device 1 according to a first embodiment. As shown in FIG. 1, the measuring device 1 has a satellite signal receiver 11, an image sensor 12, and a computing device 13.

[0018] Satellite signal receiver 11 receives satellite signals including information such as the signal transmission time from one or more satellites such as GNSS satellites. Satellite signal receiver 11 transmits the received satellite signals to computing device 13 wirelessly or via wire.

[0019] The image sensor 12 is a sensor that outputs an image signal of an image of an object that has been photographed. The image sensor 12 may be, for example, a passive image sensor such as a stereo camera, or an active image sensor that uses radar or laser light. The image sensor 12 transmits the image signal of the image of the object to the computing device 13 wirelessly or via a wire.

[0020] The satellite signal receiver 11 and the image sensor 12 are fixed to, for example, a rod-shaped fixed body A1a. The image sensor 12 is fixed to the fixed body A1a below the satellite signal receiver 11 so that the satellite signal receiver 11 can easily receive satellite signals (so as not to block them).

[0021] The satellite signal receiver 11 and the image sensor 12 are fixed to the fixed body A1a in close proximity. The closer the satellite signal receiver 11 and the image sensor 12 are, the better. This is because the closer the satellite signal receiver 11 and the image sensor 12 are, the less likely errors due to the attitude of the fixed body A1a will occur. For example, if the distance is far, the coordinates will fluctuate significantly even if the fixed body A1a tilts slightly. Therefore, in this case, if this fluctuation is not accurately reflected, it may not be possible to accurately calculate the conversion data for adjusting the coordinate system. However, since it is difficult to accurately analyze changes in attitude that the user is not aware of, such as hand shake, it is easier to suppress the occurrence of such errors by placing the satellite signal receiver 11 and the image sensor 12 as close as possible. In the following, the satellite signal receiver 11, the image sensor 12, and the fixed body A1a may be referred to as a surveying rod.

[0022] The arithmetic device 13 calculates point cloud data represented in an absolute coordinate system based on the satellite signal received from the satellite signal receiver 11 and the image signal received from the image sensor 12. The absolute coordinate system is a coordinate system that represents coordinate points by points on the earth, and includes, for example, a geodetic system of latitude, longitude, and altitude, and an orthogonal coordinate system such as ECEF (earth-centered earth-fixed). The geodetic coordinate system may be called a geodetic coordinate system or a geographic coordinate system. The arithmetic device 13 may be, for example, an information processing device such as a tablet terminal, a notebook computer, a smartphone, or a server.

[0023] <Examples of using measuring devices> Fig. 2 is a diagram showing an example of use of the measuring device 1. Fig. 2 shows a computing device 13 of the measuring device 1, a survey rod A2a, an object A2b from which point cloud data is acquired, and a user U1 who uses the measuring device 1.

[0024] User U1 holds the survey rod A2a so that the survey rod A2a is perpendicular to the ground. In other words, user U1 holds the survey rod A2a so that the satellite signal receiver 11 and the image sensor 12 are aligned in a vertical line.

[0025] The user U1 holds the survey rod A2a as described above and photographs the object A2b. For example, the user U1 moves around the object A2b and photographs the surroundings of the object A2b. When measuring the point cloud data of the object A2b using the measuring device 1, it is not necessary to set a control point. In other words, the user U1 does not need to photograph the object A2b including the control point.

[0026] As shown in Fig. 2, the user U1 may move around the object A2b while holding the arithmetic device 13. The user U1 may place the arithmetic device 13 somewhere and move around the object A2b. The subject to be photographed may not be an object, but may be a terrain.

[0027] <Example of block configuration of calculation device> Fig. 3 is a diagram showing an example of a block configuration of the arithmetic device 13. Fig. 3 also shows the satellite signal receiver 11 and the image sensor 12. As shown in Fig. 3, the arithmetic device 13 has an RTK (Real Time Kinematic) arithmetic unit 21, a self-position estimation unit 22, a point cloud data generation unit 23, a conversion data generation unit 24, and a coordinate conversion unit 25. Each unit of the arithmetic device 13 may be realized, for example, by a control unit such as a CPU (Central Processing Unit) executing a program (application) stored in a storage unit (not shown).

[0028] The RTK calculation unit 21 calculates the position (absolute coordinates) of the satellite signal receiver 11 in the ECEF coordinate system, for example, by using the RTK method of existing technology. For example, the RTK calculation unit 21 calculates the absolute coordinates of the satellite signal receiver 11 in the ECEF coordinate system based on the satellite signals received from the satellite signal receiver 11 and correction information from a reference station (not shown in FIGS. 1 and 2) whose position is known in advance.

[0029] The calculation of the position of the satellite signal receiver 11 is not limited to the RTK method. The calculation of the position of the satellite signal receiver 11 may be performed by GNSS positioning using satellite signals, such as a network-type RTK positioning method or a DGNSS (Differential GNSS) method. The GNSS positioning method used to calculate the position of the satellite signal receiver 11 does not have to be a method that uses correction information from a reference station. However, it is known that, among currently known GNSS positioning methods, a method that uses correction information can calculate a position with higher accuracy.

[0030] The self-position estimation unit 22 calculates the position (local coordinates) of the image sensor 12 based on the image signal received from the image sensor 12. For example, the self-position estimation unit 22 may calculate the local coordinates of the image sensor 12 by visual odometry, an existing technology using images. For example, the self-position estimation unit 22 may calculate the amount of movement of the image sensor 12 based on the amount of movement of feature points in multiple images (frames) captured by the image sensor 12, and calculate the local coordinates of the image sensor 12.

[0031] The local coordinates are, for example, coordinates represented in a coordinate system (local coordinate system) defined in the measuring device 1. The self-position estimation unit 22 and the point cloud data generation unit 23 described below set, for example, an appropriate coordinate system, and represent the position and point cloud data of the image sensor 12 in the set coordinate system.

[0032] The point cloud data generator 23 generates (calculates) point cloud data indicated by local coordinates of an object photographed by the image sensor 12, based on the image signal received from the image sensor 12. When the image sensor 12 is an image sensor that uses laser light, the point cloud data generator 23 may generate the point cloud data using a technique such as LiDAR (Light Detection And Ranging). The point cloud data may be, for example, data indicating the distance from the image sensor 12 to the object. The point cloud data may be referred to as three-dimensional data.

[0033] The self-location estimation unit 22 and the point cloud data generation unit 23 may be realized by Visual-SLAM (Visual Simultaneous Localization and Mapping, see Non-Patent Document 1, for example). The self-location estimation unit 22 and the point cloud data generation unit 23 may acquire the position of the image sensor 12 and acquire point cloud data of an object photographed by the image sensor 12 by Visual-SLAM.

[0034] The conversion data generating unit 24 generates conversion data for converting the point cloud data in the local coordinate system generated by the point cloud data generating unit 23 into point cloud data in the absolute coordinate system.

[0035] For example, as described in Fig. 2, the user U1 holds the survey rod A2a and photographs the object A2b in order to obtain point cloud data of the object A2b. The conversion data generation unit 24 generates conversion data using the coordinates (position) of the satellite signal receiver 11 output from the RTK calculation unit 21 and the coordinates (position) of the image sensor 12 output from the self-position estimation unit 22 when the user U1 photographs the object A2b.

[0036] Fig. 4 is a diagram for explaining an example of generating conversion data. A trajectory A4a shown in Fig. 4 indicates the coordinates (trajectory) of the satellite signal receiver 11 output from the RTK calculation unit 21 when a user U1 holds a survey rod A2a and circles around an object A2b twice. The trajectory A4a is indicated by coordinates in the ECEF coordinate system, as described for the RTK calculation unit 21 in Fig. 3.

[0037] A trajectory A4b shown in Fig. 4 indicates the coordinates (trajectory) of the image sensor 12 output from the self-position estimation unit 22 when the user U1 holds the survey rod A2a and circles around the object A2b twice. The trajectory A4b is indicated by coordinates in the local coordinate system, as described in the self-position estimation unit 22 in Fig. 3.

[0038] In the above description, the user U1 has circled the object A2b twice, but the user U1 may circle the object A2b only once, or may circle the object A2b three or more times.

[0039] In the Visual-SLAM method, since errors during shooting tend to accumulate, a process called loop closing may be performed. Loop closing is a method of correcting errors in a trajectory by orbiting so as to pass through the same point at least at the start point and end point of shooting, and indicating that the coordinates corresponding to the start point and the coordinates corresponding to the end point are the same. The trajectory A4b in this embodiment is a trajectory after this loop closing process is performed, and is a closed trajectory. In general, the coordinates of the Visual-SLAM method, which does not perform loop closing process, tend to have errors compared to coordinates obtained by satellite positioning using a high-precision method. Therefore, it is desirable for the user U1 to orbit the object A2b at least once so that the loop closing process can be performed.

[0040] Furthermore, if the user U1 does not need point cloud data for the entire circumference of the object A2b, he or she does not need to go around the object A2b once. However, in this case, there is a possibility that the loop closing process cannot be performed, and therefore there is a risk that the error will be large or the same position will be calculated as multiple different coordinates.

[0041] The transformation data generating unit 24 generates transformation data for the trajectory A4b to overlap the trajectory A4a. For example, the transformation data generating unit 24 generates a three-dimensional Helmert transformation matrix. Note that the Helmert transformation is also called a similarity transformation, and transforms coordinates between coordinate systems based on translation, rotation, and enlargement / reduction.

[0042] The trajectories A4a and A4b are trajectories obtained by passing through the same route, although the coordinate systems are different. Therefore, the trajectories A4a and A4b may not have completely the same shape depending on the scale and angle of the coordinate system, but are highly likely to have similar shapes. Therefore, in this embodiment, a three-dimensional Helmert transformation matrix is ​​generated, so that a transformation matrix for transforming the coordinate systems of the trajectories A4a and A4b can be easily generated. Furthermore, since the coordinates of the trajectory A4a are obtained in an absolute coordinate system by satellite positioning, in this embodiment, if a three-dimensional Helmert transformation matrix is ​​generated, a calibration process for correcting the deviation between the absolute coordinate system and the local coordinate system is not required. Therefore, according to this embodiment, the point cloud data in the local coordinate system obtained by Visual-SLAM can be easily transformed into point cloud data in the absolute coordinate system. In addition, since the three-dimensional Helmert transformation matrix obtained in this embodiment is only for overlapping the trajectory A4a and the trajectory A4b, the coordinate systems may not match at locations away from the trajectory A4a and the trajectory A4b due to sensor errors, etc. Therefore, in the method of this embodiment, the coordinate systems may not match completely. However, if the purpose is to photograph the object A2b, it is sufficient that the coordinate systems around the trajectory A4a and the trajectory A4b match. That is, according to this embodiment, it is possible to realize the conversion between the local coordinate system and the absolute coordinate system in the range focused on photographing the object A2b by a simple operation.

[0043] As described in Fig. 1, the satellite signal receiver 11 and the image sensor 12 are fixed to the fixed body A1a in close proximity. Therefore, when the user U1 holds the survey rod A2a and takes an image of the periphery of the object A2b, the satellite signal receiver 11 and the image sensor 12 move in approximately the same position. Therefore, when the process (translation, rotation, and enlargement / reduction) of superimposing the trajectory A4b shown in Fig. 4 on the trajectory A4a is executed, the trajectory A4a and the trajectory A4b have approximately the same shape (the degree of agreement is high), and the deviation between the trajectory A4a and the trajectory A4b is suppressed. This suppresses the coordinate transformation error included in the three-dimensional Helmert transformation matrix.

[0044] Returning to the explanation of Fig. 3, the coordinate conversion unit 25 converts the point cloud data in the local coordinate system output from the point cloud data generation unit 23 into absolute coordinates in the ECEF coordinate system using the conversion data generated by the conversion data generation unit 24. For example, the coordinate conversion unit 25 multiplies the point cloud data in the local coordinate system output from the point cloud data generation unit 23 by the three-dimensional Helmert transformation matrix generated by the conversion data generation unit 24 to convert it into absolute coordinates in the ECEF coordinate system. The coordinate conversion unit 25 converts the point cloud data converted into the absolute coordinates of the ECEF coordinate system into absolute coordinates of the geodetic system of latitude, longitude, and altitude.

[0045] <Operation flow> 5 is a flowchart showing an example of the operation of the measuring device 1. A user U1 holds a survey rod A2a, moves around an object A2b from which point cloud data is to be acquired, and photographs the object A2b.

[0046] Measurement device 1 calculates the coordinates (trajectory) of satellite signal receiver 11, which changes as user U1 moves, based on the satellite signals received by satellite signal receiver 11. The trajectory of satellite signal receiver 11 is expressed in the ECEF coordinate system. Measurement device 1 stores the calculated trajectory of satellite signal receiver 11 in a memory unit (S1).

[0047] The measuring device 1 calculates the coordinates (trajectory) of the image sensor 12 that changes with the movement of the user U1 based on the image data acquired by the image sensor 12. The trajectory of the image sensor 12 is represented in the local coordinate system of the measuring device 1. The measuring device 1 stores the calculated trajectory of the image sensor 12 in a storage unit (S2).

[0048] The measuring device 1 generates point cloud data based on image data acquired by the image sensor 12. The point cloud data is represented in a local coordinate system of the measuring device 1. The measuring device 1 stores the generated point cloud data in a storage unit (S3).

[0049] The measuring device 1 calculates a three-dimensional Helmert transformation matrix for overlapping the trajectory of the image sensor 12 stored in the storage unit in S2 with the trajectory of the satellite signal receiver 11 stored in the storage unit in S1 (S4).

[0050] The measuring device 1 multiplies the point cloud data stored in the storage unit in S3 by the three-dimensional Helmert transformation matrix calculated in S4 to convert the point cloud data into a coordinate system (S5). That is, the measuring device 1 converts the point cloud data stored in the storage unit in S3 from the local coordinate system into the absolute coordinate system of the ECFE coordinate system.

[0051] The measuring device 1 converts the point cloud data converted into the ECFE coordinate system in S5 into a geodetic coordinate system (S6).

[0052] The order of the processes S1 to S3 is not limited to that shown in Fig. 5. The order of the processes S1 to S3 may be reversed.

[0053] <Summary of the First Embodiment> As described above, the RTK calculation unit 21 acquires the position of the satellite signal receiver 11 in the absolute coordinate system based on the satellite signal received by the satellite signal receiver 11. The self-position estimation unit 22 acquires the position of the image sensor 12 in the local coordinate system based on the image data of the image captured by the image sensor 12. The point cloud data generation unit 23 acquires point cloud data based on the image data of the image captured by the image sensor 12. The conversion data generation unit 24 calculates conversion data for converting the point cloud data from the local coordinate system to the absolute coordinate system based on the position of the satellite signal receiver 11 and the position of the image sensor 12. The coordinate conversion unit 25 converts the point cloud data in the local coordinate system to the absolute coordinate system using the conversion data. This allows the user to easily acquire point cloud data in the absolute coordinate system. For example, the user only needs to hold the survey rod A2a and photograph the object from which point cloud data is to be acquired, and does not need to set a control point around the object and photograph the object so as to include the control point, so that the point cloud data in the absolute coordinate system can be easily acquired.

[0054] <Modification> Variation 1 Although the satellite signal receiver 11 and the image sensor 12 are fixed to a fixed body A1a in close proximity to each other, there is a gap (distance) between the satellite signal receiver 11 and the image sensor 12 in the vertical direction (height direction) (see FIG. 1). Therefore, the measurement device 1 may, for example, subtract the distance between the satellite signal receiver 11 and the image sensor 12 from the altitude (height) of the point cloud data of the geodetic system calculated in S6. This allows the measurement device 1 to obtain point cloud data of the object including the appropriate altitude.

[0055] Variation 2 The measuring device 1 does not need to convert the point cloud data in the ECFE coordinate system into the geodetic coordinate system. The measuring device 1 may output the point cloud data in the ECFE coordinate system. The measuring device 1 may output two point cloud data, the point cloud data in the ECFE coordinate system and the point cloud data in the geodetic coordinate system.

[0056] [Second embodiment] If the survey rod A2a is not held perpendicular to the ground, the positions of the satellite signal receiver 11 and the image sensor 12 will shift in the horizontal direction.

[0057] Figure 6 is a diagram explaining the horizontal misalignment between the satellite signal receiver 11 and the image sensor 12. Figures 6(A) and 6(B) show a survey rod A2a equipped with a fixed body A1a, a satellite signal receiver 11, and an image sensor 12. Figure 6(A) shows the survey rod A2a as seen from the side. Figure 6(B) shows the survey rod A2a as seen from the front. The dotted arrow A6a in Figure 6(A) indicates the shooting direction of the image sensor 12.

[0058] For example, as shown by dotted arrow A6a in Fig. 6(A), if the surveying rod A2a is tilted so that the shooting direction of image sensor 12 faces downward, the positions of the satellite signal receiver 11 and image sensor 12 will be misaligned in the horizontal direction, as shown by double-headed arrow A6b. Also, as shown in Fig. 6(B), if the surveying rod A2a is tilted to the left or right when viewed from the front, the positions of the satellite signal receiver 11 and image sensor 12 will be misaligned in the horizontal direction, as shown by double-headed arrow A6c.

[0059] If the positions of satellite signal receiver 11 and image sensor 12 are misaligned in the horizontal direction, when a process (translation, rotation, and enlargement / reduction) is performed to superimpose the coordinates (trajectory) of satellite signal receiver 11 and the coordinates (trajectory) of image sensor 12, the shapes of the two trajectories may differ significantly. If the shapes of the two trajectories differ significantly, the degree of overlap between the shapes of the two trajectories decreases, and the error in the coordinate transformation included in the three-dimensional Helmert transformation matrix may become large. If the error in the coordinate transformation included in the three-dimensional Helmert transformation matrix is ​​large, an error will occur in the point cloud data after the coordinate transformation.

[0060] Therefore, in the second embodiment, the position of satellite signal receiver 11 is corrected so that it is aligned vertically with image sensor 12, thereby suppressing errors contained in the point cloud data after coordinate transformation.

[0061] The following describes the differences from the first embodiment. In the second embodiment, a correction unit that corrects the position of the satellite signal receiver 11 is inserted between the RTK calculation unit 21 and the conversion data generation unit 24 in the block configuration shown in Fig. 3. The conversion data generation unit 24 generates conversion data based on the coordinates (trajectory) of the satellite signal receiver 11 corrected by the correction unit and the coordinates (trajectory) of the image sensor 12 output from the self-position estimation unit 22.

[0062] FIG. 7 is a diagram for explaining the correction according to the second embodiment. When measuring the position of the image sensor 12 using feature points included in the image of the image data, for example, the measuring device 1 can calculate the attitude of the image sensor 12. The attitude of the image sensor 12 is represented by the deflection angles θ and φ, for example, as shown in FIG. 7(A). The deflection angle θ indicates the inclination of the image sensor 12 with respect to the vertical direction when the survey rod A2a is viewed from the side (see FIG. 6(A)). The deflection angle φ indicates the inclination of the image sensor 12 in the left-right direction when the survey rod A2a is viewed from the front (see FIG. 6(B)).

[0063] Because the satellite signal receiver 11 and image sensor 12 are fixed to a fixed body A1a, the attitude (θ, φ) of the satellite signal receiver 11 can be determined by knowing the attitude of image sensor 12. The distance between the satellite signal receiver 11 and image sensor 12 is also known. Therefore, once the attitude of image sensor 12 is calculated, the horizontal deviation with respect to image sensor 12 can be corrected.

[0064] 7B, the black circle A7a indicates the position of the image sensor 12. The black circle A7b indicates the position of the satellite signal receiver 11 before correction. The double-headed arrow A7c indicates the distance (known) between the satellite signal receiver 11 and the image sensor 12.

[0065] Based on the distance (known) between satellite signal receiver 11 and image sensor 12 indicated by double arrow A7c in Fig. 7(B) and the attitude (θ, φ) of satellite signal receiver 11 indicated by double arrow A7d, measurement device 1 corrects the position of satellite signal receiver 11 so that it overlaps with image sensor 12 when viewed in the vertical direction, as indicated by black circle A7e. This corrects, for example, the horizontal deviation between the positions of satellite signal receiver 11 and image sensor 12 indicated by double arrows A6b and A6c in Fig. 6. Note that the attitude (θ, φ) of satellite signal receiver 11 indicated by double arrow A7d can be obtained by calculating the attitude of image sensor 12, as described above.

[0066] 8 is a flowchart showing an example of the operation of the measuring device 1. A user U1 holds a survey rod A2a, moves around an object A2b from which point cloud data is to be acquired, and photographs the object A2b.

[0067] The measurement device 1 calculates the coordinates (trajectory) of the satellite signal receiver 11, which changes as the user U1 moves, based on the satellite signals received by the satellite signal receiver 11. The trajectory of the satellite signal receiver 11 is expressed in the ECEF coordinate system. The measurement device 1 stores the calculated trajectory of the satellite signal receiver 11 in a memory unit (S11).

[0068] The measuring device 1 calculates the coordinates (trajectory) of the image sensor 12, which change with the movement of the user U1, and the attitude of the image sensor 12, based on the image data acquired by the image sensor 12. The trajectory and attitude of the image sensor 12 are represented in the local coordinate system of the measuring device 1. The measuring device 1 stores the calculated trajectory of the image sensor 12 and the attitude of the image sensor 12 in a storage unit (S12).

[0069] The measuring device 1 generates point cloud data based on image data acquired by the image sensor 12. The point cloud data is represented in a local coordinate system of the measuring device 1. The measuring device 1 stores the generated point cloud data in a storage unit (S13).

[0070] The measurement device 1 calculates a three-dimensional Helmert transformation matrix for overlapping the trajectory of the image sensor 12 stored in the storage unit in S12 with the trajectory of the satellite signal receiver 11 (S14).

[0071] The measurement device 1 multiplies the trajectory and orientation of the image sensor 12 stored in the storage unit in S12 by the three-dimensional Helmert transformation matrix calculated in S14, and performs coordinate transformation of the trajectory and orientation of the image sensor 12 (S15). The trajectory and orientation of the image sensor 12 after the coordinate transformation in S15 are expressed in the ECEF coordinate system.

[0072] The measurement device 1 selects the trajectory (coordinate point) of the image sensor 12 after coordinate conversion that is closest to each point (coordinate point) of the trajectory (positioning result) of the satellite signal receiver 11 stored in the memory unit in S11 (S16). For example, the measurement device 1 selects the trajectory after coordinate conversion of the trajectory of the image sensor 12 acquired at the same timing as one epoch of the positioning result of the satellite signal receiver 11 stored in S11.

[0073] The measurement device 1 corrects the trajectory of the satellite signal receiver 11 using the attitude of the image sensor 12 after the coordinate transformation that corresponds to the coordinate point selected in S16 (S17). For example, as described in FIG. 7, the measurement device 1 corrects the positioning result of the satellite signal receiver 11 based on the attitude of the image sensor 12 selected in S16 (i.e., the attitude of the satellite signal receiver 11) and the distance between the satellite signal receiver 11 and the image sensor 12.

[0074] The measurement device 1 calculates a three-dimensional Helmert transformation matrix for superimposing the trajectory of the image sensor 12 stored in the storage unit in S12 on the trajectory of the satellite signal receiver 11 corrected in S17 (S18).

[0075] The measuring device 1 multiplies the point cloud data stored in the storage unit in S13 by the three-dimensional Helmert transformation matrix calculated in S18, and performs coordinate system conversion of the point cloud data (S19). That is, the measuring device 1 converts the point cloud data stored in the storage unit in S13 from the local coordinate system to the absolute coordinate system of the ECFE coordinate system.

[0076] The measuring device 1 converts the point cloud data converted into the ECFE coordinate system in S19 into a geodetic coordinate system (S20).

[0077] The order of the processes of S11 to S13 is not limited to that of Fig. 8. The order of the processes of S11 to S13 may be interchanged. Furthermore, the order of the process of S14 is not limited to that of Fig. 8. The process of S14 may be executed after the process of S12.

[0078] <Summary of the second embodiment> As described above, the measuring device 1 corrects the horizontal deviation of the satellite signal receiver 11 with respect to the image sensor 12 based on the tilt of the satellite signal receiver 11 with respect to the vertical direction calculated based on the image data and the distance between the satellite signal receiver 11 and the image sensor 12. For example, the measuring device 1 corrects the position of the satellite signal receiver 11 so that the position of the satellite signal receiver 11 overlaps with the image sensor 12 when viewed from the vertical direction. This allows the measuring device 1 to reduce errors included in the point cloud data after coordinate transformation.

[0079] <Modification> The measuring device 1 may correct the height (vertical distance) between the satellite signal receiver 11 and the image sensor 12 when the survey rod A2a is tilted. For example, the measuring device 1 may calculate the vertical distance between the satellite signal receiver 11 and the image sensor 12 based on the attitude of the satellite signal receiver 11 calculated based on the image data and the distance between the satellite signal receiver 11 and the image sensor 12, and correct the height of the point cloud data in the absolute coordinate system. For example, the measuring device 1 may subtract the calculated vertical distance from the point cloud data that has been subjected to coordinate system transformation.

[0080] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the claims. It is understood that such modified or altered examples also belong to the technical scope of the present disclosure. In addition, the components in the embodiments may be arbitrarily combined within the scope of the present disclosure.

[0081] In the above-described embodiments, examples of generating conversion data for superimposing a local coordinate system on an absolute coordinate system have been described, but conversion data for superimposing an absolute coordinate system on a local coordinate system may be generated. The conversion data to be generated according to which coordinate system may be changed according to the user's request. In addition, the Visual-SLAM coordinate system has been described as being a local coordinate system, but it may be an absolute coordinate system. In this case, the conversion data is data that converts the deviation between the absolute coordinate systems. As long as the point cloud data is acquired based on an image captured by a sensor, the type of coordinate system (hereinafter referred to as the "sensor coordinate system") does not matter. Note that, as a method of acquiring point cloud data in an absolute coordinate system using Visual-SLAM, for example, a marker or the like whose absolute coordinates are known in advance is included in the shooting range, and the coordinates of the point cloud data are calculated based on the absolute coordinates indicated by the marker or the like.

[0082] The RTK calculation unit 21 may be referred to as a first acquisition unit. The self-position estimation unit 22 and the point cloud data generation unit 23 may be referred to as a second acquisition unit. The conversion data generation unit 24 may be referred to as a calculation unit. The coordinate conversion unit 25 may be referred to as a conversion unit. Calculation and generation may be read as calculation. Measurement may be read as measurement or positioning. "Same" may include "approximately the same".

[0083] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuit", "... assembly", "... device", "... unit", or "... module".

[0084] The present disclosure can be realized by software, hardware, or software in cooperation with hardware. Each functional block used in the description of the above embodiment may be realized partially or entirely as an LSI, which is an integrated circuit, and each process described in the above embodiment may be controlled partially or entirely by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip that includes some or all of the functional blocks. The LSI may have input and output of data. Depending on the degree of integration, the LSI may be called an IC, a system LSI, a super LSI, or an ultra LSI.

[0085] The method of integration is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, after LSI manufacturing, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0086] Furthermore, if a new integrated circuit technology that can replace LSI appears due to the progress of semiconductor technology or a derivative technology, it is possible to integrate the functional blocks using that technology. The application of biotechnology is also a possibility. [Industrial Applicability]

[0087] The present disclosure is useful for a measuring device that acquires point cloud data in an absolute coordinate system. [Explanation of symbols]

[0088] 1. Measuring equipment 11 Satellite signal receiver 12 Image Sensor 13 Arithmetic unit A1a Fixed body A2a surveying rod 21 RTK calculation section 22 Self-position estimation part 23 Point cloud data generator 24 Conversion data generation unit 25 Coordinate conversion section

Claims

1. a first acquisition unit that acquires a position of the receiver in an absolute coordinate system based on a satellite signal received by the receiver; A second acquisition unit that acquires a position of the sensor in a sensor coordinate system and point cloud data based on image data of an image captured by the sensor; a calculation unit that calculates transformation data for transforming the point cloud data from the sensor coordinate system to the absolute coordinate system based on a position of the receiver and a position of the sensor; a conversion unit that converts the point cloud data in the sensor coordinate system into the absolute coordinate system using the conversion data; A measuring device having

2. The conversion unit generates, as the conversion data, data for performing a similarity conversion to superimpose the trajectory of positions acquired by the second acquisition unit on the trajectory of positions acquired by the first acquisition unit.

2. The measuring device of claim 1.

3. The receiver and the sensor are fixed to a fixed body.

3. The measuring device according to claim 2.

4. A correction unit that corrects a height of the point cloud data in the absolute coordinate system based on a distance between the receiver and the sensor.

4. The measuring device according to claim 3.

5. 5. The measuring device according to claim 4, wherein the correction unit calculates a vertical distance between the receiver and the sensor based on an attitude of the receiver calculated based on the image data and a distance between the receiver and the sensor, and corrects a height of the point cloud data in the absolute coordinate system based on the vertical distance.

6. and a correction unit that corrects a horizontal deviation of the receiver relative to the sensor based on the attitude of the receiver calculated based on the image data and the distance between the receiver and the sensor.

4. The measuring device according to claim 3.

7. The correction unit corrects the position of the receiver so that the position of the receiver overlaps with the sensor when viewed in a vertical direction.

7. The measuring device according to claim 6.

8. The receiver and the sensor are fixed to the fixed body in close proximity to each other.

4. The measuring device according to claim 3.

9. obtaining a position of the receiver in an absolute coordinate system based on satellite signals received by the receiver; Acquire a position of the sensor in a sensor coordinate system and point cloud data based on image data of an image captured by the sensor; Calculating transformation data for transforming the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor; using the transformation data, transforming the point cloud data in the sensor coordinate system into the absolute coordinate system; Measurement method.

Images