Measurement device and measurement method

Abstract

This measurement device comprises: a first acquisition unit that acquires the position of a receiver in an absolute coordinate system on the basis of a satellite signal received by the receiver; a second acquisition unit that acquires the position of the sensor and point group data in a sensor coordinate system on the basis of image data of an image captured by the sensor; a calculation unit that, on the basis of the position of the receiver and the position of the sensor, calculates conversion data for converting the point group data from the sensor coordinate system to the absolute coordinate system; and a conversion unit that, using the conversion data, converts the point group data in the sensor coordinate system to the absolute coordinate system.

Description

Measuring device and measuring method

[0001] The present disclosure relates to a measurement device and a measurement method.

[0002] There are measurement devices that acquire point cloud data of objects such as buildings and terrain. The acquired point cloud data is represented in a local coordinate system and is not linked to map information in an absolute coordinate system, such as latitude, longitude, and altitude.

[0003] A method for converting point cloud data in this local coordinate system into an absolute coordinate system (latitude, longitude, and altitude) is disclosed, for example, in the "Public Survey Manual Using Terrestrial Laser Scanners" by the Geospatial Information Authority of Japan.

[0004] For example, control points (markers) are placed around the object from which point cloud data is to be acquired. The absolute coordinates of the control points are acquired, for example, by GNSS (Global Navigation Satellite System) positioning. The object is photographed so that the control points are included, and point cloud data (local coordinates) is acquired. The coordinates (local coordinates) of the control points included in the acquired point cloud data are compared with the absolute coordinates of the control points acquired by GNSS positioning, etc., to determine a three-dimensional Helmert transformation matrix. The point cloud data is transformed from the local coordinate system to the absolute coordinate system using the obtained transformation matrix. This obtains the absolute coordinates of the object. In principle, the transformation matrix can be determined with three control points, but four points is recommended to ensure redundancy.

[0005] International Publication No. 2019 / 193654

[0006] R. Mur-Artal, et al., "ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras", IEEE Transactions on Robotics, Volume: 33, Issue: 5, Oct. 2017

[0007] To acquire point cloud data in an absolute coordinate system, as described above, control points are set around the object from which point cloud data is to be acquired, and the absolute coordinates of the control points are acquired by GNSS positioning or the like. Then, the object is photographed so that the control points are included. Therefore, conventional methods have the problem of being cumbersome.

[0008] Non-limiting examples of the present disclosure contribute to providing a measurement device and a measurement method that can easily acquire point cloud data in an absolute coordinate system.

[0009] A measurement device according to one embodiment of the present disclosure includes a first acquisition unit that acquires the position of the receiver in an absolute coordinate system based on satellite signals received by the receiver; a second acquisition unit that acquires the position of the sensor and point cloud data in a sensor coordinate system based on image data of an image captured by the sensor; a calculation unit that calculates conversion data for converting the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor; and a conversion unit that uses the conversion data to convert the point cloud data in the sensor coordinate system to the absolute coordinate system.

[0010] A measurement method according to one embodiment of the present disclosure acquires the position of a receiver in an absolute coordinate system based on satellite signals received by the receiver, acquires the position of the sensor in a sensor coordinate system and point cloud data based on image data of an image captured by a sensor, calculates transformation data for converting the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor, and converts the point cloud data in the sensor coordinate system to the absolute coordinate system using the transformation data.

[0011] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0012] According to an embodiment of the present disclosure, point cloud data in an absolute coordinate system can be easily acquired.

[0013] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features.

[0014] FIG. 1 is a diagram showing an example of the configuration of a measurement device according to a first embodiment; FIG. 2 is a diagram showing an example of the use of the measurement device; FIG. 3 is a diagram showing an example of the block configuration of a calculation device; FIG. 4 is a diagram explaining an example of generation of conversion data;

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or redundant explanation of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0016] The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0017] [First embodiment] <Configuration example of measuring device> Fig. 1 is a diagram showing a configuration example of a measuring device 1 according to a first embodiment. As shown in Fig. 1, the measuring device 1 has a satellite signal receiver 11, an image sensor 12, and a calculation device 13.

[0018] The satellite signal receiver 11 receives satellite signals, including information such as the signal transmission time, from one or more satellites, such as GNSS satellites. The satellite signal receiver 11 transmits the received satellite signals to the computing device 13 wirelessly or via a wired connection.

[0019] The image sensor 12 is a sensor that outputs an image signal of an image of an object. The image sensor 12 may be, for example, a passive image sensor such as a stereo camera, or an active image sensor that uses radar or laser light. The image sensor 12 transmits the image signal of the image of the object to the computing device 13 wirelessly or via a wired connection.

[0020] The satellite signal receiver 11 and the image sensor 12 are fixed to, for example, a rod-shaped fixed body A1a. The image sensor 12 is fixed to the fixed body A1a below the satellite signal receiver 11 so that the satellite signal receiver 11 can easily receive satellite signals (so as not to block them).

[0021] The satellite signal receiver 11 and the image sensor 12 are fixed to the fixed body A1a in close proximity. The closer the satellite signal receiver 11 and the image sensor 12 are, the less likely errors due to the attitude of the fixed body A1a are to occur. For example, if the distance between the satellite signal receiver 11 and the image sensor 12 is large, even a slight tilt of the fixed body A1a can cause large fluctuations in coordinates. Therefore, in this case, if this fluctuation is not accurately reflected, it may be impossible to accurately calculate the transformation data to align the coordinate system. However, because it is difficult to accurately analyze changes in attitude that the user is not aware of, such as camera shake, it is easier to suppress the occurrence of such errors by placing the satellite signal receiver 11 and the image sensor 12 as close as possible. Hereinafter, the satellite signal receiver 11, the image sensor 12, and the fixed body A1a may be referred to as a surveying rod.

[0022] The computing device 13 calculates point cloud data represented in an absolute coordinate system based on the satellite signals received from the satellite signal receiver 11 and the image signals received from the image sensor 12. The absolute coordinate system is a coordinate system that represents coordinate points as points on the Earth, and includes, for example, a geodetic system based on latitude, longitude, and altitude, and an orthogonal coordinate system such as ECEF (Earth-Centered Earth-Fixed). The geodetic coordinate system may also be referred to as a geodetic coordinate system or a geographic coordinate system. The computing device 13 may be, for example, an information processing device such as a tablet terminal, a laptop computer, a smartphone, or a server.

[0023] <Example of Use of Measuring Device> Fig. 2 is a diagram showing an example of use of the measuring device 1. Fig. 2 shows the computing device 13 of the measuring device 1, a survey rod A2a, an object A2b from which point cloud data is acquired, and a user U1 who uses the measuring device 1.

[0024] The user U1 holds the surveying rod A2a so that the surveying rod A2a is perpendicular to the ground. In other words, the user U1 holds the surveying rod A2a so that the satellite signal receiver 11 and the image sensor 12 are aligned in the vertical direction.

[0025] As described above, user U1 holds the surveying rod A2a and photographs object A2b. For example, user U1 walks around object A2b and photographs the area around object A2b. When measuring point cloud data of object A2b using the measuring device 1, it is not necessary to set a control point. In other words, user U1 does not need to photograph object A2b, including the control point.

[0026] 2, the user U1 may hold the computing device 13 and move around the object A2b. Alternatively, the user U1 may place the computing device 13 somewhere and move around the object A2b. Furthermore, the subject of the image capture may not be an object, but may be a terrain.

[0027] <Example of Block Configuration of Computing Device> Fig. 3 is a diagram showing an example of the block configuration of the computing device 13. Fig. 3 also shows the satellite signal receiver 11 and the image sensor 12. As shown in Fig. 3, the computing device 13 has an RTK (Real Time Kinematic) computing unit 21, a self-position estimation unit 22, a point cloud data generation unit 23, a conversion data generation unit 24, and a coordinate conversion unit 25. Each unit of the computing device 13 may be realized, for example, by a control unit such as a CPU (Central Processing Unit) executing a program (application) stored in a storage unit (not shown).

[0028] The RTK calculation unit 21 uses, for example, the RTK method, an existing technology, to calculate the position (absolute coordinates) of the satellite signal receiver 11 in the ECEF coordinate system. For example, the RTK calculation unit 21 calculates the absolute coordinates of the satellite signal receiver 11 in the ECEF coordinate system based on the satellite signals received from the satellite signal receiver 11 and correction information from a reference station (not shown in FIGS. 1 and 2 ) whose position is known in advance.

[0029] The calculation of the position of the satellite signal receiver 11 is not limited to the RTK method. The calculation of the position of the satellite signal receiver 11 may also be performed by GNSS positioning using satellite signals, such as a network-type RTK positioning method or a DGNSS (Differential GNSS) method. The GNSS positioning method used to calculate the position of the satellite signal receiver 11 does not have to be a method that uses correction information from a reference station. However, among currently known GNSS positioning methods, it is known that methods that use correction information can calculate a position with higher accuracy.

[0030] The self-position estimation unit 22 calculates the position (local coordinates) of the image sensor 12 based on the image signal received from the image sensor 12. For example, the self-position estimation unit 22 may calculate the local coordinates of the image sensor 12 by visual odometry, an existing technology that uses images. For example, the self-position estimation unit 22 may calculate the amount of movement of the image sensor 12 based on the amount of movement of feature points in multiple images (frames) captured by the image sensor 12, and calculate the local coordinates of the image sensor 12.

[0031] The local coordinates are, for example, coordinates represented in a coordinate system (local coordinate system) defined in the measurement device 1. The self-position estimation unit 22 and the point cloud data generation unit 23 described below set, for example, an appropriate coordinate system and represent the position of the image sensor 12 and the point cloud data in the set coordinate system.

[0032] The point cloud data generator 23 generates (calculates) point cloud data represented by local coordinates of an object photographed by the image sensor 12, based on the image signal received from the image sensor 12. If the image sensor 12 is an image sensor that uses laser light, the point cloud data generator 23 may generate the point cloud data using a technique such as LiDAR (Light Detection and Ranging). The point cloud data may be, for example, data indicating the distance from the image sensor 12 to the object. The point cloud data may also be referred to as three-dimensional data.

[0033] The self-position estimation unit 22 and the point cloud data generation unit 23 may be realized by Visual-SLAM (Visual Simultaneous Localization and Mapping, see Non-Patent Document 1, for example). The self-position estimation unit 22 and the point cloud data generation unit 23 may acquire the position of the image sensor 12 and acquire point cloud data of an object photographed by the image sensor 12 by Visual-SLAM.

[0034] The conversion data generating unit 24 generates conversion data for converting the point cloud data in the local coordinate system generated by the point cloud data generating unit 23 into point cloud data in the absolute coordinate system.

[0035] 2, user U1 holds a surveying rod A2a and photographs object A2b to obtain point cloud data of object A2b. The conversion data generation unit 24 generates conversion data using the coordinates (position) of the satellite signal receiver 11 output from the RTK calculation unit 21 and the coordinates (position) of the image sensor 12 output from the self-position estimation unit 22 when user U1 photographs object A2b.

[0036] Fig. 4 is a diagram illustrating an example of generating conversion data. A trajectory A4a shown in Fig. 4 indicates the coordinates (trajectory) of the satellite signal receiver 11 output from the RTK calculation unit 21 when a user U1 holds a survey rod A2a and circles around an object A2b twice. The trajectory A4a is shown in coordinates in the ECEF coordinate system, as described for the RTK calculation unit 21 in Fig. 3.

[0037] 4 indicates the coordinates (trajectory) of the image sensor 12 output from the self-position estimation unit 22 when the user U1 holds the surveying rod A2a and circles around the object A2b twice. The trajectory A4b is indicated by coordinates in the local coordinate system, as described for the self-position estimation unit 22 in FIG.

[0038] In the above description, the user U1 has circled the periphery of the object A2b twice, but the user U1 may circle only once, or three or more times.

[0039] In the Visual-SLAM method, errors tend to accumulate during image capture, so a process called loop closing may be performed. Loop closing is a technique for correcting trajectory errors by orbiting the object so that it passes through the same point at least at the start and end points of image capture and indicating that the coordinates corresponding to the start and end points are the same. The trajectory A4b in this embodiment is the trajectory after this loop closing process has been performed, and is a closed trajectory. Generally, coordinates obtained using the Visual-SLAM method, which does not perform loop closing, are more likely to have errors than coordinates obtained using satellite positioning using a high-precision method. Therefore, it is desirable for the user U1 to orbit the object A2b at least once so that the loop closing process can be performed.

[0040] Furthermore, if the user U1 does not need point cloud data for the entire circumference of the object A2b, he or she does not need to go around the object A2b once. However, in this case, there is a possibility that the loop closing process cannot be performed, which may result in a large error or the same position being calculated as multiple different coordinates.

[0041] The transformation data generation unit 24 generates transformation data for superimposing the trajectory A4b on the trajectory A4a. For example, the transformation data generation unit 24 generates a three-dimensional Helmert transformation matrix. The Helmert transformation, also known as a similarity transformation, converts coordinates between coordinate systems based on translation, rotation, and scaling.

[0042] Note that the trajectories A4a and A4b are trajectories obtained by traveling the same route, although they are in different coordinate systems. Therefore, although the trajectories A4a and A4b may not have exactly the same shape depending on the scale and angle of the coordinate systems, they are likely to have similar shapes. Therefore, in this embodiment, by generating a three-dimensional Helmert transformation matrix, it is possible to easily generate a transformation matrix that converts the coordinate systems of the trajectories A4a and A4b. Furthermore, since the coordinates of the trajectory A4a are obtained in an absolute coordinate system using satellite positioning, in this embodiment, once the three-dimensional Helmert transformation matrix is ​​generated, calibration processing to correct the misalignment between the absolute coordinate system and the local coordinate system is unnecessary. Therefore, according to this embodiment, point cloud data in a local coordinate system obtained by Visual-SLAM can be easily converted into point cloud data in an absolute coordinate system. Note that the 3D Helmert transformation matrix calculated in this embodiment is solely for the purpose of superimposing the trajectories A4a and A4b. Therefore, the coordinate systems may not match at locations distant from the trajectories A4a and A4b due to sensor errors, etc. Therefore, the method of this embodiment may not result in a perfect match of the coordinate systems. However, if the purpose is to photograph the object A2b, it is sufficient that the coordinate systems around the trajectories A4a and A4b match. In other words, according to this embodiment, a simple operation can be used to convert between the local coordinate system and the absolute coordinate system within the range focused on photographing the object A2b.

[0043] As described with reference to FIG. 1 , the satellite signal receiver 11 and the image sensor 12 are fixed to a fixed body A1a in close proximity. Therefore, when user U1 holds a surveying rod A2a and photographs the area around object A2b, the satellite signal receiver 11 and the image sensor 12 move in approximately the same position. Therefore, when the process of superimposing trajectory A4b shown in FIG. 4 on trajectory A4a (translation, rotation, and scaling) is performed, trajectories A4a and A4b have approximately the same shape (high degree of correspondence), and deviation between trajectories A4a and A4b is reduced. This reduces coordinate transformation errors contained in the three-dimensional Helmert transformation matrix.

[0044] Returning to the description of Fig. 3, the coordinate conversion unit 25 converts the point cloud data in the local coordinate system output from the point cloud data generation unit 23 into absolute coordinates in the ECEF coordinate system using the conversion data generated by the conversion data generation unit 24. For example, the coordinate conversion unit 25 multiplies the point cloud data in the local coordinate system output from the point cloud data generation unit 23 by the three-dimensional Helmert transformation matrix generated by the conversion data generation unit 24 to convert it into absolute coordinates in the ECEF coordinate system. The coordinate conversion unit 25 converts the point cloud data converted into the absolute coordinates in the ECEF coordinate system into absolute coordinates in a geodetic system of latitude, longitude, and altitude.

[0045] 5 is a flowchart showing an example of the operation of the measuring device 1. A user U1 holds a survey rod A2a, moves around an object A2b from which point cloud data is to be acquired, and photographs the object A2b.

[0046] The measurement device 1 calculates the coordinates (trajectory) of the satellite signal receiver 11, which changes as the user U1 moves, based on the satellite signals received by the satellite signal receiver 11. The trajectory of the satellite signal receiver 11 is expressed in the ECEF coordinate system. The measurement device 1 stores the calculated trajectory of the satellite signal receiver 11 in a memory unit (S1).

[0047] The measuring device 1 calculates the coordinates (trajectory) of the image sensor 12, which change as the user U1 moves, based on the image data acquired by the image sensor 12. The trajectory of the image sensor 12 is expressed in the local coordinate system of the measuring device 1. The measuring device 1 stores the calculated trajectory of the image sensor 12 in a storage unit (S2).

[0048] The measuring device 1 generates point cloud data based on the image data acquired by the image sensor 12. The point cloud data is expressed in the local coordinate system of the measuring device 1. The measuring device 1 stores the generated point cloud data in a storage unit (S3).

[0049] The measuring device 1 calculates a three-dimensional Helmert transformation matrix for overlapping the trajectory of the image sensor 12 stored in the storage unit in S2 with the trajectory of the satellite signal receiver 11 stored in the storage unit in S1 (S4).

[0050] The measuring device 1 multiplies the point cloud data stored in the storage unit in S3 by the three-dimensional Helmert transformation matrix calculated in S4 to convert the point cloud data into a coordinate system (S5). That is, the measuring device 1 converts the point cloud data stored in the storage unit in S3 from the local coordinate system into the absolute coordinate system of the ECFE coordinate system.

[0051] The measuring device 1 converts the point cloud data converted into the ECFE coordinate system in S5 into a geodetic coordinate system (S6).

[0052] The processing order of S1 to S3 is not limited to that shown in Fig. 5. The processing order of S1 to S3 may be reversed.

[0053] Summary of the First Embodiment As described above, the RTK calculation unit 21 acquires the position of the satellite signal receiver 11 in the absolute coordinate system based on the satellite signals received by the satellite signal receiver 11. The self-position estimation unit 22 acquires the position of the image sensor 12 in the local coordinate system based on image data of images captured by the image sensor 12. The point cloud data generation unit 23 acquires point cloud data based on the image data of images captured by the image sensor 12. The conversion data generation unit 24 calculates conversion data for converting the point cloud data from the local coordinate system to the absolute coordinate system based on the positions of the satellite signal receiver 11 and the image sensor 12. The coordinate conversion unit 25 uses the conversion data to convert the point cloud data in the local coordinate system into the absolute coordinate system. This allows a user to easily acquire point cloud data in the absolute coordinate system. For example, the user only needs to hold the surveying rod A2a and photograph an object from which point cloud data is to be acquired. This eliminates the need to set control points around the object and photograph the object so that the control points are included. This allows for easy acquisition of point cloud data in the absolute coordinate system.

[0054] <Modifications> Modification 1 The satellite signal receiver 11 and image sensor 12 are fixed to a fixed body A1a in close proximity, but there is a gap (distance) in the vertical direction (height direction) between the satellite signal receiver 11 and the image sensor 12 (see FIG. 1 ). Therefore, the measurement device 1 may, for example, subtract the distance between the satellite signal receiver 11 and the image sensor 12 from the altitude (height) of the point cloud data of the geodetic system calculated in S6. This allows the measurement device 1 to obtain point cloud data of the object that includes the appropriate altitude.

[0055] Modification 2 The measurement device 1 does not need to convert the point cloud data in the ECFE coordinate system into the geodetic coordinate system. The measurement device 1 may output point cloud data in the ECFE coordinate system. The measurement device 1 may output two types of point cloud data: point cloud data in the ECFE coordinate system and point cloud data in the geodetic coordinate system.

[0056] Second Embodiment When the surveying rod A2a is not held perpendicular to the ground, the positions of the satellite signal receiver 11 and the image sensor 12 are misaligned in the horizontal direction.

[0057] Figure 6 is a diagram illustrating the horizontal misalignment between the satellite signal receiver 11 and the image sensor 12. Figures 6A and 6B show a surveying rod A2a that includes a fixed body A1a, a satellite signal receiver 11, and an image sensor 12. Figure 6A shows the surveying rod A2a as seen from the side. Figure 6B shows the surveying rod A2a as seen from the front. The dotted arrow A6a in Figure 6A indicates the shooting direction of the image sensor 12.

[0058] For example, as shown by the dotted arrow A6a in Figure 6A, if the surveying rod A2a is tilted so that the imaging direction of the image sensor 12 faces downward, the positions of the satellite signal receiver 11 and the image sensor 12 will be misaligned in the horizontal direction, as shown by the double-headed arrow A6b. Also, as shown in Figure 6B, if the surveying rod A2a is tilted to the left or right when viewed from the front, the positions of the satellite signal receiver 11 and the image sensor 12 will be misaligned in the horizontal direction, as shown by the double-headed arrow A6c.

[0059] If the positions of the satellite signal receiver 11 and the image sensor 12 are misaligned in the horizontal direction, when a process (translation, rotation, and scaling) is performed to overlap the coordinates (trajectory) of the satellite signal receiver 11 and the coordinates (trajectory) of the image sensor 12, the shapes of the two trajectories may differ significantly. If the shapes of the two trajectories differ significantly, the degree of overlap between the shapes of the two trajectories decreases, which may increase the error in the coordinate transformation included in the three-dimensional Helmert transformation matrix. If the error in the coordinate transformation included in the three-dimensional Helmert transformation matrix is ​​large, errors will occur in the point cloud data after the coordinate transformation.

[0060] Therefore, in the second embodiment, the position of the satellite signal receiver 11 is corrected so that it is aligned vertically with the image sensor 12, thereby suppressing errors contained in the point cloud data after coordinate transformation.

[0061] The following describes differences from the first embodiment. In the second embodiment, a correction unit that corrects the position of the satellite signal receiver 11 is inserted between the RTK calculation unit 21 and the conversion data generation unit 24 in the block configuration shown in Figure 3. The conversion data generation unit 24 generates conversion data based on the coordinates (trajectory) of the satellite signal receiver 11 corrected by the correction unit and the coordinates (trajectory) of the image sensor 12 output from the self-position estimation unit 22.

[0062] FIG. 7 is a diagram illustrating correction according to the second embodiment. For example, when measuring the position of the image sensor 12 using feature points included in the image data, the measuring device 1 can calculate the orientation of the image sensor 12. The orientation of the image sensor 12 is expressed by the deflection angles θ and φ, as shown in FIG. 7A . The deflection angle θ indicates the tilt of the image sensor 12 relative to the vertical direction when the surveying rod A2a is viewed from the side (see FIG. 6A ). The deflection angle φ indicates the tilt of the image sensor 12 in the left-right direction when the surveying rod A2a is viewed from the front (see FIG. 6B ).

[0063] Because the satellite signal receiver 11 and image sensor 12 are fixed to the fixed body A1a, the attitude (θ, φ) of the satellite signal receiver 11 can be determined by knowing the attitude of the image sensor 12. The distance between the satellite signal receiver 11 and the image sensor 12 is also known. Therefore, once the attitude of the image sensor 12 is calculated, it is possible to correct any horizontal deviation with respect to the image sensor 12.

[0064] 7B, the black circle A7a indicates the position of the image sensor 12. The black circle A7b indicates the position of the satellite signal receiver 11 before correction. The double-headed arrow A7c indicates the distance (known) between the satellite signal receiver 11 and the image sensor 12.

[0065] Based on the known distance between the satellite signal receiver 11 and the image sensor 12, indicated by the double-headed arrow A7c in Fig. 7B, and the attitude (θ, φ) of the satellite signal receiver 11, indicated by the double-headed arrow A7d, the measurement device 1 corrects the position of the satellite signal receiver 11 so that it overlaps with the image sensor 12 when viewed vertically, as indicated by the black circle A7e. This corrects, for example, the horizontal deviation between the positions of the satellite signal receiver 11 and the image sensor 12, indicated by the double-headed arrows A6b and A6c in Fig. 6. The attitude (θ, φ) of the satellite signal receiver 11, indicated by the double-headed arrow A7d, can be obtained by calculating the attitude of the image sensor 12, as described above.

[0066] 8 is a flowchart showing an example of the operation of the measuring device 1. A user U1 holds a survey rod A2a and moves around an object A2b from which point cloud data is to be acquired, and photographs the object A2b.

[0067] The measurement device 1 calculates the coordinates (trajectory) of the satellite signal receiver 11, which changes as the user U1 moves, based on the satellite signals received by the satellite signal receiver 11. The trajectory of the satellite signal receiver 11 is expressed in the ECEF coordinate system. The measurement device 1 stores the calculated trajectory of the satellite signal receiver 11 in a memory unit (S11).

[0068] The measuring device 1 calculates the coordinates (trajectory) of the image sensor 12, which change as the user U1 moves, and the attitude of the image sensor 12, based on the image data acquired by the image sensor 12. The trajectory and attitude of the image sensor 12 are expressed in the local coordinate system of the measuring device 1. The measuring device 1 stores the calculated trajectory and attitude of the image sensor 12 in a storage unit (S12).

[0069] The measuring device 1 generates point cloud data based on the image data acquired by the image sensor 12. The point cloud data is expressed in the local coordinate system of the measuring device 1. The measuring device 1 stores the generated point cloud data in a storage unit (S13).

[0070] The measuring device 1 calculates a three-dimensional Helmert transformation matrix for overlapping the trajectory of the image sensor 12 stored in the storage unit in S12 with the trajectory of the satellite signal receiver 11 (S14).

[0071] The measuring device 1 multiplies the trajectory and orientation of the image sensor 12 stored in the storage unit in S12 by the three-dimensional Helmert transformation matrix calculated in S14, and performs coordinate transformation of the trajectory and orientation of the image sensor 12 (S15). The trajectory and orientation of the image sensor 12 after the coordinate transformation in S15 are expressed in the ECEF coordinate system.

[0072] For each point (coordinate point) of the trajectory (positioning result) of the satellite signal receiver 11 stored in the storage unit in S11, the measurement device 1 selects the trajectory (coordinate point) of the image sensor 12 after coordinate transformation that is closest (S16). For example, the measurement device 1 selects the trajectory after coordinate transformation of the trajectory of the image sensor 12 acquired at the same timing as one epoch of the positioning result of the satellite signal receiver 11 stored in S11.

[0073] The measuring device 1 corrects the trajectory of the satellite signal receiver 11 using the attitude of the image sensor 12 after coordinate transformation that corresponds to the coordinate point selected in S16 (S17). For example, as described in FIG. 7 , the measuring device 1 corrects the positioning result of the satellite signal receiver 11 based on the attitude of the image sensor 12 selected in S16 (i.e., the attitude of the satellite signal receiver 11) and the distance between the satellite signal receiver 11 and the image sensor 12.

[0074] The measuring device 1 calculates a three-dimensional Helmert transformation matrix for superimposing the trajectory of the image sensor 12 stored in the storage unit in S12 on the trajectory of the satellite signal receiver 11 corrected in S17 (S18).

[0075] The measuring device 1 multiplies the point cloud data stored in the storage unit in S13 by the three-dimensional Helmert transformation matrix calculated in S18, and performs coordinate system transformation of the point cloud data (S19). That is, the measuring device 1 transforms the point cloud data stored in the storage unit in S13 from the local coordinate system to the absolute coordinate system of the ECFE coordinate system.

[0076] The measuring device 1 converts the point cloud data converted into the ECFE coordinate system in S19 into a geodetic coordinate system (S20).

[0077] The order of the processes of S11 to S13 is not limited to that shown in Fig. 8. The order of the processes of S11 to S13 may be reversed. The order of the process of S14 is not limited to that shown in Fig. 8. The process of S14 may be executed after the process of S12.

[0078] Summary of Second Embodiment As described above, the measuring device 1 corrects the horizontal deviation of the satellite signal receiver 11 with respect to the image sensor 12 based on the tilt of the satellite signal receiver 11 with respect to the vertical direction calculated based on the image data and the distance between the satellite signal receiver 11 and the image sensor 12. For example, the measuring device 1 corrects the position of the satellite signal receiver 11 so that the position of the satellite signal receiver 11 overlaps with the image sensor 12 when viewed from the vertical direction. This allows the measuring device 1 to reduce errors contained in the point cloud data after coordinate transformation.

[0079] <Modification> The measuring device 1 may correct the height (vertical distance) between the satellite signal receiver 11 and the image sensor 12 when the surveying rod A2a is tilted. For example, the measuring device 1 may calculate the vertical distance between the satellite signal receiver 11 and the image sensor 12 based on the attitude of the satellite signal receiver 11 calculated based on the image data and the distance between the satellite signal receiver 11 and the image sensor 12, and correct the height of the point cloud data in the absolute coordinate system. For example, the measuring device 1 may subtract the calculated vertical distance from the coordinate system-converted point cloud data.

[0080] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims. It is understood that such modifications or alterations also fall within the technical scope of the present disclosure. Furthermore, the components in the embodiments may be combined in any manner without departing from the spirit of the present disclosure.

[0081] In the above-described embodiments, examples have been described in which transformation data is generated to overlay a local coordinate system on an absolute coordinate system. However, transformation data may also be generated to overlay an absolute coordinate system on a local coordinate system. The coordinate system for which transformation data is generated may be changed according to user needs. Furthermore, while the Visual-SLAM coordinate system has been described as being a local coordinate system, it may also be an absolute coordinate system. In this case, the transformation data is data that converts the deviation between absolute coordinate systems. The type of coordinate system (hereinafter referred to as the "sensor coordinate system") is not important as long as point cloud data is acquired based on images captured by a sensor. Note that, as a method for acquiring point cloud data in an absolute coordinate system using Visual-SLAM, for example, a marker or the like whose absolute coordinates are known in advance may be included in the imaging range, and the coordinates of the point cloud data may be calculated based on the absolute coordinates indicated by the marker or the like.

[0082] The RTK calculation unit 21 may be referred to as a first acquisition unit. The self-position estimation unit 22 and the point cloud data generation unit 23 may be referred to as a second acquisition unit. The conversion data generation unit 24 may be referred to as a calculation unit. The coordinate conversion unit 25 may be referred to as a conversion unit. Calculation and generation may be read as calculation. Measurement may be read as measurement or positioning. "Same" may include "approximately the same".

[0083] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuit," "... assembly," "... device," "... unit," or "... module."

[0084] The present disclosure can be realized by software, hardware, or software in conjunction with hardware. Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may also be called an IC, system LSI, super LSI, or ultra LSI.

[0085] The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0086] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility.

[0087] The disclosures of the specification, drawings and abstract contained in Japanese Patent Application No. 2023-119955, filed on July 24, 2023, are incorporated herein by reference in their entirety.

[0088] The present disclosure is useful for a measurement device that acquires point cloud data in an absolute coordinate system.

[0089] REFERENCE SIGNS LIST 1 Measuring device 11 Satellite signal receiver 12 Image sensor 13 Calculating device A1a Fixed body A2a Survey rod 21 RTK calculation unit 22 Self-position estimation unit 23 Point cloud data generation unit 24 Conversion data generation unit 25 Coordinate conversion unit

Claims

1. A measurement device having a first acquisition unit that acquires the position of the receiver in an absolute coordinate system based on a satellite signal received by the receiver; a second acquisition unit that acquires the position of the sensor and point cloud data in the sensor coordinate system based on image data of an image captured by the sensor; a calculation unit that calculates conversion data for converting the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor; and a conversion unit that uses the conversion data to convert the point cloud data in the sensor coordinate system to the absolute coordinate system.

2. The measuring device according to claim 1, wherein the conversion unit generates, as the conversion data, data for performing a similarity transformation to superimpose the trajectory of positions acquired by the second acquisition unit onto the trajectory of positions acquired by the first acquisition unit.

3. The measuring device according to claim 2, wherein the receiver and the sensor are fixed to a fixed body.

4. The measuring device according to claim 3, further comprising a correction unit that corrects the height of the point cloud data in the absolute coordinate system based on the distance between the receiver and the sensor.

5. The measuring device described in claim 4, wherein the correction unit calculates the vertical distance between the receiver and the sensor based on the attitude of the receiver calculated based on the image data and the distance between the receiver and the sensor, and corrects the height of the point cloud data in the absolute coordinate system based on the vertical distance.

6. The measuring device according to claim 3, further comprising a correction unit that corrects horizontal deviation of the receiver relative to the sensor based on the attitude of the receiver calculated based on the image data and the distance between the receiver and the sensor.

7. The measuring device according to claim 6, wherein the correction unit corrects the position of the receiver so that the position of the receiver overlaps with the sensor when viewed in the vertical direction.

8. The measuring device according to claim 3, wherein the receiver and the sensor are fixed to the fixed body in close proximity to each other.

9. A measurement method comprising: acquiring the position of the receiver in an absolute coordinate system based on a satellite signal received by the receiver; acquiring the position of the sensor in a sensor coordinate system and point cloud data based on image data of an image captured by a sensor; calculating transformation data for transforming the point cloud data from the sensor coordinate system to the absolute coordinate system based on the position of the receiver and the position of the sensor; and transforming the point cloud data in the sensor coordinate system to the absolute coordinate system using the transformation data.

Images