Surveying support system, information display terminal, surveying support method, and surveying support program

Abstract

To perform an efficient scanning in a site.SOLUTION: The survey support system of the present invention is a survey support system which includes an information display terminal and a survey device for measuring the point group of a three-dimensional space. The survey support system includes: a terminal display unit for displaying a first image of a measurement target place imaged from the position of the information display terminal and a second image generated from information about the measurement target place; and a survey supporting unit for supporting a measurement by the survey device by using the terminal display unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , ,

[0005] , ,

[0001] The present disclosure relates to a survey support system, an information display terminal, a survey support method, and a survey support program.

Background Art

[0002] In recent years, construction work utilizing ICT technology has been carried out at construction sites. There is a demand for improving the efficiency and reducing the manpower of work using ICT technology, including labor shortages and recent infection prevention measures for infectious diseases.

[0003] As a system for acquiring three-dimensional data such as terrain and features using such ICT technology, there is known a system that measures an object from a plurality of points using a ground-mounted three-dimensional scanner device, acquires three-dimensional point cloud data, and displays it on a terminal (Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, as a method for checking the acquisition status of such three-dimensional point cloud data, there was no choice but to check the distribution of point cloud data on a plane map or to check the three-dimensional point cloud data by drawing it into a virtual space or the like imitating a three-dimensional space. However, with the method of checking the distribution of point cloud data on a plane map, it is not known whether the required amount of point cloud data in three dimensions is satisfied, and with the method of checking by drawing imitating a three-dimensional space, there is a limit to the space to be drawn and checked at one time, and it is a burden to check all areas. For this reason, it is not easy to proceed with the scanning work while grasping the acquisition status of the point cloud data.

[0006] In summary, the purpose of this disclosure is to provide a surveying support system, an information display terminal, a surveying support method, and a surveying support program that enable efficient scanning on-site in a system for measuring point clouds in three-dimensional space. [Means for solving the problem]

[0007] To achieve the above-mentioned objectives, the surveying support system according to the embodiment of this disclosure is a surveying support system comprising an information display terminal and a surveying device for measuring a point cloud in three-dimensional space, comprising: a terminal display unit that displays a first image of the place to be measured captured from the position of the information display terminal and a second image generated from information relating to the place to be measured; and a surveying support unit that uses the terminal display unit to support measurement by the surveying device.

[0008] Furthermore, in order to achieve the above-mentioned objectives, the information display terminal according to the embodiment of this disclosure includes a terminal display unit that displays a first image of the place to be measured captured by a terminal imaging unit and a second image generated from information relating to the place to be measured, and a surveying support unit that uses the terminal display unit to assist in the measurement of a surveying device that measures point clouds in three-dimensional space.

[0009] Furthermore, in order to achieve the above-mentioned objectives, the surveying support method according to the embodiment of this disclosure is a surveying support method in a surveying support system comprising an information display terminal and a surveying device for measuring a point cloud in three-dimensional space, and includes the steps of displaying a first image of the place to be measured, captured from the position of the information display terminal, and a second image generated from information relating to the place to be measured, on the terminal display unit, and using the terminal display unit to support measurement by the surveying device.

[0010] Furthermore, in order to achieve the above-mentioned objectives, the surveying support program according to the embodiment of this disclosure is a surveying support program in a surveying support system comprising an information display terminal and a surveying device for measuring point clouds in three-dimensional space, wherein the program causes a computer to perform the following steps: displaying a first image of the place to be measured, captured from the position of the information display terminal, and a second image generated from information relating to the place to be measured, on the terminal display unit; and using the terminal display unit to support measurement by the surveying device. [Effects of the Invention]

[0011] According to this disclosure, scanning can be performed efficiently on-site. [Brief explanation of the drawing]

[0012] [Figure 1] This is a diagram illustrating the configuration of a surveying system according to an embodiment of this disclosure. [Figure 2] This is a diagram showing the configuration of the pile-compatible table. [Figure 3] This is a flowchart illustrating the processing flow. [Figure 4] This is an example of a screen displayed on an information display terminal. [Figure 5] This is an example of a screen displayed on an information display terminal. [Figure 6] This is an example of a screen displayed on an information display terminal. [Figure 7] This is an example of a screen displayed on an information display terminal. [Figure 8] This is an example of a screen displayed on an information display terminal. [Figure 9] This is an example of a screen displayed on an information display terminal. [Figure 10] This is an example of a screen displaying the first and second images on an information display terminal. [Modes for carrying out the invention]

[0013] <Overview> For example, at an outdoor construction site such as a civil engineering project, before the start of construction or at the start of work, three-dimensional point cloud data (hereinafter sometimes referred to as point cloud data) is acquired using a surveying device such as a three-dimensional scanner, and an operation to confirm the site conditions is performed. Such site conditions are submitted as a result to the relevant institutions that require reports in a predetermined format such as a report. Acquisition of point clouds at the site typically involves installing a surveying device and performing the operation of acquiring the surrounding point clouds at that location multiple times by changing the location. Alternatively, it is also possible to mount a three-dimensional scanner or the like on a moving body and acquire point clouds while moving within the management area.

[0014] The confirmation of the acquisition status of the three-dimensional point cloud data is performed using an information display terminal such as a smartphone, a tablet, or a PC that can be browsed by the operator. At that time, the amount of point clouds is displayed by point cloud drawing in a virtual space simulating a three-dimensional space, and there is a method of outputting using so-called cross-reality (XR) such as augmented reality (AR) or mixed reality (MR). In the present embodiment, a system or the like that supports surveying by combining the point cloud drawing simulating this three-dimensional space with an actually captured image (including video) will be described.

[0015] Hereinafter, embodiments of the present disclosure will be described based on the drawings. FIG. 1 is a configuration diagram of a surveying system 1 according to an embodiment of the present disclosure.

[0016] <Configuration of the system> The surveying system 1 includes an information display terminal 100 used by an operator 2 and a surveying device 200. The operator 2 uses the surveying system 1 having these configurations to confirm the three-dimensional point cloud data acquired outdoors, for example, at a civil engineering construction site.

[0017] One embodiment of the surveying apparatus 200 for measuring and acquiring three-dimensional point group data is, for example, a three-dimensional scanner apparatus, and more specifically, for example, a three-dimensional laser scanner apparatus. The surveying apparatus 200 includes a surveying instrument storage unit 220, a scanner unit 260, an attitude driving unit 281, an attitude detection unit 282, a surveying instrument display unit 250, a surveying instrument operation unit 240, a surveying instrument communication unit 230, a surveying instrument processing unit 210, a surveying instrument position acquisition unit 270, and a surveying instrument imaging unit 290, and each of these components is electrically connected.

[0018] The surveying instrument storage unit 220 is a storage device using a memory or a magnetic disk, and stores various design information at the site. This design information includes, for example, BIM (Building Information Modeling), etc. Note that the design information is not limited to BIM, and may be, for example, three-dimensional CAD data. Also, it may be an image-like drawing with a scale added to an image file.

[0019] The scanner unit 260 is, for example, a laser scanner, and has a distance measuring unit 261 and a deflection unit 262. The distance measuring unit 261 has a function of performing distance measurement and angle measurement by irradiating laser light, which is distance measuring light, and receiving the reflected light.

[0020] The attitude driving unit 281 is an actuator that rotationally drives the scanner unit 260 in the horizontal and vertical directions. By driving the attitude driving unit 281, the direction of the scanner unit 260 can be changed.

[0021] The attitude detection unit 282 is a rotation angle sensor (encoder) capable of detecting the horizontal angle and vertical angle driven by the attitude driving unit 281. Also, the attitude detection unit 282 may have an inclination measuring device (tilt sensor) for detecting the inclination angle of the surveying apparatus 200. By the attitude detection unit 282, the direction in which the scanner unit 260 is pointing can be detected.

[0022] The surveying instrument display unit 250 is a display capable of displaying various types of information, such as a virtual space based on design information stored in the surveying instrument memory unit 220, measurement results from the scanner unit 260, and analysis results from the surveying instrument processing unit 210.

[0023] The surveying instrument control unit 240 is the part that allows for setting and operating the scanner unit 260 for measurement, the attitude drive unit 281 for driving, and so on. This surveying instrument control unit 240 may be physical buttons or it may be a touchscreen integrated with the surveying instrument display unit 250.

[0024] The surveying instrument communication unit 230 is a communication device capable of communicating with at least various information terminals. For example, the surveying instrument communication unit 230 may be a communication device that can connect to a network such as the Internet, or it may communicate by connecting to the information display terminal 100 wirelessly or via wired connection.

[0025] The surveying instrument position acquisition unit 270 has the function of acquiring the position of the surveying device 200, whether stationary or moving. Specifically, for example, it has the function of acquiring the position by setting up the instrument point of the surveying device 200, and can measure its own position by surveying a target such as a retroreflective prism installed at a known position. Alternatively, the surveying instrument position acquisition unit 270 may be a GNSS receiver. The position information acquired by the surveying instrument position acquisition unit 270 is the position information on the local coordinate system of the surveying device 200. Here, local coordinates refer to coordinates based on the design information of the site that is the target of the point cloud survey.

[0026] The surveying instrument imaging unit 290 is, for example, a camera that captures images. The images captured by the surveying instrument imaging unit 290 may be moving images or still images, and a camera capable of capturing so-called panoramic images that are long in the horizontal direction is preferred. Specifically, the surveying instrument imaging unit 290 may be a 360° spherical camera capable of capturing images in the horizontal direction, or a panoramic image may be captured by rotating a camera with a predetermined field of view 360 degrees. The images captured by the surveying instrument imaging unit 290 can be transmitted as image information to the information display terminal 100 via the surveying instrument communication unit 230.

[0027] The surveying instrument processing unit 210 is the central processing unit that performs various controls on the surveying device 200, and has functions realized by a program stored in the surveying instrument storage unit 220, including a point cloud data generation unit 221, a measured position calculation unit 222, and a display control unit 223.

[0028] The display control unit 223 generates a three-dimensional virtual space display of the construction site based on the design information stored in the surveying instrument storage unit 220, and has the function of displaying point cloud data displayed in the virtual space, as well as the measured position calculated by the measured position calculation unit 222, on the surveying instrument display unit 250.

[0029] The point cloud data generation unit 221 has the function of generating three-dimensional point cloud data from the distances of each distance measurement point (point cloud) measured by the scanner unit 260, and the horizontal angle and vertical angle detected by the attitude detection unit 282.

[0030] The measured position calculation unit 222 has the function of calculating the measured position of the three-dimensional point cloud data generated by the point cloud data generation unit 221.

[0031] The surveying device 200 may have other surveying functions. For example, it may be a total station (TS) with a three-dimensional scanner function. It may also have a mobile function that allows it to autonomously navigate a predetermined route, or its route may be controllable by remote operation. Examples of mobile objects with a mobile function include vehicles, robots, and unmanned aerial vehicles (UAVs). Vehicles also include heavy machinery that moves around the site.

[0032] The information display terminal 100 includes, for example, smartphones, feature phones, tablets, handheld computer devices (such as PDAs (Personal Digital Assistants)), wearable devices (such as glasses-type devices and watch-type devices). By installing application software on a general-purpose terminal, it can be used as an information display terminal according to this embodiment. These information display terminals 100 are equipped with a terminal display unit 150 and can be easily carried to construction sites. Furthermore, the terminal display unit 150 can be viewed hands-free or held with one hand. In addition, it is equipped with an internal power source such as a battery and can operate for a certain period of time without requiring an external power source.

[0033] The information display terminal 100 includes a terminal communication unit 130, a terminal storage unit 120, a terminal processing unit 110, a terminal input unit 140, a terminal display unit 150, a terminal position acquisition unit 160, a direction acquisition unit 170 (terminal direction acquisition unit), and a terminal imaging unit 180.

[0034] The terminal processing unit 110 executes functions and / or methods realized by code or instructions included in a program stored in the terminal storage unit 120 (not shown). The terminal processing unit 110 may include, for example, a central processing unit (CPU), MPU, GPU, microprocessor, processor core, multiprocessor, ASIC, FPGA, etc., and may realize each process disclosed in each embodiment by logic circuits and dedicated circuits formed on an integrated circuit, etc. Furthermore, these circuits may be realized by one or more integrated circuits, and the multiple processes shown in each embodiment may be realized by a single integrated circuit. In addition, although not shown, it may include a main memory unit that temporarily stores a program read from the terminal storage unit 120 and provides a working area to the terminal processing unit 110.

[0035] The terminal communication unit 130 can communicate with the surveying instrument communication unit 230 of the surveying device 200, and can receive three-dimensional point cloud data measured and calculated by the surveying device 200, position information of the surveying device 200, and image information captured by the surveying instrument imaging unit 290. The calculation of the measured position information for the three-dimensional point cloud data may be performed on the surveying device 200 side or on the information display terminal 100 side. Communication may be performed by wired or wireless, and any communication protocol may be used as long as communication between the two devices is possible.

[0036] The terminal input unit 140 is implemented by any or a combination of any type of device capable of receiving input from the user, i.e., the worker 2, and transmitting the information related to that input to the terminal processing unit 110. For example, it includes hardware input means such as buttons, software input means displayed on a display unit such as a touch panel, a remote controller, and voice input means such as a microphone.

[0037] The terminal display unit 150 is implemented by any or a combination of any type of device capable of displaying a screen. For example, this includes flat displays such as liquid crystal or OLED displays, curved displays, foldable screens provided on foldable terminals, head-mounted displays, or devices capable of displaying by projection onto a material using a small projector.

[0038] The terminal position acquisition unit 160 is, for example, a GNSS receiver and has the function of acquiring the stationary or moving position of the information display terminal 100. The position information acquired by the terminal position acquisition unit 160 is the global coordinates of the information display terminal 100.

[0039] The terminal imaging unit 180 is, for example, a camera that captures images, and is located on the back side of the terminal display unit 150 of the information display terminal 100, and can display the captured images on the terminal display unit 150. The images captured by the terminal imaging unit 180 may be moving images or still images.

[0040] The direction acquisition unit 170 is, for example, an electronic compass and has the function of detecting the Earth's magnetic field using a magnetic sensor and calculating the direction of the information display terminal 100. As a result, the direction acquisition unit 170 can acquire the direction of the imaging direction of the terminal imaging unit 180, that is, the direction that the worker 2 is looking at through the terminal display unit 150.

[0041] The terminal storage unit 120 has the function of storing various programs and data that are needed. In addition, it can store surveying information (three-dimensional point cloud data, position information of the surveying device 200) received by the terminal communication unit 130. For example, the terminal storage unit 120 stores information about the land used at the construction site (elevation, etc.) and design information including slope design information. The terminal storage unit 120 can be implemented using various storage media such as HDD, SSD, and flash memory.

[0042] Design information includes the design drawings necessary for construction work. Construction work refers to the construction of structures such as buildings, roads, railways, tunnels, bridges, ditches, waterways, and rivers. Design drawings include plan views, longitudinal section views, transverse section views, and the linear data, point data, the position and coordinates of each point and line segment, elevation, etc. contained within them.

[0043] Furthermore, the terminal storage unit 120 stores a pile correspondence table 128. As shown in Figure 2, the pile correspondence table 128 stores pile identification information 128a and pile location information 128b in correspondence. The pile correspondence table 128 can also store data for multiple pile identification information 128a and pile location information 128b. The pile identification information 128a is information that identifies the reference point RP installed at the site under measurement, as illustrated in Figure 1. Specifically, the pile identification information 128a corresponds to the identifier RP1 of the pile provided as the reference point RP. In this embodiment, the identifier RP1 is, for example, characters such as the pile number displayed on the pile, a passive tag (for example, an RFID tag) attached to the pile, etc.

[0044] The terminal storage unit 120 stores application software programs that implement various functions, including a surveying information acquisition unit 121, a range setting unit 122, a classification setting unit 123, a point cloud quantity calculation unit 124, a point cloud quantity display unit 125, a point cloud quantity management unit 126, and a surveying support unit 127. The terminal storage unit 120 may also store programs that have the functions of the point cloud data generation unit 221 and the actual measurement position calculation unit 222 stored in the surveying device 200, and these functions may be realized by executing them in the terminal processing unit 110.

[0045] The survey information acquisition unit 121 has the function of acquiring survey information, including point cloud data to which positional information is linked, from the surveying device 200. More specifically, three-dimensional point cloud data is generated by the point cloud data generation unit 221 through measurements by the scanner unit 260 of the surveying device 200. Positional information for each point cloud in the three-dimensional point cloud data is assigned and linked by the measured position calculation unit 222. The survey information acquisition unit 121 acquires the point cloud data to which the positional information is linked through communication between the surveying instrument communication unit 230 and the terminal communication unit 130.

[0046] The range setting unit 122 has the function of setting the display range of the point cloud. More specifically, it sets a predetermined range on the map of the construction site (on local coordinates) as the display range for which the amount of point cloud data should be managed. The range setting may be specified by the worker 2 using the terminal input unit 140 of the information display terminal 100 to input the display range of the point cloud, or it may be obtained using site management software, etc., which is installed and can be executed on a predetermined information processing management server that the information display terminal 100 can access by communication via the terminal communication unit 130. Alternatively, the range setting unit 122 may specify the range by reading the management area information contained in the design information etc. stored in the terminal storage unit 120. The display range can be specified, for example, by being enclosed by a boundary line that separates the area inside and outside the range on the map. In addition, for the set display range, location information on the map or design drawing is also provided so that the range is set on the map and can be displayed in link with the map or design drawing.

[0047] The division setting unit 123 has the function of dividing the display range into predetermined unit divisions. More specifically, it divides the display range set by the range setting unit 122 into a set of multiple unit divisions separated by predetermined unit area divisions. As a typical example, although not limited, it can be divided into square sections of the same area, so-called grid units. In addition to squares, any shape that can divide the display range into multiple continuous unit divisions without gaps, such as hexagons and triangles, can be used. The size of each unit division can also be freely set. Each unit division that intersects a boundary line may include parts that extend beyond the boundary line, or it may have a shape in which a part is cut off so as not to extend beyond the boundary line. Furthermore, location information is assigned to each unit division using the location information on the map that is assigned to the display range.

[0048] The point cloud quantity calculation unit 124 has the function of calculating the amount of point clouds contained in the space for each unit division. More specifically, the point cloud quantity calculation unit 124 can use the position information linked to the acquired three-dimensional point cloud data to compare it with the position information of each unit division and calculate the amount of point clouds contained in the space of each unit division. "Point cloud quantity" is a quantitative indicator related to point clouds, including the number of point clouds, the number of point clouds per unit area, the number of point clouds per unit volume, the average elevation value of the point clouds contained in the unit division, and other statistical quantities, and is not limited to just the number of point clouds.

[0049] Furthermore, only point clouds meeting predetermined conditions can be included in the calculation of the point cloud quantity. Specifically, it can be used to calculate only point clouds within a predetermined distance from the surveying device 200. For example, only point clouds within a radius of 30m from the surveying device 200 can be used for calculation. The predetermined distance here is determined by calculating the measurement distance that the surveying device 200 can use on-site using a predetermined verification method. A predetermined verification method might be an accuracy verification test in which the distance between two points obtained from measuring the distance between two or more known points using a total station, etc., is compared with the distance between two points obtained using the surveying device 200, and the difference is checked to see if it falls within a predetermined range. Also, since the measurement accuracy of the surveying device 200 generally decreases as the angle of incidence of the laser beam to the road surface becomes shallower, it is also possible to use only point clouds with an incidence angle greater than or equal to a predetermined angle for calculating the point cloud quantity.

[0050] Furthermore, the point cloud quantity calculation unit 124 may have a function to calculate the amount of point clouds contained in the space within a predetermined elevation range for each unit division. More specifically, it sets upper and lower limits for elevation values ​​and calculates the amount of point clouds contained in the space of the unit division within that elevation range.

[0051] Furthermore, the point cloud quantity calculation unit 124 may also have a function to calculate an index (sufficiency rate, etc.) relating to the sufficiency of the required point cloud quantity for the display range or unit division. More specifically, it can read the required point cloud quantity per unit division stored in the design information of the terminal storage unit 120, compare it with the actually calculated point cloud quantity, and calculate an index relating to the sufficiency of the required point cloud quantity. The index relating to the sufficiency of the required point cloud quantity may be a sufficiency rate (for example, "80% sufficient") or a deficiency rate (for example, "20% deficiency"). The index may also be expressed numerically, as a rank such as high, medium, or low, or as a predetermined threshold indicating only sufficiency or deficiency. Alternatively, the sufficiency rate, etc., may be calculated for all unit divisions to calculate the sufficiency rate, etc., for the entire display range. The overall satisfaction rate for the display range may be, for example, the average of the satisfaction rates for all unit categories, or it may be the number of unit categories that meet the required satisfaction rate divided by the total number of unit categories.

[0052] Furthermore, the point cloud quantity calculation unit 124 may also have a function to calculate predicted values ​​for the range and amount of point cloud data that can be acquired when a point cloud survey is performed at a given location, based on the location of the acquired point cloud. More specifically, when the surveying device 200 is installed in a location within the display range and a point cloud is to be acquired, the position information of the surveying device 200 is acquired as described above, and an estimate of the amount of three-dimensional point cloud data that can be acquired from around the surveying device 200 is calculated. Regarding the range and amount of point cloud data that can be acquired, information regarding the performance of the surveying device 200 is stored in the unit storage unit in advance, and the acquisition range and number of point cloud data can be calculated based on the performance of the device, centered on the location where the surveying device 200 is installed.

[0053] Furthermore, the point cloud quantity calculation unit 124 may also have a function to calculate an added value for the overlapping sections where, when calculating predicted values, there are overlapping sections within the predicted range where the point cloud quantity has already been calculated based on survey information and is considered an existing value, by adding the calculated point cloud quantity and the predicted point cloud quantity for the overlapping sections. More specifically, the unit can retrieve the survey information of already measured three-dimensional point cloud data as existing values ​​from the terminal storage unit 120, and if, among the predicted values ​​calculated as described above, there are areas where the observation range overlaps with the existing values, i.e., unit sections where the observation range overlaps, it can calculate an added value for that unit section by adding the existing value and the planned value to show an estimate of how the point cloud quantity will increase if surveying is performed from now on.

[0054] Furthermore, the point cloud quantity calculation unit 124 may calculate the sum of values ​​in unit divisions where the above observation ranges overlap, not only using existing values ​​and predicted values, but also using already measured existing values ​​and sums of existing values.

[0055] The point cloud quantity display unit 125 has the function of displaying the calculated point cloud quantity for each unit category on the information display terminal 100 for each section of the display range. More specifically, if the point cloud quantity is the number of point clouds, the number of point clouds per unit area, or the number of point clouds per unit volume, the calculated point cloud quantity, the predicted value of the point cloud quantity, and the sum of the calculated point cloud quantity and the existing value of the point cloud quantity can be displayed differently depending on the level of the point cloud quantity, so that the level of the point cloud quantity can be seen. Most typically, rather than being limited, this is color coding. For example, unit categories with a high point cloud quantity can be displayed in red, unit categories with a low point cloud quantity in blue, and yellow in between, using a color distribution. Also, if the point cloud quantity is the elevation value of the point cloud, the average value of the elevation values ​​of the point clouds included in the unit category, or a statistical amount thereof, unit categories with a high elevation can be displayed in red, parts with a low elevation in blue, and yellow in between, using a color distribution.

[0056] Furthermore, the point cloud quantity display unit 125 may also have the function of displaying the amount of point clouds included within a predetermined elevation range for each unit division, an index indicating the sufficiency of the required amount of point clouds for the display range or unit division, displaying the amount of point clouds for each division according to the predicted value, displaying the amount of point clouds according to the sum obtained by adding the predicted value to the calculated existing value, and displaying the amount of point clouds according to the sum obtained by adding two existing values. More specifically, the method of displaying the indicators so that the elevation can be seen, as in the color coding display described above, can also be applied to the amount of point clouds included within a predetermined elevation range for each unit division, an index indicating the sufficiency of the required amount of point clouds for the display range or unit division, displaying the amount of point clouds for each division according to the predicted value, and the amount of point clouds according to the sum obtained by adding the predicted value to the calculated existing value.

[0057] Furthermore, the point cloud quantity display unit 125 aligns the position information acquired by the terminal position acquisition unit 160 with the local coordinates including the point cloud display range, and can display the position of the information display terminal 100 on the local coordinates on the terminal display unit 150. As a method for aligning the global coordinates acquired by the terminal position acquisition unit 160 with the local coordinates of the site being surveyed (coordinate transformation), for example, worker 2 can move the information display terminal 100 to a known point on the local coordinates and perform an operation to align the coordinates. The point cloud quantity display unit 125 then aligns the known point on the local coordinates with the position information acquired by the terminal position acquisition unit 160, thereby aligning it to the local coordinates. As known points, for example, control points or points surveyed with a total station or GSNN that contain global coordinate position information can be used.

[0058] Furthermore, the point cloud quantity display unit 125 can overlay an image (second image) related to the measured area, which indicates unit divisions and point cloud quantity generated and drawn using CG (Computer Graphics) or the like, onto the image (first image) captured by the terminal imaging unit 180 (first image) in three-dimensional space, based on the position of the information display terminal 100 on local coordinates and the orientation of the imaging direction of the terminal imaging unit 180. This enables XR (X Reality or Cross Reality) display, such as AR (Augmented Reality) or MR (Mixed Reality). The information about the area to be measured, which is displayed overlaid on the image captured by the terminal imaging unit 180, is not limited to unit divisions or point cloud quantity. It may also display indicators regarding the sufficiency of the required point cloud quantity calculated by the point cloud quantity calculation unit 124, the range that can be obtained when a point cloud survey is performed, and predicted values ​​of the point cloud quantity, etc., for each unit division. The current installation location of the surveying device 200, the installation location of the surveying device 200 at past surveying points, the location of known points, the location of a mobile object whose location is known, etc., may also be displayed. It is preferable that the installation location of the surveying device 200, the location of known points, the location of a mobile object, etc., be indicated by icons so that the operator 2 can easily identify them.

[0059] When the point cloud quantity display unit 125 displays the scanned point cloud quantity (which may include information on missing measurements) on the terminal display unit 150, the survey plan can be easily modified, such as by prioritizing scanning in locations with a small point cloud quantity (for example, locations with a low point cloud density and many missing points).

[0060] Furthermore, the information display terminal 100 can issue point cloud measurement instructions to the surveying device 200, whose position is displayed superimposed on the image, via the terminal input unit 140. Specifically, the operator 2 specifies the surveying device 200 displayed on the terminal display unit 150 via the terminal input unit 140 of the information display terminal 100, sets the conditions for measuring the point cloud, and executes the survey. The conditions for measuring the point cloud include, for example, setting the horizontal angle, vertical angle range, and measurement distance to define the scanning range. At this time, the point cloud quantity display unit 125 can display the image captured by the surveying instrument imaging unit 290 of the surveying device 200, which is the target of the operation, on the terminal display unit 150. The operator 2 can identify the orientation of the surveying device 200 from the image captured by the surveying instrument imaging unit 290 and set the conditions for the survey.

[0061] The surveying device 200 may measure the point cloud within a limited range according to the conditions for surveying the point cloud set by the operator 2 as described above, or it may measure the point cloud within the display range of the point cloud set by the range setting unit 122. Specifically, the point cloud quantity display unit 125 calculates the range that the surveying device 200 can measure based on the orientation of the surveying device 200, using the current position information of the surveying device 200 and the image information captured by the surveying instrument imaging unit 290, and sends an instruction to the surveying device 200 to perform the measurement excluding the range outside the display range set by the range setting unit 122 of the information display terminal 100.

[0062] The point cloud quantity management unit 126 has a function to display information indicating a shortage of point cloud quantity on the information display terminal 100 if there is a section within the display range where the point cloud quantity has not reached a predetermined required amount. More specifically, as in the example above, the unit determines whether the point cloud quantity is sufficient or insufficient for each unit section using a predetermined threshold for the point cloud quantity which is stored in the terminal storage unit 120 in advance, and displays an alert on the terminal display unit 150 of the information display terminal 100 to inform the operator 2 that the quantity is insufficient for the unit section. Furthermore, if the point cloud quantity is the elevation value of the point cloud, the average value of the elevation value, or a statistical value thereof, the unit may compare it with other reference three-dimensional data, such as the design elevation value of the design information to be compared, and determine whether it is higher or lower than the reference value for each unit section, and similarly display an alert to inform the operator 2 that the unit section is outside the reference.

[0063] Furthermore, the point cloud quantity management unit 126 has the function of calculating the accuracy of the instrument placement of the surveying device 200 from point cloud data measured from multiple installation locations, and displaying information regarding the accuracy of the instrument placement on the information display terminal 100. More specifically, in a unit section containing multiple point cloud data measured from different directions, it calculates comparison indicators such as average elevation and median for each of the multiple point cloud data, and calculates the accuracy of the instrument placement based on the difference between these comparison indicators.

[0064] For example, if there is point cloud data (first point cloud data Da, second point cloud data Db, third point cloud data Dc) measured from at least three installation locations within the same unit division, the difference (Ea-Eb, Ea-Ec, Eb-Ec) of the average elevation (Ea, Eb, Ec) calculated from each point cloud data is calculated as the accuracy of each instrument installation. The point cloud quantity management unit 126 determines that there is no problem with any of the instrument installations if this difference is less than a predetermined value. On the other hand, if any two of the differences are greater than or equal to a predetermined value, it determines that there is a problem with the instrument installation. For example, if the difference Eb-Ec, which does not involve the first point cloud data Da, is less than a predetermined value, while the differences Ea-Eb and Ea-Ec, which do involve the first point cloud data Da, are both greater than or equal to a predetermined value, it is determined that there is a problem with the instrument installation at the location where the first point cloud data Da was measured.

[0065] The point cloud quantity management unit 126 then displays the difference between the comparison indices described above for each unit category on the information display terminal 100 as information regarding the accuracy of instrument placement. For example, unit categories where the difference between the comparison indices is less than a predetermined value may be displayed in blue, unit categories where the difference is greater than or equal to the predetermined value may be displayed in red, and if there is no point cloud data measured from at least three locations within the same unit category, it may be displayed in white, or the numerical value of the difference between the comparison indices may be displayed for each unit category. This display of instrument placement accuracy may be displayed together with the display of the point cloud quantity described above, or it may be displayed on a separate screen. Furthermore, for unit categories that are judged to have problems, an alert may be displayed as a detailed display requesting re-examination of the surveying position, installation method, etc. of the surveying device 200 that measured the point cloud data, and requesting re-measurement. Note that it is not necessarily required to calculate for each unit category where overlap occurs; the accuracy of instrument placement may be calculated and displayed for the entire portion where the point cloud data overlaps, such as showing the percentage of unit categories where the difference with the comparison indices of other point cloud data is greater than or equal to a predetermined value.

[0066] The surveying support unit 127 performs surveying support functions to assist the surveying device 200 by utilizing the display function of the terminal display unit 150, which displays a first image of the area to be measured, captured from the position of the information display terminal 100, and a second image generated by computer graphics (CG) or the like from information about the area to be measured. As a display function, the terminal display unit 150 displays information related to surveying support in an image (e.g., AR) that combines the first and second images. The second image may include an image of the terrain based on point cloud data acquired by the surveying device 200. Furthermore, if the information display terminal 100 has a Lidar (Light Detection and Ranging) sensor and an Inertial Measurement Unit (IMU), the terminal display unit 150 may display the terrain recognized by the information display terminal 100. Alternatively, the information display terminal 100 may recognize the terrain using equipment that implements SLAM (Simultaneous Localization and Mapping) (also called Visual SLAM, SfM (Structure from Motion or Shape from Motion)) functionality using images. This allows for a simple understanding of the terrain even in areas where point cloud data acquisition by the surveying device 200 is insufficient. Furthermore, when the surveying support unit 127 displays a terrain image as the first image on the terminal display unit 150, it may also display a 2D map on the ground as a second image. Alternatively, the surveying support unit 127 may have a function to display a 2D map of the target area on the terminal display unit 150. This allows the operator 2 to be provided with information regarding the terrain of the area being surveyed and other information about the area being measured.

[0067] The surveying support unit 127 has one or more of the following functions as surveying support functions: for example, a stake point location display function, a hazard zone display function, a blind spot information display function, a position correction function, a guidance function, a scan instruction function, a warning function, and a coordination function.

[0068] The stake point location display function displays the stake location RP on the terminal display unit 150. Within the range of the first image displayed on the terminal display unit 150, the terminal display unit 150 displays a stake point icon image as a second image at the actual stake location RP.

[0069] The hazard zone display function displays hazard zones in the area being measured on the terminal display unit 150. Specifically, the terminal display unit 150 displays an image indicating the hazard zone as a second image containing information about the area being measured, along with the first image acquired by imaging. Examples of hazard zones include: (1) work zones determined from data such as the working range of heavy machinery in operation, acquired from other systems (e.g., systems including external devices connected via the Internet); (2) work zones set from today's work content entered from an external network or other terminals; (3) unstable zones in the scan data acquired by the surveying device 200 (e.g., sloping zones indicated by point cloud data); and (4) zones with a high (or increased) degree of danger at the site, determined based on weather information (e.g., zones including land loosened by rainfall). One or more of these can be combined. In this way, by identifying hazard zones and displaying them on the terminal display unit 150, the worker 2 can avoid hazard zones along their route. Furthermore, in the guidance function described in detail later, when the information display terminal 100 requests a recommended route, it is also possible to display a route that avoids dangerous areas as the recommended route.

[0070] The blind spot information display function displays information about the measurement site that is difficult to see from the current location of the information display terminal 100. Specifically, the terminal display unit 150 displays information about the measurement site located in the blind spot in the first image using the second image. The terminal display unit 150 can display one or more of the following: the location of a reference point that is not within the line of sight, the boundary location of the work area, the location of heavy machinery or other vehicles that are hidden in the shadow of an embankment, etc.

[0071] The position correction function is a function that corrects the position information of the information display terminal 100. Here, we will explain an example in which the information display terminal 100 performs correction based on known position information of a stake. The information display terminal 100 reads the identifier RP1 from the stake installed at the measurement site and corrects the position information of the information display terminal 100 based on the position of the stake indicated by the stake position information 128b corresponding to the stake identification information 128a associated with the identifier RP1.

[0072] Specifically, the information display terminal 100 uses the terminal imaging unit 180 to capture images of the stake characters displayed or engraved as identifier RP1, reads them using a character recognition program, and automatically recognizes the stakes that will serve as reference points. The stake information acquisition function aligns the coordinate information acquired by the terminal position acquisition unit 160, exemplified by the GNSS receiver of the information display terminal 100, with the coordinate information of the site where the measurement is being taken. The information display terminal 100 reads the identifier RP1 of multiple (e.g., two or more) stakes and performs the operation of aligning the coordinate information acquired by the terminal position acquisition unit 160 with the coordinate information of the site multiple times (e.g., two or more times), thereby matching the rotation angle between the coordinate information of the terminal position acquisition unit 160 and the coordinate information of the site. The position correction function performs a coordinate transformation from position information based on latitude and longitude to position information based on site coordinates. After that, the information display terminal 100 uses its internal electronic compass to match the rotation (or direction) with the coordinates of the site. The information display terminal 100 may also display information related to position accuracy on the terminal display unit 150, reflecting it as visual information. In this way, the position information of the information display terminal 100 is corrected using reference points based on stake position information.

[0073] The guidance function is a function in which the surveying support unit 127 guides the information display terminal 100 to move along an appropriate route to a selected location on the site to be measured. If the information display terminal 100 is held by a worker 2 on the land such as the site to be measured, the movement of the worker 2 can be guided along with the information display terminal 100.

[0074] Specifically, the point cloud quantity display unit 125 displays the point cloud quantity for each unit division, which is a division of the display range of the survey information, as a second image on the terminal display unit 150 of the information display terminal 100, within the imaging range of the terminal imaging unit 180 of the information display terminal 100. When a destination is specified from the current position of the information display terminal 100, the information display terminal 100 presents a recommended route from the current position of the information display terminal 100 to the destination as a second image displayed on the terminal display unit 150. The destination can be specified using any method; for example, the operator 2 may input it into the terminal input unit 140, or it may be done automatically when the surveying device 200 finishes the current survey based on a procedure such as a scan plan described later.

[0075] Furthermore, the information display terminal 100 may acquire a pre-set survey scan plan from the terminal storage unit 120, the surveying instrument storage unit 220, and the storage unit of an external device (not shown), determine the destination of the information display terminal 100 based on this scan plan, and present a recommended route to the terminal display unit 150. The scan plan can be set, for example, as a route that allows for efficient surveying with fewer setups or movements of the surveying device 200, based on already acquired point cloud data (terrain data) or known terrain data within the area where surveying work is planned or the area where surveying work is currently being performed (display range DA, etc., described later). However, the content of the scan plan is not limited to this.

[0076] Furthermore, the information display terminal 100 may accept the selection of a stake icon image and present a recommended route from the current location of the information display terminal 100 to the location of the selected stake.

[0077] If the position of the information display terminal 100 has been corrected using the position correction function described above, the position information of the information display terminal 100 using GNSS and a compass will match the coordinate system of the site being measured. For example, the terminal display unit 150 displays the next scan candidate position as general information by combining the first and second images (for example, as an AR screen). The surveying support unit 127 can display a route with less elevation as a recommended route on the terminal display unit 150, based on the point cloud data that has already been scanned. The surveying support unit 127 may also display multiple route candidates that allow for efficient movement (for example, recommended routes), and the surveying support unit 127 can display one of the routes selected by the worker 2 from among the multiple recommended routes as the next route to proceed on the terminal display unit 150. As an example of route display, for example, an arrow can be displayed on the screen of the terminal display unit 150 from the current position of the information display terminal 100 to the planned or selected scan location. The terminal display unit 150 may also display the distance from the current position to the target position (next position). Thus, the surveying support unit 127 has a guidance function that guides the surveying device 200 to its next position.

[0078] The warning function alerts the operator 2 if an inappropriate area is selected as a scan instruction for the surveying device 200. The terminal display unit 150 displays the work management area. For example, if the scan instruction selects an area beyond the management area, the terminal display unit 150 displays a warning (alert). The warning function may also provide warnings through other output means, such as sound output using a speaker.

[0079] The collaboration function allows the information display terminal 100 to communicate and collaborate with equipment installed in other locations (for example, external equipment installed in an office, etc.) via an external network (for example, a cloud network). As an example of the collaboration function, the information display terminal 100 displays instructions received remotely from an external device. An example of an instruction is to display the drawing content on the external device on the terminal display unit 150, and the information display terminal 100 can display images input from the external device (for example, images captured by the external device, images drawn or generated by the external device, or images received from the information display terminal 100, etc.) on the terminal display unit 150 and share them with the worker 2 on site.

[0080] Furthermore, external devices may be configured to display the contents of the terminal display unit 150 on their own display unit (for example, by duplicating and displaying part or all of it). In this way, the information display terminal 100 can also reflect drawing input from other devices that share the first and second images into the drawing contents of the second image on the terminal display unit 150.

[0081] The display image on the terminal display unit 150 can be, for example, (1) an illustration or rough sketch drawn on a shared AR screen on a PC in an office, or (2) an illustration or rough sketch drawn on a 2D map displayed on a PC in an office. The terminal display unit 150 reflects these display images as AR images, etc. The terminal display unit 150 can also confirm the display content input from an external device on a 2D map.

[0082] A surveying system 1, comprising an information display terminal 100 and a surveying device 200 for measuring point clouds in three-dimensional space, and having a measurement support function provided by a surveying support unit 127, functions as a surveying support system. The surveying support system may also have functions other than surveying support. Furthermore, the surveying support system may support only a portion of the work processes in surveying, or it may support a series of work processes in surveying.

[0083] <Processing flow> Figure 3 shows a flowchart illustrating the processing flow of the method and program using the surveying system 1 according to the embodiment of this disclosure.

[0084] First, in step S101, the terminal location acquisition unit 160 acquires the location information of the information display terminal 100.

[0085] In step S102, the point cloud quantity display unit 125 aligns the position information of the information display terminal 100 acquired in step S101 with local coordinates that include the display range of the point cloud.

[0086] In step S103, the survey information acquisition unit 121 acquires survey information, including point cloud data linked to positional information, from the surveying device 200. Note that this acquisition of point cloud data may be performed after the range setting in step S104 and the section setting in step S105, which will be described later.

[0087] In step S104, the range setting unit 122 sets the display range of the point cloud. Figure 4 shows an example of a screen displayed on the terminal display unit 150 of the information display terminal 100. In this example, the information display terminal 100 is a so-called tablet terminal, the terminal display unit 150 is, for example, a liquid crystal display or an OLED display, and the terminal input unit 140 is an input unit realized by the touch panel function of these displays. In this figure, the terminal display unit 150 displays a map, aerial photograph, design drawing, or other map display (not shown). The map display is displayed, for example, as a background within the display range. On this map display, the operator 2 inputs the display range while referring to the map display, and the range setting unit 122 sets the display range based on that input. Input can be done by freely drawing boundary lines using a finger or stylus, or by setting two diagonal points to set a rectangular range, etc. In this figure, the display range DA is displayed on the map display. At this point, position information is also assigned to the display range DA.

[0088] In step S105, the division setting unit 123 divides the display range into predetermined unit divisions. Figure 5 shows an example of a screen displayed on the terminal display unit 150 of the information display terminal 100. In this example, the display range DA is divided and separated into multiple unit divisions GU. Note that in the diagram, the unit divisions are exaggerated and shown as large for clarity, but in reality, it would be easier to understand if they were made more detailed, such as 1 dot or 1 pixel. Also, in this diagram, the unit divisions outside the boundary line are displayed in the area where the boundary line of the display range DA intersects, but the area outside the boundary line does not need to be displayed. Note that the acquired three-dimensional point cloud data can be managed regardless of whether it is enclosed or not within the display range.

[0089] In step S106, the point cloud quantity calculation unit 124 calculates the point cloud quantity contained in the space for each unit division. The point cloud quantity is calculated for all of the above-mentioned unit divisions.

[0090] In step S107, the point cloud quantity display unit 125 displays the calculated point cloud quantity for each unit of the display range on the information display terminal 100. Figure 6 shows an example of the screen displayed on the terminal display unit 150 of the information display terminal 100. In this example, the surveying device 200 has been installed and surveyed in the unit of position SP, and the point cloud quantity from the three-dimensional point cloud data acquired at this surveying device position SP is shown in varying shades of color according to the point cloud quantity of the unit units surrounding position SP. In reality, the point cloud is not acquired at the location where the surveying device 200 is installed and directly below it, so the point cloud quantity will be low, but in this figure, for ease of understanding, the point cloud quantity is shown to be higher the closer it is to the surveying device position SP. For example, between unit unit GU2 and unit unit GU1, unit unit GU2 is displayed in a darker color, making it intuitively clear that the acquired point cloud quantity is higher. Similarly, as information regarding the accuracy of instrument installation, the difference in comparative indicators such as elevation based on point cloud data from multiple installation locations can also be displayed. In this way, the point cloud acquisition status and differences in comparison indicators are clearly displayed using color coding, and the display range can be viewed at a glance. This allows operator 2 to easily check whether the amount of points in the 3D point cloud data and the accuracy of the instrument setup are being acquired in the required number and with the required accuracy in each section of the display range where the amount of points should be managed. This enables efficient scanning on-site and allows for visual confirmation of the scanning status and data acquisition results.

[0091] Figure 7 shows an example of a screen displayed on the terminal display unit 150 of the information display terminal 100. This figure explains an example in which the predicted value of the point cloud amount that can be obtained by the next measurement is added to the existing value of the point cloud that has already been acquired, and the result is displayed as the added value. For example, in the preceding stage of this figure, as shown in Figure 6, the surveying device 200 has already been installed at a certain location SP and the point cloud has been acquired. In contrast, when the surveying device 200 is installed at a new location NSP, a predicted value of how the point cloud will be acquired and added is calculated, and for areas where the observation ranges overlap, the predicted value is added to the existing value to calculate the added value. Then, the unit divisions within the display range are displayed according to the point cloud amount of the added value. For example, unit division GU3 is the unit division in which the observation range of the surveying device 200 at location SP and the observation range of the surveying device 200 at location NSP overlap. In this unit category GU3, the point cloud quantity, which is an existing value already measured at position SP, and the point cloud quantity, which is a predicted value expected to be obtained by measuring at position NSP, are added together and displayed as the sum. Therefore, the point cloud quantity in unit category GU3 is displayed in a darker color than in unit category GU4, where the observation ranges of the two points do not overlap. This allows worker 2 to conduct the survey while considering where to set up the surveying device 200 to acquire the point cloud in order to proceed with the work efficiently.

[0092] Figure 8 shows an example of a screen displayed on the terminal display unit 150 of the information display terminal 100. This figure illustrates an example of an alert display related to the sufficiency rate, etc. In the unit divisions included in the display range DA shown in this figure, the point cloud quantity is high throughout almost the entire area, but as indicated by "5% missing" in the upper left of the screen, the point cloud quantity is low in some divisions. For example, unit division GU5 is a unit division in which the point cloud quantity has not reached the required point cloud quantity, and such unit divisions are displayed with an alert, for example, by hatching, to distinguish them from other unit divisions. Whether or not to display an alert is determined by the point cloud quantity management unit 126 based on the level of the point cloud quantity in this unit division compared with the required point cloud quantity stored in advance, as described above. In this way, by displaying an alert for unit divisions that have not reached the required point cloud quantity, the operator 2 can perform the point cloud acquisition work without any omissions or errors. In addition, by displaying an alert for unit divisions with low accuracy of point cloud data, the operator can easily improve the accuracy of the point cloud data by re-verifying the surveying position, installation method, etc. of the surveying device 200 that measured the point cloud data and requesting re-measurement.

[0093] Then, in step S108, the terminal display unit 150 displays the location of the information display terminal 100 (terminal location). For example, as shown in Figure 7 above, the location of the information display terminal 100 is displayed as terminal location T on the terminal display unit 150 on the unit division displayed in step S105.

[0094] Furthermore, in step S109, the point cloud quantity display unit 125 overlays the unit divisions and point cloud quantity onto the image captured by the terminal imaging unit 180 in three-dimensional space, based on the position of the information display terminal 100 on local coordinates and the orientation of the imaging direction of the terminal imaging unit 180, and performs AR display. The conventional two-dimensional image and AR display can be switched as needed by switching operations via the terminal input unit.

[0095] Figure 9 shows an example of the AR display screen on the terminal display unit 150 of the information display terminal 100. As shown in the figure, the terminal display unit 150 displays the image captured by the terminal imaging unit 180 as is during AR display, and the unit divisions and the point cloud quantities for each unit division are displayed in different colors on the image. In addition, in Figure 9, the current installation position PSP of the surveying device 200, the surveying device position SP of the surveying device 200 at past surveying points, the reference point position RP, and the azimuth information DI are each displayed as icons. Furthermore, a panoramic image captured by the surveying instrument imaging unit 290 is displayed as a pop-up above the surveying device 200 at the current surveying point. This panoramic image captures the scenery as seen from the surveying instrument imaging unit 290, and includes, for example, worker 2, the reference point RP which is also captured in the AR display, and the moving object V.

[0096] Furthermore, in step S110, remote control such as point cloud measurement instructions is performed on the designated surveying device 200 via the terminal input unit 140. For example, in Figure 9, when the operator 2 selects the current installation position PSP of the surveying device 200 displayed on the terminal display unit 150, a survey setting screen (not shown) opens. The operator 2 inputs the surveying conditions into the survey setting screen and performs the execution operation, and the surveying device 200 performs point cloud surveying according to the surveying conditions. From the reference point RP, which is also visible in the AR display of the panoramic image captured by the surveying instrument imaging unit 290, the operator 2 can determine the orientation of the surveying device 200. For example, by selecting the reference point RP in this panoramic image, the operator 2 can indicate the approximate position of the reference point scan for instrument installation of the surveying device 200 based on the reference point RP. Upon receiving this instruction, the surveying device 200 observes the reference point RP selected by the operator 2 and identifies the instrument coordinates on the local coordinate system. Furthermore, by setting the surveying conditions to the range that can be surveyed within the display range DA, the surveying device 200 performs point cloud measurements excluding the range outside the display range.

[0097] In step S111, the information display terminal 100 executes the surveying support function of the surveying support unit 127 described above. Figure 10 shows an example in which the terminal display unit 150 displays a first image 31 of the area to be measured and a second image 32 generated by CG or the like from information about the area to be measured. Also in Figure 10, for explanatory purposes, a schematic image of the first image 31 is shown by a dashed line, and the first image 31 and the second image 32 are superimposed. The area to be measured displayed as the first image 31 includes an unleveled embankment 311 as part of the terrain. Here, the stake point position display function, hazard zone display function, blind spot information display function, position correction function, guidance function, warning function, and cooperation function will be explained using the display contents of Figure 10 as an example.

[0098] The stake location display function and the blind spot information display function display the location and number of stakes, which are information about the measurement area that is difficult to see, as stake icon images 321. For example, stake icon image 321a indicates stake number "GCP001", and stake icon image 321b indicates stake number "GCP002". Although multiple stakes may be included in the imaging range on the screen of the terminal display unit 150, the presence of the stakes may not be clearly visible in the first image due to the distance between the stakes and the information display terminal 100, or because the stakes are located in blind spots in the terrain. However, since the stake icon image 321 is displayed on the terminal display unit 150 as a second image 32, the location of the stakes can be easily determined even when the stakes are difficult to see from the position of the information display terminal 100.

[0099] The hazard zone display function, for example, displays the extent of the area and the type of hazard within the terminal display unit 150 using a second image 32 in an appropriate manner if there is a hazard zone within the display range shown in Figure 10.

[0100] The guidance function, for example, displays a recommended route R from the current location to the destination location on the information display terminal 100 when worker 2 specifies a destination location from the measurement site displayed on the terminal display unit 150. In the example in Figure 10, stake number "GCP002" is selected as the destination by selecting the stake point icon image 321b. The recommended route R is displayed by an arrow connecting the current location to the destination. In the example in Figure 10, since the information display terminal 100 has already determined from point cloud data acquisition that there is an embankment 311 between the current location and the destination stake number "GCP002", the recommended route R is displayed as a curved route that avoids the embankment 311 rather than a straight route. Multiple recommended routes R may be displayed (for example, routes that pass through both the left and right sides of the embankment 311 as viewed from the information display terminal 100), allowing worker 2 to select the route to actually use. The terminal display unit 150 may also display the distance for each recommended route R.

[0101] The position correction function corrects the position information of the information display terminal 100 by, for example, moving the information display terminal 100 to the stake with stake number "GCP002," and then obtaining stake identification information 128a from the identifier of that stake (same as the identifier RP1 in Figure 1) using the information display terminal 100. An example of the method for correcting the position information is as described above.

[0102] The warning function can issue a warning, such as by displaying it on the terminal display unit 150, if an area outside the display range DA is selected as an inappropriate area for scanning.

[0103] The collaboration function allows drawing input from other devices to be reflected in the drawing content of the second image on the terminal display unit 150. In Figure 10, a frame indicating the handwritten input area is displayed as an image 322 of an illustration or rough sketch input by another device. This allows the worker 2 on site, along with the information display terminal 100, to share information with managers and others in other locations while viewing the screen display.

[0104] After completing the process in step S111, the surveying device 200 is moved to the next installation location to scan areas with a small amount of point cloud data, and the information display terminal 100 can repeat the processes from step S101 to step S111 as needed. In this case, some of the processes from step S101 to step S111 can be omitted.

[0105] This disclosure describes a technology that supports surveying by using AR technology to confirm the current situation during scanning work at surveying and measurement sites. The surveying support system according to the embodiment of this disclosure is a surveying support system comprising an information display terminal 100 and a surveying device 200 for measuring point clouds in three-dimensional space, comprising a terminal display unit 150 that displays a first image 31 of the site to be measured, captured from the position of the information display terminal 100, and a second image 32 generated from information about the site to be measured, and a surveying support unit 127 that uses the terminal display unit 150 to support measurement by the surveying device 200.Therefore, it is possible to configure a surveying support system, information display terminal 100, surveying support method, and surveying support program that allow for intuitive and easy understanding of the work status and the content of the next work, and enable efficient scanning at the site.

[0106] Furthermore, by providing a point cloud quantity display unit 125 that displays the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal 100 as a second image 32 on the information display terminal 100, the operator 2 can move to any location on the measurement site and check the acquisition status of the point cloud data near each area, thereby making it easy and reliable to grasp the acquisition status.

[0107] Furthermore, if the information display terminal 100 is configured to display a recommended route R from the current location of the information display terminal 100 to the destination location as a second image 32 on the terminal display unit 150, the worker 2 can move safely and efficiently while holding the information display terminal 100.

[0108] Furthermore, if the information display terminal 100 is configured to determine its destination based on a pre-set survey scan plan and present a recommended route R, the workload for the worker 2, such as considering the route to the next destination while observing the point cloud data acquisition status, can be reduced.

[0109] Furthermore, if the information display terminal 100 displays a pile point icon image 321 as a second image 32 at the actual pile location, accepts the selection of the pile point icon image 321, and presents a recommended route R from the current location of the information display terminal 100 to the selected location, then even if the pile location is difficult to see, the worker 2 can easily select the pile and quickly see the route of movement from the current location to the selected pile.

[0110] Furthermore, a configuration was described in which the information display terminal 100 is equipped with a terminal storage unit 120 that stores a pile correspondence table 128 that associates pile identification information 128a and pile position information 128b, and reads an identifier RP1 linked to the pile identification information 128a from a pile installed at the measurement site, and corrects the position information of the information display terminal 100 with the position of the pile (pile position information 128b) corresponding to the pile identification information 128a. As a result, the position of the information display terminal 100 corresponds to the position information (or coordinate information) of the site, and the positional misalignment between the image captured by the terminal imaging unit 180 and the point cloud data image acquired by the surveying device 200 (i.e., the positional misalignment between the first image 31 and the second image 32) can be suppressed.

[0111] Furthermore, the terminal display unit 150 displays information about the measurement site located in a blind spot in the first image 31 using the second image 32. This allows the worker 2 to easily grasp information about the measurement site even if there are stakes or other objects at the measurement site that are difficult for the worker 2 to see from the information display terminal 100.

[0112] Furthermore, by displaying the second image 32 on the terminal display unit 150, which shows an image indicating a hazardous area, the worker 2 can be notified of areas requiring caution when passing through, areas where passage is prohibited, etc., thereby ensuring the worker 2's safety.

[0113] Furthermore, by configuring the information display terminal 100 to reflect drawing input from other devices that share the first image 31 and the second image 32 in the drawing content of the second image 32 on the terminal display unit 150, information can be easily transmitted from other devices located remotely to the worker 2. With this configuration, multiple users, such as multiple workers 2 and administrators, can cooperate to perform surveying and other tasks.

[0114] Furthermore, the surveying system 1 of this disclosure includes an information display terminal 100 and a surveying device 200 for measuring point clouds in three-dimensional space. The system includes a surveying information acquisition unit 121 that acquires surveying information, including point cloud data linked to positional information, from the surveying device 200; a range setting unit 122 that sets the display range of the point cloud; a division setting unit 123 that divides the display range into predetermined unit divisions; a point cloud amount calculation unit 124 that calculates the amount of point clouds contained in the space for each unit division; a point cloud amount display unit 125 that displays the amount of point clouds for each unit division calculated by the point cloud amount calculation unit for each division of the display range on the information display terminal; and a terminal position acquisition unit 160 that can acquire the position information of the information display terminal 100. The point cloud amount display unit 125 also combines the position information acquired by the terminal position acquisition unit 160 with local coordinates including the display range of the point cloud, and can display the position of the information display terminal 100 on the local coordinates on the information display terminal 100. This allows for easy confirmation of the location of the information display terminal 100, as well as whether the required number of points in the three-dimensional point cloud data has been acquired in each section of the display range where the point cloud data should be managed. This enables efficient scanning on-site and allows for visual confirmation of the scanning status and data acquisition results.

[0115] In particular, the point cloud quantity display unit 125 can easily align the position information of the information display terminal 100 with the local coordinates by performing alignment between the information display terminal 100 and the local coordinates in response to user operation when the information display terminal 100 is positioned on a known point on the local coordinates.

[0116] Furthermore, the point cloud quantity display unit 125 displays unit divisions overlaid on the image captured by the terminal imaging unit 180, or displays the position of the surveying device 200, based on the position of the information display terminal 100 on the local coordinate system and the orientation of the imaging direction of the terminal imaging unit 180, thereby allowing the worker 2 to more easily grasp the situation at the survey site.

[0117] Furthermore, by allowing the measurement range of the point cloud to be specified via the terminal input unit 140 to the surveying device 200, which displays the position overlaid on the image, scanning can be performed more efficiently on-site.

[0118] Furthermore, since the surveying device 200 can measure point clouds only within the display range set by the range setting unit 122, it is possible to reduce the measurement of unnecessary point clouds, thereby making on-site scanning even more efficient.

[0119] Furthermore, by providing a point cloud quantity management unit 126 that displays information indicating a point cloud shortage on an information display terminal if there is a section within the display range where the point cloud quantity does not reach a predetermined required amount, the operator 2 can perform the point cloud acquisition work without any omissions.

[0120] Furthermore, the point cloud quantity calculation unit 124 calculates the amount of point cloud data contained within a predetermined elevation range for each unit category, and the point cloud quantity display unit 125 displays the amount of point cloud data contained within the predetermined elevation range for each category on the information display terminal 100, thereby allowing the user to confirm the acquisition status of point cloud data within a desired elevation range.

[0121] Furthermore, the point cloud quantity calculation unit 124 calculates an index regarding the sufficiency of the required point cloud quantity for the display range or unit division, and the point cloud quantity display unit 125 displays the sufficiency index on the information display terminal 100, thereby allowing for quantitative confirmation of the point cloud acquisition status.

[0122] Furthermore, the surveying information acquisition unit 121 acquires the position information of the surveying device 200, the point cloud quantity calculation unit 124 calculates the range and predicted point cloud quantity that can be acquired when a point cloud survey is performed at that position based on the position of the surveying device 200, and the point cloud quantity display unit 125 displays the predicted values ​​for each category on the information display terminal. This allows the worker 2 to understand how a point cloud can be acquired when the surveying device 200 is installed at that position, and enables efficient progress in the point cloud acquisition work.

[0123] Furthermore, when calculating predicted values, the point cloud quantity calculation unit 124 calculates an added value for the overlapping categories within the predicted range where the point cloud quantity has already been calculated based on survey information and is considered an existing value. The point cloud quantity display unit 125 then displays the added value for each category on the information display terminal 100. This allows users to understand how point clouds can be acquired when the surveying device 200 is installed at that location, based on past measurement results, and enables efficient point cloud acquisition.

[0124] Furthermore, the point cloud quantity calculation unit 124 can reduce unnecessary data and streamline the processing of point cloud data by selecting only point cloud data that meets predetermined conditions based on measurement distance, incident angle, etc., from the point cloud data acquired by the survey information acquisition unit 121 as the target for point cloud quantity calculation.

[0125] Furthermore, the point cloud quantity management unit 126 calculates the accuracy of the instrument placement of the surveying device 200 based on point cloud data measured from multiple installation locations, and displays information regarding the accuracy of the instrument placement on the information display terminal 100. This allows for easy confirmation and correction of the quality of the instrument placement, thereby further improving the efficiency of scanning.

[0126] This concludes the description of the embodiments of the present disclosure, but the embodiments of the present disclosure are not limited to these embodiments.

[0127] For example, in the above embodiment, the scanner unit 260 of the surveying device 200 is a laser scanner, but the scanner unit that performs measurements to acquire three-dimensional point cloud data is not limited to this. For example, a LIDAR (Light Detection and Ranging) that measures distance by measuring scattered light from pulsed laser irradiation may be used as the scanner unit. Alternatively, the scanner unit may be equipped with an imaging unit such as a camera, and the point cloud data generation unit may generate point cloud data from one or more images captured by the imaging unit using so-called SfM (Structure from Motion) or photogrammetry techniques.

[0128] Furthermore, although the above embodiment shows an example of displaying the unit divisions and the point cloud quantities of each unit division on the terminal display unit 150 of the information display terminal 100, the information may also be displayed on other display units. For example, the unit divisions and information of each unit division may be transmitted to a surveying device, and the surveying instrument display unit of the surveying device may be used as the information terminal display unit of this disclosure to display the unit divisions and the point cloud quantities of each unit division.

[0129] Furthermore, in the processing flow of the above embodiment, the AR display-related processing from S108 to S110 is performed after the point cloud quantity display-related processing from steps S103 to S107 in Figure 3, but the order of processing is not limited to this. For example, the processing from S108 to S110 may be performed first. In other words, the point cloud may be measured by the surveying device via remote operation based on AR display on the information display terminal (processing from steps S108 to S110), and the point cloud quantity may be displayed based on the three-dimensional point cloud data acquired by the measurement (processing from steps S103 to S107).

[0130] Furthermore, the processing in steps S101 and S102 may be performed not only before the processing in step S103, but also at a timing between the processing in steps S107 and S108.

[0131] Furthermore, the manner in which the first image and the second image are displayed on the terminal display unit may include the manner in which the first image and the second image are displayed at corresponding (or related) positions on the terminal display unit 150. The manner in which the first image and the second image are displayed may, for example, be displayed simultaneously on the screen of the terminal display unit 150, or the first image and the second image may be displayed individually with the ability to switch between them as needed, or the first image and the second image may be displayed overlapping or separately within the terminal display unit 150. [Explanation of Symbols]

[0132] 1. Surveying System 2 Workers 31 First Image 32 Second image 100 Information display terminals 110 Terminal Processing Unit 120 Terminal Storage Unit 121 Survey information acquisition section 122 Range setting section 123 Classification section 124 Point cloud amount calculation section 125 Point cloud amount display section 126 Point cloud quantity management department 127 Survey Support Department 128 Stake Compatible Table 128a Stake Identification Information 128b Pile location information 130 Terminal Communication Unit 140 Terminal Input Section 150 Terminal display unit 160 Terminal location acquisition unit 170 Direction acquisition part 180 Terminal imaging unit 200 Surveying equipment 210 Surveying Instrument Processing Unit 220 Survey instrument storage section 221 Point Cloud Data Generation Unit 222 Measurement position calculation unit 223 Display Control Unit 230 Surveying Instruments Communication Department 240 Survey instrument operation section 250 Survey instrument display section 260 Scanner Unit 261 Ranging section 262 Deflection section 270 Survey instrument position acquisition part 281 Attitude drive unit 282 Attitude detection unit 290 Surveying instrument imaging unit 311 Embankment 321 Stake Point Icon Image 321a Stake Point Icon Image 321b Stake Point Icon Image 322 Image DI direction information DA display range Da first point cloud data Db Second point cloud data Dc 3rd point cloud data GU Unit Classification GU1 Unit Classification GU2 Unit Classification GU3 Unit Classification GU4 Unit Classification GU5 Unit Classification PSP installation position R Recommended Route RP reference point RP1 identifier SP surveying device position T terminal position V Mobile Unit

Claims

1. A surveying support system comprising an information display terminal and a surveying device for measuring point clouds in three-dimensional space, A terminal display unit that displays a first image of the measurement site captured from the location of the information display terminal and a second image generated from information about the measurement site, A surveying support unit that uses the terminal display unit to assist in measurements by the surveying device, The information display terminal is equipped with a point cloud quantity display unit that displays the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal as the second image on the information display terminal. The aforementioned information display terminal displays a stake point icon image as the second image at the actual stake location, accepts the selection of the stake point icon image, and presents a recommended route from the current location of the information display terminal to the selected location as a surveying support system.

2. It is equipped with a storage unit that stores a pile correspondence table that associates pile identification information and pile location information, The information display terminal reads an identifier associated with the stake identification information from a stake installed at the measurement site, and corrects the location information of the information display terminal based on the location of the stake corresponding to the stake identification information. The surveying support system according to claim 1.

3. A surveying support system comprising an information display terminal and a surveying device for measuring a point cloud in three-dimensional space, A terminal display unit that displays a first image of the measurement site captured from the location of the information display terminal and a second image generated from information about the measurement site, A surveying support unit that uses the terminal display unit to assist in measurements by the surveying device, The information display terminal is equipped with a point cloud quantity display unit that displays the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal as the second image on the information display terminal. The terminal display unit is a surveying support system that displays information about the area to be measured, which is located in a blind spot in the first image, using the second image.

4. A surveying support system comprising an information display terminal and a surveying device for measuring a point cloud in three-dimensional space, A terminal display unit that displays a first image of the measurement site captured from the location of the information display terminal and a second image generated from information about the measurement site, A surveying support unit that uses the terminal display unit to assist in measurements by the surveying device, The information display terminal is equipped with a point cloud quantity display unit that displays the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal as the second image on the information display terminal. The second image above is a surveying support system that includes an image indicating a hazardous area.

5. A surveying support system comprising an information display terminal and a surveying device for measuring a point cloud in three-dimensional space, A terminal display unit that displays a first image of the measurement site captured from the location of the information display terminal and a second image generated from information about the measurement site, A surveying support unit that uses the terminal display unit to assist in measurements by the surveying device, The information display terminal is equipped with a point cloud quantity display unit that displays the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal as the second image on the information display terminal. The aforementioned information display terminal is a surveying support system that reflects drawing input from other devices that share the first image and the second image onto the drawing content of the second image on the terminal display unit.

6. A terminal display unit that displays a first image of the measurement site captured by the terminal imaging unit and a second image generated from information about the measurement site, A surveying support unit that uses the aforementioned terminal display unit to assist in the measurement of a surveying device that measures point clouds in three-dimensional space, The terminal includes a point cloud quantity display unit that displays the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the terminal imaging unit as the second image on the terminal display unit. The surveying support unit is an information display terminal that uses the terminal display unit to display a stake point icon image as the second image at the actual stake location, accepts the selection of the stake point icon image, and presents a recommended route from the current location to the selected location.

7. A surveying support method in a surveying support system comprising an information display terminal and a surveying device for measuring point clouds in three-dimensional space, The steps include displaying a first image of the measurement site captured from the location of the information display terminal and a second image generated from information about the measurement site on the terminal display unit, The steps include: using the terminal display unit to support the measurement by the surveying device with the surveying support unit; The step of using the point cloud quantity display unit to display the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal as the second image on the information display terminal, The information display terminal displays a stake point icon image as the second image at the actual stake location, accepts the selection of the stake point icon image, and presents a recommended route from the current location of the information display terminal to the selected location. A surveying support method that uses a computer to perform the surveying.

8. A surveying support program in a surveying support system comprising an information display terminal and a surveying device for measuring point clouds in three-dimensional space, The steps include displaying a first image of the measurement site captured from the location of the information display terminal and a second image generated from information about the measurement site on the terminal display unit, The steps include: using the terminal display unit to support the measurement by the surveying device with the surveying support unit; The step of using the point cloud quantity display unit to display the point cloud quantity for each unit division obtained by dividing the display range of survey information within the imaging range of the information display terminal as the second image on the information display terminal, The information display terminal displays a stake point icon image as the second image at the actual stake location, accepts the selection of the stake point icon image, and presents a recommended route from the current location of the information display terminal to the selected location. A surveying support program that allows a computer to perform the following actions.

Images