Target direction acquisition device, control system, method and program for pointing a measuring device in the direction of a target

Abstract

The present invention relates to a target direction acquisition device, a control system, a method of directing a measurement device to the direction of a target, and a program. A technique is obtained by which the direction of a target can be obtained on the side of a measurement device without requiring very new hardware. A method of directing a laser scanner as a measurement device to the direction of a reflection prism as a target has: a step of obtaining a captured image in which the laser scanner is captured from the side of the reflection prism using a smartphone; a step of acquiring the direction of the reflection prism as viewed from the laser scanner on the basis of the captured image; and a step of rotating the laser scanner on the basis of the direction of the reflection prism as viewed from the laser scanner so that the reflection prism is directly opposite.

Description

Technical Field

[0001] This invention relates to the control of a measuring device. Background Technology

[0002] Sometimes it is necessary to determine the coordinates of a specific location at the measurement site. In this case, a target (e.g., a reflecting prism) is placed at that location, and the target is positioned using a measuring device. Here, there are methods where the measuring device operates automatically, and the operator moves the target by hand while simultaneously positioning it.

[0003] In this method, there are cases where the measuring device needs to be turned (pointed) towards the target. The techniques corresponding to this problem are known from Japanese Patent Nos. 4648025, 4177765, and 4177784. Summary of the Invention

[0004] The technologies described in Japanese Patent Nos. 4648025, 4177765, and 4177784 utilize dedicated hardware, thus increasing costs. Against this backdrop, the object of the present invention is to provide a technology that allows the orientation of a target to be obtained from one side of a measuring device without requiring very new hardware.

[0005] The present invention is a target orientation acquisition device for obtaining the direction of a target as observed from a measuring device. It is a target orientation acquisition device based on obtaining the direction of the target as observed from the measuring device by taking an image of the target from one side of the target.

[0006] In this invention, the measuring device is described in a manner that includes a display for identifying the orientation of the measuring device. In this invention, the orientation of the target observed from the measuring device is obtained based on the orientation of the measuring device in the captured image and the direction of the optical axis of the captured image.

[0007] The present invention can also be implemented as a control system, the control system comprising: a target orientation acquisition device; and a control device that performs the following control: based on the target orientation observed from the measuring device, rotates the measuring device to point the measuring device toward the target orientation.

[0008] The present invention can also be used as a method for pointing a measuring device toward a target, the method comprising: obtaining an image captured by the measuring device from one side of the target; obtaining, based on the captured image, the direction of the target as observed from the measuring device; and rotating the measuring device, based on the direction of the target as observed from the measuring device, so that the measuring device points toward the target.

[0009] The present invention can also be used as a method for pointing a measuring device toward a target, the method comprising: obtaining an image captured by the measuring device from one side of the target using a terminal equipped with a camera; sending the captured image to a processing server using the communication function of the terminal equipped with the camera; obtaining, in the processing server, the direction of the target observed from the measuring device based on the captured image; and rotating the measuring device based on the direction, so that the measuring device points toward the target.

[0010] The present invention can also be understood as a program for causing a computer to execute control to direct the measuring device toward the target. The program causes the computer to perform the following steps: obtaining an image of the measuring device taken from one side of the target; obtaining the direction of the target as observed from the measuring device based on the image; and rotating the measuring device to direct the measuring device toward the target based on the direction of the target observed from the measuring device.

[0011] According to the present invention, it is possible to obtain the direction of the target from one side of the measuring device without requiring very new hardware. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the implementation method.

[0013] Figure 2 It is a model diagram showing the relationship of angles.

[0014] Figure 3 The accompanying photograph is a substitute photograph illustrating an example of an image taken by a laser scanner.

[0015] Figure 4 The accompanying photograph is a substitute photograph illustrating an example of an image taken by a laser scanner.

[0016] Figure 5 This is a block diagram of a laser scanner.

[0017] Figure 6 This is a block diagram for handling the server.

[0018] Figure 7This is a flowchart illustrating an example of the processing sequence. Detailed Implementation

[0019] 1. Summary exist Figure 1 The outline of the implementation is shown below. In this technique, a laser scanner 100, which serves as a measuring device, is pointed in the direction of a reflecting prism 400, which serves as a target. First, an image of the laser scanner 100 is captured by a smartphone 200 from one side of the reflecting prism 400. Furthermore, based on this image, the direction of the reflecting prism 400 as observed from the laser scanner 100 is obtained, and based on this direction, the laser scanner 100 is rotated so that it points towards the reflecting prism 400.

[0020] exist Figure 1 The image shows a laser scanner 100 as a measuring device. This example illustrates the use of the laser scanner 100 to obtain a group of laser scan points. As a measuring device, a total station or a total station equipped with a laser scanner can be used.

[0021] exist Figure 1 The diagram shows a measuring target, or reflecting prism 400, that can be held and moved by the operator 300. The reflecting prism 400 is a measuring reflecting prism that reflects incident light in a direction that is reversed by 180°.

[0022] In this example, the smartphone 200 is mounted on a support rod 410 supporting the reflecting prism 400, and the relative positional relationship between the reflecting prism 400 and the smartphone 200 is fixed. The distance between the reflecting prism 400 and the smartphone 200 is minimized. In particular, the distance in the left-right direction (horizontal direction) orthogonal to the direction of the laser scanner 100 observed from the reflecting prism 400 is 50 cm or less, preferably 30 cm or less. The smartphone 200 is a commercially available ordinary smartphone with data communication functions utilizing a camera, a wireless LAN device, and a network line.

[0023] In this example, the smartphone 200 is used as the operating terminal (controller) of the laser scanner 100. Alternatively, a tablet PC or a dedicated terminal could also be used as the operating terminal.

[0024] Operator 300 sets up a reflecting prism 400 at the location where the laser scanner 100 will be used for positioning. At this time, operator 300 uses the camera function of smartphone 200 to take a picture of the laser scanner 100. The image data of the laser scanner 100 is sent to processing server 500. The transmission of image data from smartphone 200 to processing server 500 is performed using a network line.

[0025] Alternatively, the operator 300 can take pictures of the laser scanner 100 while holding the smartphone 200. In this case, the picture is taken when the distance between the smartphone 200 and the reflecting prism 400 in the horizontal direction orthogonal to the line connecting the reflecting prism 400 and the laser scanner 100 is 50cm or less, preferably 30cm or less. Furthermore, when taking pictures, the laser scanner 100 is aimed at while the image of the laser scanner 100 displayed on the smartphone 200 screen is magnified as much as possible.

[0026] The processing server 500 uses artificial intelligence (AI) estimation function to estimate the orientation of the smartphone 200 (reflecting prism 400) based on the image of the laser scanner 100 captured by the smartphone 200.

[0027] Figure 2 This is a model diagram viewed from a vertically upward viewpoint. Additionally, strictly speaking, the positions of the smartphone 200 and the reflecting prism 400 are offset, but they are considered the same here.

[0028] exist Figure 2 In the diagram, vector V1 shows the orientation of the laser scanner 100 as observed from the horizontal direction of the smartphone 200 (reflecting prism 400). Vector V1 aligns with the direction of the optical axis of the smartphone 200. Furthermore, vector V2 shows the orientation of the front of the laser scanner 100.

[0029] The AI ​​infers how the laser scanner 100 appears in an image captured by the smartphone 200, thereby obtaining the direction of V2 relative to the depth direction (direction of vector V1) of the captured image. Specifically, the angle θ formed by the direction of the laser scanner 100 as observed from the reflecting prism 400 (vector V1) and the direction of the front of the laser scanner 100 (vector V2) is inferred.

[0030] If the relationship between vectors V1 and V2 can be determined, then the orientation of the reflecting prism 400 as observed from the front side (vector 2) of the laser scanner 100 can be determined.

[0031] exist Figure 3 The image shown is an illustration of a laser scanner 100 captured using a smartphone 200. Figure 3 For ease of understanding, the laser scanner 100 is shown as viewed from a slightly obliquely upward position (strictly speaking, the upper surface of the laser scanner 100 is not visible when viewed from a horizontal direction).

[0032] like Figure 3As shown, an arrow 105 is displayed on the side of the laser scanner 100 as a mark indicating the facing direction, and a △ symbol 106 is displayed on the front as a mark indicating that it is facing forward.

[0033] exist Figure 3 In this case, the depth direction of the paper is the direction of the optical axis of the camera of the smartphone 200 (the direction of vector V1). Figure 3 In this case, when viewed from the laser scanner 100, the smartphone 200 can be observed in a slightly right direction (approximately the 1 o'clock position). Its state can be quantitatively estimated using an AI estimation model. Specifically, based on... Figure 3 The captured image, with estimated vector V1 (reference). Figure 2 The quantitative relationship between the vector V2 and the vector V2 is obtained. From this, the following information is obtained: From the perspective of the laser scanner 100, with the front as the reference, in which direction can the reflecting prism 400 be seen?

[0034] For example, in Figure 3 In this case, the following information can be inferred: When viewed from the laser scanner 100, the smartphone 200 (reflecting prism 400) can be seen at a direction 30° to the right.

[0035] exist Figure 4 The image shown is another example of an image captured by a laser scanner 100 using a smartphone 200. In this case, the smartphone 200 can be seen from the laser scanner 100 in a rear-right direction (approximately at the 5 o'clock position). Its state is quantitatively estimated using an AI estimation model.

[0036] Specifically, according to Figure 4 The image was calculated using an AI estimation model. Figure 2 The angle θ in the image. In this case, for example, the orientation angle of the smartphone 200 (reflecting prism 400) as observed from the laser scanner 100 is estimated to be 150° to the right.

[0037] In addition, such as Figure 4 As shown, a ◇ symbol 107 indicating the back is displayed on the back of the laser scanner 100. By detecting this display, it can be identified that the image of the laser scanner 100 was taken from the back.

[0038] If you know Figure 2 In the case of θ, the laser scanner 100 is rotated so that θ = 0, thereby enabling the front of the laser scanner 100 to face the smartphone 200 (reflective prism 400).

[0039] In this case, the laser scanner 100 has remote operation capabilities using a wireless LAN. In this example, a control signal is generated by the processing server 500 to cause the laser scanner 100 to perform a horizontal rotation such that θ = 0, and this signal is sent to the smartphone 200. Furthermore, this control signal is sent from the smartphone 200 to the laser scanner 100, causing the laser scanner 100 to rotate horizontally such that θ = 0. Thus, the laser scanner 100 points in the direction of the reflecting prism 400.

[0040] 2. Hardware Structure like Figure 1 As shown, the laser scanner 100 includes a tripod 101, a base 102 supported on the tripod 101, a main body 103 held on the base 102 in a horizontally rotatable state, and an optical system 104 disposed on the main body 103 and capable of vertical rotation.

[0041] While the optical system 104 is rotated vertically, a laser for ranging is emitted from the optical system 104, thereby performing a laser scan of the vertical plane. In addition, the main body 103 is rotated horizontally at the same time, thereby performing a laser scan of the celestial sphere.

[0042] Figure 5 This is a block diagram of a laser scanner 100. The laser scanner 100 includes a laser scanner unit 111, a horizontal rotation control unit 112, and a communication device 113.

[0043] The laser scanner unit 111 performs the control and processing of the laser scanning described above. The horizontal rotation control unit 112 controls the horizontal rotation of the main body unit 103 relative to the base 102.

[0044] The communication device 113 communicates with external devices. Here, communication is conducted using a wireless LAN standard. In this example, communication device 113 is used for communication with smartphone 200. The laser scanner 100 can be remotely operated using the communication device 113. For example, a dedicated terminal, PC, tablet, or smartphone can be used as an operating terminal (controller) to remotely operate the laser scanner (measuring device) 100 using the communication device 113.

[0045] In this example, the remote operation described above can be used to start and stop laser scanning, select various laser scanning modes, and utilize... Figure 7 The processing execution of the target pointing instruction.

[0046] Figure 6This is a block diagram of the processing server 500. The processing server 500 includes an image data acquisition unit 501, a target direction estimation unit 502, and a control signal generation unit 503. The processing server 500 is a computer, equipped with a CPU, memory, hard disk drive, and various input / output interfaces.

[0047] The computer constituting the processing server 500 is equipped with a device for implementing... Figure 6 The application software program for the functional units shown. Figure 6 The processing server 500 shown is implemented in software. It can also be comprised of a portion or all of dedicated electronic circuitry. For example, an FPGA can also be used to construct a portion or all of the processing server 500.

[0048] The image data acquisition unit 501 acquires image data from images captured by the camera of the smartphone 200. The target direction estimation unit 502, based on the images captured by the laser scanner 100 acquired by the image data acquisition unit 501, uses an AI estimation model obtained from deep learning to estimate the direction from which the reflecting prism 400 is visible, with the front of the laser scanner 100 as a reference. Specifically, it makes... Figure 2 The vector 1 in the image has been reversed by 180° to the direction of the reflecting prism 400 as observed from the laser scanner 100.

[0049] The control signal generation unit 503 generates a signal for making Figure 2 The control signal that rotates the laser scanner 100 so that the front of the laser scanner 100 points towards the direction of the reflecting prism 400 is such that the angle θ becomes θ = 0.

[0050] 3. AI estimation model The AI ​​estimation model used in the target direction estimation unit 502 will be explained below.

[0051] The AI ​​estimation model takes the image captured by the laser scanner 100 and taken by the smartphone 200 as input, and... Figure 2 The angle θ in the image is used as the output. The AI ​​estimation model obtains an estimation based on the input image using deep learning within a learning network with AI estimation capabilities. Figure 2 It is obtained by the function of θ in the equation.

[0052] Learning networks are algorithms that deepen the layers of neural networks, enabling deep learning. There are no particular limitations on learning networks.

[0053] (The creation of AI estimation models) Sample images of the laser scanner were taken from various horizontal angles. The value of angle θ in these sample images is known and serves as teaching data.

[0054] For example, prepare 1000 sample images. For all of these 1000 sample images, prepare θ. Furthermore, divide these 1000 sample images into 300 samples (Group A: learning data group) and 700 samples (Group B: test data group).

[0055] First, using samples from group A, the relationship between the input image and θ is learned to obtain the AI ​​inference model. Then, the backpropagation algorithm is used for further learning.

[0056] In the backpropagation algorithm, the difference between the obtained θ (estimated value) and the corresponding forward θ (forward solution value) is calculated. The weights of the neural network are then adjusted, i.e., the learning network is adjusted, to minimize this difference. Then, the model is evaluated using data from group B.

[0057] The combination of groups A and B is randomly varied and the above process is repeated multiple times. Using this process, deep learning is performed to obtain inferences from the images captured by the laser scanner 100. Figure 2 The AI ​​estimation model for θ in the target direction estimation unit 502 is used. Using this AI estimation model, θ in the target direction estimation unit 502 (reference) is calculated. Figure 2 The presumption of ).

[0058] 4. An example of the processing order The following illustrates an example of the processing sequence performed by the processing server 500. Figure 7 The diagram shows an example of a flowchart used to perform a process. (Used for execution) Figure 7 The processing program is stored in the storage device (semiconductor memory or hard disk device) of the computer constituting the processing server 500 and executed by the CPU of the computer. Alternatively, the program can be stored on a suitable storage medium.

[0059] Here, the smartphone 200 and the reflecting prism 400 are considered to be in the same position. Furthermore, application software for implementing the function of the laser scanner 100 as a controller is installed on the smartphone 200, enabling remote operation of the laser scanner 100 using the smartphone 200.

[0060] First, assume the following situation: Operator 300 is at a location where he needs to perform a measurement while keeping the reflecting prism 400 in a certain state, and in this state, he needs to point the laser scanner 100 toward himself (the direction of the reflecting prism 400).

[0061] In this state, the operator 300 directs the camera of the smartphone 200 toward the laser scanner 100 to take a picture of the laser scanner 100. At this time, the image on the screen is magnified as much as possible, and the image of the laser scanner 100 is captured in the center of the smartphone 200 screen.

[0062] After the shooting is completed, the operator 300 operates the smartphone 200 to begin the process of instructing the laser scanner 200 to point in the direction of the viewpoint (the position of the smartphone 200) at the time of shooting.

[0063] When the above operation is performed, firstly, the image data obtained by photographing the laser scanner 100 is sent from the smartphone 200 to the processing server 500, whereby the processing server 500 retrieves the image data (step S101). This processing is performed by... Figure 6 The image data acquisition unit 501 performs this process.

[0064] Next, based on the image obtained in step S101, the orientation of the smartphone 200 (reflecting prism 400) in the horizontal direction as observed from the laser scanner 100 is estimated (step S102). This process is performed by... Figure 6 The target orientation estimation unit 502 in the middle performs the process. Based on this process, the target orientation is estimated. Figure 2 θ in the equation.

[0065] Next, a control signal is generated to rotate the main body 103 horizontally so that θ = 0, i.e., the front of the laser scanner 100 faces the smartphone 200 (reflective prism 400) (step S103). Figure 6 The control signal generation unit 503 performs this processing.

[0066] The control signal is sent to the smartphone 200. The smartphone 200 is connected to the laser scanner 100 via a wireless LAN line. The smartphone 200 can remotely operate the laser scanner 100 using the wireless LAN line. Using this remote operation function, the aforementioned horizontal rotation control signal is sent from the smartphone 200 to the laser scanner 100 via the wireless LAN line. The laser scanner 100, having received the control signal, then... Figure 2 The horizontal rotation θ⇒0 is used to make the front face the smartphone 200 (reflective prism 400).

[0067] In this state, laser scanning is performed within a specific range (e.g., a horizontal angle of ±5°) to measure the position of the reflecting prism 400.

[0068] 5. Advantages If the laser scanner 100 can be operated remotely, then the user only needs a smartphone 200 with a camera function to create a system that can point the laser scanner 100 at the reflecting prism 400, which serves as a target. A processing server 500 is necessary, but this can be addressed by preparing software and utilizing existing hardware. 6. Other The orientation of the laser scanner 100 in the captured image can also be calculated using conventional image analysis. In this case, the orientation of the front of the laser scanner 100 can be determined from the captured image, and calculated based on the extension direction of the edge or surface of the main body 103. Figure 2 The angle θ in the equation can also be used in conjunction with estimations made using AI.

[0069] Alternatively, a PC can be used to perform the functions of the server 500, and the operator 300 can carry the PC. Furthermore, a tablet PC with a camera can be used instead of a smartphone 200. In this case, if the tablet PC has sufficient computing power, it can also be used to perform the functions of the server 500.

[0070] Alternatively, the laser scanner 100 may be configured to have the functionality of a processing server 500. In this case, the laser scanner 100 includes a computer with the functionality of a processing server 500. Alternatively, it may be possible to prepare a PC with the functionality of a processing server 500 and execute the functionality of the processing server 500 on that PC.

[0071] Alternatively, the front and back can be distinguished based on the shape of the laser scanner 100. In this case, the laser scanner 100 needs to have a shape that distinguishes between the front and back.

[0072] There are also cases where the laser scanner is indistinguishable from the front to the back. In this case, there is no distinction between the front and the back, so the laser scanner is rotated so that either the front or the back of the laser scanner is facing the smartphone 200 (reflecting prism 400).

[0073] In the method of operating the laser scanner 100 using a dedicated wireless controller, the following process can also be performed: taking a picture with the laser scanner 100 using a smartphone 200 ⇒ processing in the server 500 Figure 2 The estimation of angle θ in the middle ⇒ The display of angle θ to smartphone 200 ⇒ The operation of the controller by operator 300 who observes the display ⇒ The horizontal rotation of laser scanner 100 so that θ = 0.

Claims

1. A control system for acquiring the orientation of a target as observed from a measuring device, comprising: A target orientation acquisition device acquires the target orientation as observed from the measuring device by capturing an image taken from one side of the target by the measuring device; and The control device performs the following control: based on the direction of the target observed from the measuring device, it rotates the measuring device so that the measuring device points in the direction of the target.

2. The control system according to claim 1, wherein, The measuring device has an identification display for identifying the orientation of the measuring device.

3. The control system according to claim 1 or 2, wherein, The orientation of the target observed from the measuring device is obtained based on the orientation of the measuring device in the captured image and the orientation of the optical axis of the captured image.

4. A method for pointing a measuring device toward a target, comprising: The step of obtaining an image captured by the measuring device from one side of the target; The step of obtaining the orientation of the target as observed from the measuring device based on the captured image; and The step of rotating the measuring device to point it toward the target, based on the direction of the target observed from the measuring device.

5. A method for pointing a measuring device toward a target, comprising: The step of obtaining an image captured by the measuring device from one side of the target using a terminal equipped with a camera; The step of using the communication function of the terminal with the camera to send the captured image to the processing server; In the processing server, the step of obtaining the orientation of the target as observed from the measuring device based on the captured image; and The step of rotating the measuring device based on the stated direction, so that the measuring device points towards the target.

6. A program product for causing a computer to perform control to orient a measuring device toward a target, comprising a program for causing the computer to perform the following steps: The step of obtaining an image captured by the measuring device from one side of the target; The step of obtaining the orientation of the target as observed from the measuring device based on the captured image; and The step of rotating the measuring device to point it toward the target, based on the direction of the target observed from the measuring device.

Images