Guidance system, guidance system control method, and guidance system control program

Abstract

To provide a guidance system and the like capable of accurately indicating an object with a view point of an operator in displaying the object on a display unit.SOLUTION: A guidance system 1 comprises a measurement device 10 which measures an object 300 from a view point different from a view point of a user and a display unit 204 which displays the object from the view point of the user. The measurement device is constituted to measure a distance by irradiating the object with distance measurement light and receiving the distance measurement light reflected from the object, changes measurement information of the object obtained by measurement by the measurement device to the view point of the user, and displays it on the display unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a guidance system that provides guidance to an operator who operates a work machine or the like, a control method for the guidance system, and a control program for the guidance system. [Background technology]

[0002] Conventionally, there have been proposals to display reference information for guidance to assist an operator in operating a work machine or other device on a display or the like that the operator can see (for example, Patent Document 1, etc.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. WO2017 / 191853 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in this proposal, when reference information such as information about the operator's blind spot was displayed on the display, shapes viewed from various angles different from the operator's viewpoint were displayed as reference information, which caused a problem in that it was difficult for the operator to quickly grasp the situation, etc. Furthermore, when displaying external information captured by a camera on a display to enable the operator to grasp the external information, there was a problem that the camera image did not allow the operator to clearly grasp, for example, the depth of structures and other external information.

[0005] Therefore, an object of the present invention is to provide a guidance system, a control method for a guidance system, and a control program for a guidance system that can accurately display an object from the operator's viewpoint when displaying the object on a display unit. [Means for solving the problem]

[0006] According to the present invention, the above-mentioned object is achieved by a guidance system having a surveying instrument which surveys an object from a viewpoint different from that of a user, and a display unit which displays the object from the user's viewpoint, wherein the surveying instrument is configured to irradiate distance measuring light towards the object and measure distance by receiving reflected distance measuring light from the object, and the surveying information of the object surveyed by the surveying instrument is converted to the viewpoint of the user and displayed on the display unit.

[0007] According to the above configuration, the information of the object displayed on the display unit is measurement information (eg, three-dimensional shape information, etc.), so that shape information of the object is accurately displayed on the display unit. Furthermore, the measurement information of the object displayed on the display unit is displayed from the user's viewpoint, making it easy for the user to grasp the object.

[0008] Preferably, the guidance system has a surveying instrument side imaging device that images the object from the surveying viewpoint of the surveying instrument, and a user side imaging device that images the object from the viewpoint of the user, and is characterized in that the surveying information of the surveying instrument is converted to the viewpoint of the user based on relative position and orientation information between the surveying instrument side imaging device and the user side imaging device and displayed on the display unit.

[0009] According to the above configuration, the relative position and orientation information between the imaging device on the surveying device side and the imaging device on the user side, for example, a method such as Sfm (Structure from Motion) is used to restore the shape of the object shown in the images acquired by both imaging devices, and at the same time, based on the relative position and orientation information obtained by estimating the position and orientation of both imaging devices, the surveying information is converted to the user's viewpoint and displayed on the display unit. Therefore, the survey information of the surveying instrument can be accurately changed to the viewpoint of the user.

[0010] Preferably, the ranging light of the surveying instrument is irradiated only toward at least one of the object or its surroundings, and shape information of the object is generated by receiving the reflected ranging light from the object, and a method of irradiating the ranging light suitable for the generated shape information is selected and executed, thereby enabling three-dimensional information of the object to be obtained.

[0011] According to the above configuration, the surveying instrument generates shape information such as a circle, and then selects an irradiation method (e.g., a circular shape) appropriate for this shape information. This makes it possible to obtain three-dimensional information of the target object by irradiating distance measuring light only to the necessary parts, enabling quick and efficient surveying. In particular, in the above-described configuration, scanning for irradiating distance measuring light can be performed uniformly within a desired range, improving the reliability of the acquired information.

[0012] Preferably, the guidance system is characterized in that it acquires target position information to which the object is to be moved, and displays difference information between current position information and the target position information on the display unit from the user's viewpoint.

[0013] According to the above configuration, difference information between the current position information and the target position information (e.g., the direction to move, the inclination, etc.) can be displayed on the display unit from the user's viewpoint, thereby providing the user with appropriate guidance information.

[0014] According to the present invention, the above-mentioned object is achieved by a control method for a guidance system having a surveying instrument that surveys an object from a viewpoint different from that of a user, and a display unit that displays the object from the user's viewpoint, wherein the surveying instrument irradiates distance measuring light towards the object, receives distance measuring light reflected from the object to measure the distance, and converts the surveying information of the object surveyed by the surveying instrument to the viewpoint of the user and displays it on the display unit.

[0015] According to the present invention, the above object is achieved by a control program of a guidance system for performing a survey device that surveys an object from a different viewpoint from the user's side, and a display unit that displays the object from the user's side. The survey device is achieved by a control program of a guidance system for performing a function of irradiating the object with distance measurement light, receiving reflected distance measurement light from the object, and for changing the survey information of the object surveyed by the survey device to the viewpoint of the user's side and displaying it on the display unit. Effect of the Invention

[0016] The present invention has the advantage of being able to provide a guidance system, a control method for a guidance system, and a control program for a guidance system that can accurately display an object from the operator's viewpoint when displaying the object on a display unit. [Brief description of the drawings]

[0017] [Figure 1] 1 is a schematic diagram showing a guidance system 1 according to an embodiment of the present invention. [Diagram 2] 2 is a schematic diagram showing a main internal structure of a housing 7 of the surveying instrument 10 of FIG. [Diagram 3] FIG. 2 is a schematic block diagram showing the main configuration of a crane 200 in FIG. 1. [Figure 4] 3 is a schematic block diagram showing the main configuration of the surveying instrument 10 shown in FIGS. 1 and 2. FIG. [Diagram 5] 2 is a schematic block diagram showing the main configuration of a first various information storage unit 110 on the surveying instrument side. FIG. [Figure 6] FIG. 4 is a schematic block diagram showing the main configuration of a second various information storage unit on the surveying instrument side. [Figure 7] FIG. 11 is a schematic block diagram showing the main configuration of a third various information storage unit on the surveying instrument side. [Figure 8] 4 is a main flowchart showing a guidance preparation process of the guidance system 1 of the present embodiment. [Figure 9] 4 is a main flowchart showing a guidance process according to the present embodiment. [Figure 10] 13 is an example of a screen showing information such as the displacement of the reinforcing bar 300. [Figure 11] 13 is an example of a screen showing the relationship between a reinforcing bar 300 and a target position. [Figure 12] 13 is another example of a screen showing the relationship between the reinforcing bar 300 and the target position. [Figure 13] 13 is an example of a screen showing the degree of inclination of a reinforcing bar 300 with an arrow. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments described below are preferred specific examples of the present invention, and therefore various technically preferable limitations are imposed thereon. However, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited thereto.

[0019] (Overall configuration of guidance system 1) FIG. 1 is a schematic diagram showing a guidance system 1 according to an embodiment of the present invention. The guidance system 1 in Figure 1 includes, for example, a crane 200 for suspending an object, such as a reinforcing bar 300, and moving it to a target position, a crane side display 204 which is a display unit installed in the operator's seat 210 of the crane 200, and a user side imaging device, such as a crane camera 203 which images the outside, particularly the reinforcing bar 300 and its surroundings, from the viewpoint of a user, such as an operator, who sits on the operator's seat 210 and operates the crane.

[0020] This crane side display 204 is a "touch panel type display." A touch panel is an electronic component that combines a display unit, for example, a display, with a position input device, and is an input device that allows the operator to input various information by touching the display on the display.

[0021] In addition, the present system 1 surveys the reinforcing bar 300 suspended by the crane 200 from a direction different from that of the crane 200, and also includes a surveying instrument 10 equipped with a surveying instrument side imaging device, for example, a surveying instrument camera 27 (imaging unit) (see Figure 4, etc.), which images the reinforcing bar 300 and its surroundings.

[0022] The crane 200 in Figure 1 has a crane side communication device 202 (see Figure 3) for communicating with the surveying instrument 10 etc., and the surveying instrument 10 also has a surveying instrument side communication device 101 for transmitting to the crane 200 etc., and these are configured to be able to communicate with each other.

[0023] (Main hardware configuration of surveying device 10, etc.) Next, the main hardware configuration of the "surveying instrument 10" shown in FIG. 1 will be described. As shown in FIG. 1, the surveying device 10 is mounted on a tripod 2 installed at an installation reference point R via a base unit 3, and is configured to be capable of three-dimensional measurement (surveying) of a measurement object, such as a reinforcing bar 300.

[0024] The base unit 3 of the surveying instrument 10 has a protractor 8 that rotates horizontally and can detect the rotation angle, and also has a vertical rotation part 9 that can rotate vertically and can be fixed at a desired angle, and the surveying instrument 10 is configured to be directly attached to this vertical rotation part 9. For this reason, the surveying instrument 10 is configured to rotate vertically around the mechanical reference point, and also to rotate horizontally around the mechanical reference point.

[0025] (Major hardware configuration in the housing 10 of the surveying device 20) FIG. 2 is a schematic diagram showing the main internal structure of the housing 7 of the surveying instrument 10 of FIG. As shown in Figure 2, the rear surface of the housing 7 has a surveying instrument side display unit 11 and an operation unit 12, and inside the housing 7 there are a measurement unit 20 having a ranging optical axis 4, a calculation processing unit 24, an emission direction detection unit 25 that detects the emission direction of the ranging light, an attitude detection unit 26 that detects the horizontal inclination of the surveying instrument 10, an imaging unit (surveying instrument camera) 27 having an imaging optical axis 5, and an optical axis deflection unit 36 ​​that deflects the ranging optical axis 4.

[0026] As shown in FIG. 2, the measurement unit 20, the attitude detection unit 26, the imaging unit 26, and the optical axis deflection unit 36 ​​are integrally arranged. 2 includes a distance measurement light emitting section 21, a light receiving section 22, and a distance measurement section 23. The distance measurement light emitting section 21 emits distance measurement light and has an emission optical axis 31 on which a light emitting element 32 (e.g., a laser diode (LD)) is provided, and further on which a light projecting lens 33 is provided.

[0027] In addition, a first reflecting mirror 34 serving as a deflection optical member is provided on the emission optical axis 31, and a second reflecting mirror 35 serving as a deflection optical member is disposed on the light receiving optical axis 37 so as to face the first reflecting mirror 34. The first reflecting mirror 34 and the second reflecting mirror 35 are configured so that the exit optical axis 31 coincides with the distance measurement optical axis 4 . Further, an optical axis deflection unit 36 ​​is disposed on the distance measurement optical axis 4 .

[0028] The light receiving unit 22 in Figure 2 receives reflected distance measurement light from the reinforcing bar 300, which is the object to be measured. The light receiving unit 22 has a light receiving optical axis 37 parallel to the emission optical axis 31, and the light receiving optical axis 37 is common with the distance measurement optical axis 4. A light receiving element 38, for example a photodiode (PD), is provided on the light receiving optical axis 37, and an imaging lens 39 is also provided. The imaging lens 39 forms an image of the reflected distance measuring light on the light receiving element 38, which receives the reflected distance measuring light and generates a light receiving signal. The light receiving signal is input to the distance measuring unit 23.

[0029] (Regarding the optical axis deflection unit 36 ​​of the surveying instrument 10) An optical axis deflection unit 36 ​​is disposed on the objective side of the imaging lens 39 on the light receiving optical axis 37 in FIG. The distance measuring unit 23 controls the light emitting element 32 to emit a laser beam as distance measuring light. This laser beam is configured such that the distance measurement optical axis 4 is deflected by the optical deflection unit 36 ​​(distance measurement optical deflection section 36a) so as to head toward the distance measurement point (reinforcing bar 300). Specifically, a laser beam is irradiated toward the reinforcing bar 300 in Figure 1, and the reflected distance measuring light reflected from the reinforcing bar 300, which is the object to be measured, enters the light receiving unit 22 via the optical axis deflection unit 36 ​​(reflected distance measuring light deflection section 36b) and the imaging lens 39.

[0030] The reflected distance measuring light deflection section 36b re-deflects the distance measuring light axis 4 deflected by the distance measuring light deflection section 36a so as to return it to its original state, and causes the light receiving element 38 to receive the reflected distance measuring light. The light receiving element 38 sends a light receiving signal to the distance measuring unit 23, and the distance measuring unit 23 measures the distance to the measurement point (reinforcing bar 300) based on the light receiving signal from the light receiving element 38.

[0031] As shown in FIG. 2, the optical axis deflection unit 36 ​​includes a pair of optical prisms 41a and 41b. The Fresnel prisms used as these optical prisms 41a, 41b each have prism elements 42a, 42b and a number of prism elements 43a, 43b arranged in parallel, and are in the shape of a plate.

[0032] (Regarding laser trajectory control using the optical axis deflection unit 36) Distance measurement light is emitted from the light emitting element 32 in FIG. 2, and the distance measurement light is collimated by the light projecting lens 33, passes through the distance measurement light deflection unit 36a (petism elements 42a, 42b), and is emitted toward the reinforcing bar 300, which is the measurement object. Here, by transmitting through the distance measurement light deflection unit 36a, the distance measurement light is deflected by the prism elements 42a and 42b in the required direction, that is, the direction of the reinforcing bar 300, and is emitted.

[0033] The reflected distance measuring light reflected by the reinforcing bar 300 passes through the reflected distance measuring light deflection unit 36b (prism elements 43a, 43b) and is incident thereon, and is focused on the light receiving element 38 by the imaging lens 39. Thereafter, the reflected distance measuring light passes through the reflected distance measuring light deflection unit 36 ​​b , and is deflected by the prism elements 43 a and 43 b so that the optical axis of the reflected distance measuring light coincides with the light receiving optical axis 37 . That is, the direction and angle of deflection of the emitted distance measuring light can be changed arbitrarily by combining the rotational positions of the prism elements 42a and 42b. Specifically, as described later, the laser distance measuring light can be changed into a straight line, a circle, an ellipse, a continuous Z-shape (zigzag shape), or the like.

[0034] (Regarding the surveying device camera (imaging unit) 27 of the surveying device 10) 2, the surveying equipment camera (imaging unit) 27 has an imaging optical axis 5. This imaging optical axis 5 is configured to be capable of acquiring an image of the reinforcing bar 300 by pointing it at, for example, the reinforcing bar 300 in FIG. Also, an imaging lens 48 and an imaging element 49 are provided on the imaging optical axis 5. This imaging element 49 is a CCD or CMOS sensor that is a collection of pixels, and the position of each pixel on the imaging element can be specified. For example, the position of each pixel is specified in a coordinate system with the optical axis of the surveying equipment camera 27 as the origin.

[0035] The hardware configuration of the surveying instrument 10 shown in Figs. 1 and 2 is as described above. Below, the main configuration including the crane 200 in Fig. 1 and the software of the surveying instrument 10 will be described.

[0036] It also has the crane 200 and surveying instrument 10 in Figure 1, and a computer, which has a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc. (not shown), which are connected via a bus, etc.

[0037] FIG. 3 is a schematic block diagram showing the main components of the crane 200 of FIG. As shown in FIG. 3, the crane 200 has a "crane control unit 201", which controls not only the above-mentioned "crane side communication device 202", "crane camera 203", and "crane side display 204", but also a "crane side various information input unit 205" and a "crane side various information storage unit 206", etc.

[0038] FIG. 4 is a schematic block diagram showing the main configuration of the surveying instrument 10 shown in FIGS. As shown in FIG. 4, the surveying instrument 10 has a "surveying instrument control unit 102", which controls the above-mentioned "surveying instrument side communication device 101", "surveying instrument camera 27", "surveying instrument side display unit 11", "surveying unit 20", "distance measurement unit 23", "arithmetic processing unit 24", "attitude processing unit 26" and "emission direction detection unit 25", as well as the "surveying instrument side first various information storage unit 110", "surveying instrument side second various information storage unit 120" and "surveying instrument side third various information storage unit 130".

[0039] 5 to 7 are schematic block diagrams showing the main components of the "surveying instrument side first various information storage unit 110", the "surveying instrument side second various information storage unit 120" and the "surveying instrument side third various information storage unit 130", respectively. The contents of these components will be described later. Incidentally, the "crane camera 203" in FIG. 1 and the "surveying equipment camera 27" in FIG. 2 are configured so that their times can be synchronized with each other and data can be acquired at a specified time. In addition, each camera 203, 27 can obtain the exact shooting time using a GNSS (Global Navigation Satellite System) such as a GPS (Global Positioning System), and can also obtain the exact time by accessing an NTP (Network Time Protocol) server via the Internet (not shown). Here, an NTP server is a server that acquires and distributes accurate time information.

[0040] (Main operations of Guidance System 1) FIG. 8 is a main flow chart showing a guidance preparation process of the guidance system 1 of this embodiment, and FIG. 9 is a main flow chart showing a guidance process. Each of these steps will be described below, along with the contents of each block in FIG. 5 to FIG.

[0041] In this embodiment, an example will be described in which an operator sits in the operator's seat 210 of the crane 200 shown in FIG. 1 and operates the crane 200 to move the reinforcing bar 300 to a target position. At that time, the operator can view the crane side display 204 arranged in the cockpit 210, and this crane side display 204 is configured to enable the operator to grasp the reinforcing bar 300 from his / her viewpoint. Furthermore, the crane-side display 204 is configured to display a guidance screen that assists the operator in moving the reinforcing bar 300 using the crane 200.

[0042] (About the guidance preparation process) First, the guidance preparation process in FIG. 8 will be described. In step (hereinafter referred to as "ST") 1 of FIG. 8, relative position information of the crane camera 203, the surveying instrument 10, and the surveying instrument camera 27 is stored in advance in the "relative position information storage unit 111" of FIG. 5 of the "surveying instrument 10".

[0043] Next, the relative position and orientation information of the crane camera 203 and the surveying equipment camera 27 is specified. Specifically, the process is executed in ST2 and thereafter. In ST2, the "feature point processing unit (program) 112" in Fig. 5 operates to create feature point information on the image from the image information of the crane camera 203 and the surveying equipment camera 27, and store the information in the "feature point information storage unit 113" in Fig. 5.

[0044] Next, the process proceeds to ST3. In ST3, the processing unit 112 operates to refer to the "feature point information storage unit 113," identify common feature points on the images of the crane camera 203 and the surveying equipment camera 27, and store them in the "common point feature part storage unit 114" in FIG.

[0045] Next, the process proceeds to ST4. In ST4, the "matching processing unit (program) 115" in Fig. 5 operates to search for "common feature points" between the crane camera 203 and the survey camera 27 by referring to the "common feature point storage unit 114", perform matching processing, and store the result in the "matching information storage unit 116" in Fig. 5.

[0046] Next, the process proceeds to ST5. In ST5, the "relative position and orientation information generating unit (program) 117" in Fig. 5 operates to obtain the relative position and orientation between the crane camera 203 and the surveying equipment camera 27 based on the information in the "matching information storage unit 116" and store it in the "relative position and orientation information storage unit 121" in Fig. 6.

[0047] Specifically, using a technique such as SfM (Structure from Motion), the shape of the object shown in the given image is restored, and at the same time, the positions and orientations of the crane camera 203 and the surveying equipment camera 27 are estimated and stored as relative position and orientation information. Therefore, through these processes, the position and posture of the crane camera 203 and the surveying equipment camera 27 can be clearly grasped, and an image of the reinforcing bar 300 in Figure 1 captured from the perspective of the surveying equipment camera 27 can be changed to the perspective of the crane camera 203. It should be noted that, when the relative position and orientation information is updated, the position and orientation information of each of the cameras 203 and 27 is requested to be acquired again.

[0048] (Guidance process) Next, the operation of the guidance system of this embodiment will be specifically described with reference to the flowchart of FIG. 9, taking as an example an operator operating the crane 200 of FIG. 1 to move the reinforcing bar 300 to a target position. First, in ST11 of FIG. 9, an operator operating the crane 200 in the control room 210 of the crane 200 taps on the reinforcing bar 300 displayed on the crane side display 204 in the control room 210 to identify it.

[0049] Next, the process proceeds to ST12. In ST12, the crane-side display 204, which displays the screen captured by the crane camera 203, identifies the coordinate position information of the tapped point on the crane-side display 204, and transmits the coordinate information to the surveying instrument 10 via the crane-side communication device 202.

[0050] Next, the process proceeds to ST13. In ST13, the surveying instrument 10 refers to the "relative position and orientation information storage unit 121" in Fig. 6, and identifies the steel frame 2 identified by the crane camera 203 on the screen of the surveying instrument side camera 27, based on the position and orientation information of the surveying instrument camera 27 and the crane camera 203.

[0051] Next, the process proceeds to ST14. In ST14, the "pixel processing unit (program) 122" in Fig. 6 operates to pick up pixels of an area that is likely to be the steel frame 2 recognized within the screen captured by the measuring device camera 27 of the measuring device 10, and store these in the "extracted pixel storage unit 123" in Fig. 6.

[0052] Next, the process proceeds to ST15. In ST15, the "first principal component detection unit (program) 124" in Fig. 6 operates, and refers to the "extracted pixel storage unit 123", and the surveying instrument 10 irradiates the picked-up area with a laser in the "principal component direction (e.g., the X-axis direction on the screen)" (not in all directions), performs "principal component analysis", detects it as a "first principal component", and stores it in the "first principal component storage unit 125". At this time, the surveying instrument 10 deflects the laser irradiation direction using the optical deflection unit 36 ​​shown in FIG.

[0053] Next, the process proceeds to ST16. In ST16, the "second principal component detection unit (program) 126" in Fig. 6 operates to irradiate the laser in a direction perpendicular to the "first principal component" (the Y-axis direction on the screen), perform component analysis, detect it as the "second principal component", and store it in the "second principal component storage unit 127" in Fig. 6.

[0054] Next, the process proceeds to ST17. In ST17, the "object shape estimation unit (program) 131" in Fig. 7 operates to compare the first and second principal components of the "first principal component storage unit 125" and "second principal component storage unit 127" in Fig. 6, estimate shape information of the reinforcing bar 300, obtain three-dimensional information of the reinforcing bar 300 and its position and orientation information (obtain shape information, position information, and orientation information of the steel frame 2), and store them in the "object three-dimensional information storage unit 132" in Fig. 7.

[0055] Specifically, the first and second principal components are compared, and if the ratios of the two are close, the lengths of the components in 12 directions centered on the principal component are examined to determine whether the shape is a square or a circle, etc. Also, based on the ratio of the first principal component to the second principal component, for example, if there is a large amount of the first principal component, it is determined to be a rectangle or the like.

[0056] After estimating the shape in this way, the laser is irradiated using an irradiation method suited to the shape. For example, if the shape is circular, the laser is irradiated in a circular shape, and if it is rectangular, the laser is irradiated continuously in a Z shape (zigzag) along the longitudinal direction and measured (surveyed). Then, three-dimensional information and position and orientation information of the reinforcing bar 300 are obtained (shape information, position information, and orientation information of the steel frame are obtained).

[0057] Thus, in this embodiment, the surveying instrument 10 generates shape information such as a circle, and then selects an irradiation method (e.g., a circular shape) that is suitable for this shape information. By irradiating the distance measuring light only to the necessary parts, three-dimensional information of the target object can be obtained, enabling quick and efficient surveying.

[0058] This makes it possible to achieve much faster and more efficient measurements than in the past, when distance measuring light was irradiated in all directions. Furthermore, in this embodiment, scanning for irradiating distance measuring light can be performed uniformly within a required range, so that the reliability of the acquired information is significantly improved.

[0059] Furthermore, since the position of the steel frame 2 is known in the above-mentioned process and the target point to which the steel frame 2 will move is already known, the distance to the target point, the inclination, etc. are also known. Therefore, the surveying device 10 refers to the "storage unit 132 for storing three-dimensional information of object" in FIG. Then, based on the current position, shape, inclination, etc. of the steel frame 2, which is the object specified by the operator, as well as 3D information, the 3D information of the reinforcing bar 300 is changed to the viewpoint of the crane camera 203, transmitted to the crane 200, and displayed as point cloud information, etc. on the crane side display 204.

[0060] Conventionally, there has been a problem in that the accurate position (coordinate position) and accurate shape of the target reinforcing bar 300 cannot be grasped only from the image captured by the crane camera 203. In contrast to this, in this embodiment, the surveying device 10 capable of three-dimensional surveying can grasp not only the exact position of the reinforcing bar 300 but also the exact shape of the target object. Based on this information, the three-dimensional information of the reinforcing bars 300 surveyed by the surveying device 10 can be displayed on the crane side display 204, which is viewed by the operator in the cockpit 210 of the crane 200, changed to the viewpoint of the crane camera 203.

[0061] This allows the operator to grasp the exact position and shape of the reinforcing bar 300, and also makes it possible to grasp the blind spot of the crane camera 203 on the crane side display 204. Therefore, when the reinforcing bar 300 is suspended by the crane 200 and moved to a target location, the operator can accurately grasp the position and shape of the reinforcing bar 300 and operate it accordingly. Furthermore, as described below, by displaying the difference or the like, which is information on the difference between the target point of the movement of the reinforcing bar 300, on the crane-side display 204, the reinforcing bar 300 can be moved more accurately to the target point.

[0062] (Specific aspects displayed on the crane side display 204) First, the steel frame 2 is displayed on the crane side display 204 from the viewpoint of the crane camera 203, and the parts of the steel frame 2 that are in the blind spot from the crane camera 203 side can be shown using virtual lines or point cloud data.

[0063] Furthermore, the steel frame 2 displayed on the crane-side display 204 may be configured to show difference information such as "direction in which to move" and "inclination". For example, as shown in FIG. 10, arrows may indicate Xm up, Xm in the back, Xm to the right, Xm in front, Xm down, Xm to the left, etc. As shown in FIG. 10, when the deviation of the steel frame 2 from the target point or the like is indicated by an arrow or the like, the arrow may be displayed larger as the deviation becomes larger. FIG. 10 is an example of a screen showing information such as the displacement of the reinforcing bar 300.

[0064] Also, as shown in FIG. 11, if the target position is further back than the current position of the steel frame 2, the target position may be shown by a dotted line, and if the target position is closer to the current position of the steel frame 2, the position may be shown by a thick solid line, as shown in FIG. 12. In this way, by displaying the current position and the target position of the reinforcing bar 300 from the same viewpoint and overlapping on the same screen, the operator of the crane 200 can accurately and easily move the reinforcing bar 300 to the target position. 11 and 12 are examples of screens showing the relationship between the reinforcing bar 300 and the target position.

[0065] Furthermore, in this embodiment, in addition to the above-mentioned "movement guidance mode," it is also possible to display a "tilt guidance mode" as shown in FIG. Figure 13 is an example screen showing the degree of inclination of the reinforcing bar 300 with an arrow. As shown in Figure 13, the operator can easily understand that the reinforcing bar 300 is inclined and the degree of inclination by simply visually checking the reinforcing bar 300 and the arrow on the crane side display 204.

[0066] In this embodiment, if there is an obstacle between the target position and the current position, such as behind the steel frame 2 (in the operator's blind spot), a warning may be displayed. This warning can be configured, for example, to notify by sound or vibration when the steel frame 2 is about to come into contact with an obstacle or the like.

[0067] In this embodiment, the crane camera 203 and the surveying equipment camera 27 are used as examples. However, the present invention is not limited to this, and other cameras such as a camera of a mobile terminal may also be arranged. At this time, the survey (measurement) data of the target object, such as the reinforcing bar 300, may be displayed on the display of the mobile terminal from the viewpoint of the camera of the mobile terminal.

[0068] The camera may be attached to glasses or the like worn by the worker, or may be attached to a part of the body, such as an arm, leg, or head. In this case, the glasses may be configured so that the camera image and the survey data are displayed on a display. In addition, when the camera is attached to a moving object such as glasses, a technology such as SLAM (Simultaneous Localization and Mapping) that simultaneously estimates the camera's position and creates an environmental map will be used to continuously update the camera's position and orientation.

[0069] Although the above-described embodiment has been described with reference to an example of the case in which it is implemented as an apparatus, the present invention is not limited to this, and may be stored and distributed in a storage medium such as a magnetic disk (a floppy disk, a hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a magneto-optical disk (MO), or a semiconductor memory, as a program that can be executed by a computer.

[0070] The storage medium may be any medium that can store a program and is computer readable. The storage format of the storage medium is not particularly limited.

[0071] In addition, an OS (operating system), database management software, network software, or other MW (middleware) running on a computer may execute some of the processes for realizing this embodiment based on instructions from a program installed on the computer from a storage medium.

[0072] Furthermore, the storage medium in the present invention is not limited to a medium independent of a computer, but also includes a storage medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like.

[0073] Furthermore, the computer in the present invention only needs to execute each process in this embodiment based on a program stored in a storage medium, and may be a device consisting of a single personal computer (PC) or the like, or may be a system in which multiple devices are connected to a network.

[0074] In addition, the computer in this invention is not limited to a personal computer, but also includes an arithmetic processing device, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions of the present invention by a program.

[0075] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the claims. [Explanation of symbols]

[0076] REFERENCE SIGNS LIST 1 Guidance system, 2 Tripod, 3 Base unit, 7 Housing, 8 Protractor, 9 Vertical rotation unit, 10 Surveying instrument, 11 Surveying instrument display unit, 12 Operation unit, 20 Surveying unit, 21 Distance measurement light emitting unit, 22 Light receiving unit, 23 Distance measurement unit, 24 Calculation processing unit, 25 Emission direction detection unit, 26 Attitude processing unit, 27 Surveying instrument camera (imaging unit), 31 Emission optical axis, 32 Light emitting element, 33 Light projecting unit Lens, 34...first reflecting mirror, 35...second reflecting mirror, 36...optical axis deflection unit, 36a...distance measuring light deflection section, 36b...reflected distance measuring light deflection section, 37...light receiving optical axis, 38...light receiving element, 39...imaging lens, 41a, 41b...optical prism, 42a, 42b...prism element, 43a, 43b...prism element, 48...imaging lens, 49...imaging element, 101...surveying instrument side communication device, 102...surveying instrument control section, 110...first various information storage section on the surveying instrument side memory unit, 111...relative position information memory unit, 112...feature point processing unit, 113...feature point information memory unit, 114...common point feature memory unit, 115...matching processing unit, 116...matching information memory unit, 117...relative position and orientation information generation unit, 120...second various information memory unit on the surveying device side, 121...relative position and orientation information memory unit, 122...pixel processing unit, 123...extracted pixel memory unit, 124...first principal component detection unit, 125...first principal component memory unit, 126... Second principal component detection unit, 127, second principal component storage unit, 130, surveying device side third various information storage unit, 131, object shape estimation unit, 132, object 3D information etc. storage unit, 200, crane, 201, crane control unit, 202, crane side communication device, 203, crane camera, 204, crane side display, 205, crane side various information input unit, 206, crane side various information storage unit, 210, operator's seat, 300, reinforcing bar, R, installation reference point

Claims

1. A guidance system having a surveying instrument for surveying an object from a viewpoint different from a viewpoint of a user, and a display unit for displaying the object from the viewpoint of the user, The surveying instrument is configured to irradiate distance measuring light toward the object and receive the distance measuring light reflected from the object to measure the distance, A guidance system characterized in that the survey information of the object surveyed by the surveying instrument is converted to a viewpoint of the user and displayed on the display unit.

2. a surveying instrument side imaging device for imaging the object at a surveying viewpoint of the surveying instrument; The guidance system according to claim 1, further comprising a user-side imaging device that images the object from the user's viewpoint, and based on relative position and orientation information between the surveying instrument-side imaging device and the user-side imaging device, the surveying information of the surveying instrument is converted to a user-side viewpoint and displayed on the display unit.

3. a distance measuring light of the surveying instrument is irradiated only toward at least one of the object and its surroundings, and shape information of the object is generated by receiving the distance measuring light reflected from the object; The guidance system according to claim 1 or 2, characterized in that it is configured to acquire three-dimensional information of the object by selecting and executing an irradiation method of the distance measuring light that is suitable for the generated shape information.

4. acquiring target position information to which the object is to be moved; 4. The guidance system according to claim 1, wherein difference information between the current position information and the target position information is displayed on the display unit from a user's viewpoint.

5. A method for controlling a guidance system having a surveying instrument for surveying an object from a viewpoint different from a viewpoint on the user's side and a display unit for displaying the object from the viewpoint on the user's side, comprising the steps of: the surveying instrument irradiates distance measuring light toward the object and receives distance measuring light reflected from the object to measure distance; A method for controlling a guidance system, comprising converting the survey information of the object surveyed by the surveying instrument to a viewpoint on the user's side and displaying the converted information on the object on the display unit.

6. A guidance system having a surveying instrument for surveying an object from a viewpoint different from that of a user and a display unit for displaying the object from the viewpoint of the user, The surveying instrument has a function of irradiating a distance measuring light toward the object and receiving the distance measuring light reflected from the object to measure the distance; A control program for a guidance system for executing a function of converting the survey information of the object surveyed by the surveying instrument to the viewpoint of the user and displaying it on the display unit.

Images