Guidance image generating device and joint work support system

Abstract

To provide a guidance image generating device capable of generating a guidance image for supporting the joining work between a joining member and a member to be joined.SOLUTION: A guidance image generating device 62 includes: a positional information acquisition unit 621 that acquires positional information including the relative distance and angle between a pipe held by a work machine and a pipe to be joined; and a guidance image generation unit 622 that generates a guidance image PT25 for supporting an operation of causing the work machine to perform the operation of joining the pipe to the pipe to be joined based on the positional information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a guidance image generation device that generates a guidance image and a joining work support system.

Background Art

[0002] Conventionally, a technique for joining a pipe such as a water pipe to a pipe to be joined using an attachment for pipe joining attached to a work machine has been known (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technique of Patent Document 1 has a problem that it is difficult to accurately grasp the relative position between the joining member and the member to be joined from the driver's seat of the work machine, and it is difficult to perform the operation only by visual inspection.

[0005] An object of the present invention is to realize a guidance image generation device that generates a guidance image for assisting the joining operation between a joining member and a member to be joined.

Means for Solving the Problems

[0006] A guidance image generation device according to an embodiment of the present invention includes a position information acquisition unit that acquires position information including a relative distance and a relative angle between a joining member held by a work machine and a member to be joined joined to the joining member, and a guidance image generation unit that generates a guidance image for assisting an operation for causing the work machine to perform an operation of joining the joining member to the member to be joined based on the position information (first configuration).

[0007] In the above configuration, it is possible to generate a guidance image for assisting an operation for causing the work machine to perform an operation of joining the joining member held by the work machine to the joined member joined to the joining member, based on position information including the relative distance and relative angle between the joining member held by the work machine and the joined member joined to the joining member.

[0008] Thereby, for example, it is possible to present a guidance image for assisting the operation to an operator who operates the work machine to perform a joining operation of joining the joining member and the joined member.

[0009] Therefore, it is possible to realize a guidance image generation device that generates a guidance image for assisting the joining operation between the joining member and the joined member.

[0010] In the first configuration, the guidance image generation unit generates a guidance image in which the outer shape of a portion including the joining end of the joining member and the outer shape of a portion including the joined end of the joined member are arranged based on the position information (second configuration).

[0011] The joining end of the joining member and the joined end of the joined member are important portions to be connected in the joining operation. In the above configuration, it is possible to generate a guidance image that allows an operator to easily grasp the relative positions of the joining end of the joining member, which is important in the joining operation of the pipe member, and the joined end of the joined member.

[0012] In the first configuration, the guidance image generation unit generates the guidance image including a symbol indicating at least one of the axial directions of the joining member and the joined member (third configuration).

[0013] In the above configuration, it is possible to present at least one of the axial directions of the joining member and the joined member to an operator who performs a joining operation of connecting the joining member and the joined member in the axial direction, by the symbol of the guidance image. Therefore, the operation of aligning the axes of the joining member and the joined member, which are pipe members, can be made more efficient.

[0014] In the first configuration, the guidance image generation unit generates the guidance image including a virtual straight line passing through at least one of the outer edges in the radial direction of the joining member and the joined member and parallel to the axial direction in the captured image (fourth configuration).

[0015] In the above configuration, the guidance image can present a virtual straight line passing through at least one of the outer edges in the radial direction of the joining member and the joined member and parallel to the axial direction to an operator who performs a joining operation of axially connecting the joining member and the joined member, which are pipe members. Therefore, it becomes easier for the operator to grasp the trajectory when the joining member and the joined member are moved in the axial direction.

[0016] In the first configuration, the position information acquisition unit detects the relative distance and the relative angle based on a captured image in which at least one of the joining member and the joined member is imaged (fifth configuration).

[0017] In the above configuration, a guidance image can be generated in the connection operation between the joining member, which is one of a pair of pipe members to be joined, and the joined member, which is the other pipe member. Therefore, the joining operation of the pipe member in which the joining member extends in the axial direction can be made more efficient.

[0018] In the fifth configuration, it further has a memory that stores shape-related information, which is information regarding the three-dimensional shapes of the joining member and the joined member. The position information acquisition unit detects the relative distance and the relative angle based on a pattern matching process between at least a part of the shapes of the joining member and the joined member imaged in the captured image and the shapes of the joining member and the joined member shown in the shape-related information (sixth configuration).

[0019] In the above configuration, the guidance image generation unit detects the relative distance and the relative angle between the joining member and the joined member by a pattern matching process between at least a part of the shapes of the joining member and the joined member imaged in the captured image and the three-dimensional shapes of the joining member and the joined member shown in the shape-related information.

[0020] Therefore, the detection accuracy of the relative position between the joining member and the member to be joined is improved. Thus, since the guidance image generation unit generates a guidance image based on the accurately detected relative position, the accuracy of operation support by the guidance image can be improved.

[0021] In the sixth configuration, the guidance image generation unit generates a specific direction view image showing a state of the three-dimensional shapes of the joining member and the member to be joined as viewed from at least one direction as the guidance image (seventh configuration).

[0022] In the above-described configuration, since a specific direction view image showing a state of the joining member and the member to be joined as viewed from at least one direction is generated as the guidance image, a guidance image that allows an operator to intuitively grasp the relative position between the joining member and the member to be joined can be generated.

[0023] In the seventh configuration, it further includes a display control unit that causes the display device to display the guidance image. The guidance image generation unit generates a first direction view image showing a state of the three-dimensional shapes of the joining member and the member to be joined as viewed from a first direction and a second direction view image showing a state as viewed from a second direction different from the first direction as the specific direction view image. The display control unit causes the display device to simultaneously display the first direction view image and the second direction view image as the guidance image (eighth configuration).

[0024] In the above-described configuration, a plurality of the specific direction view images can be presented to the operator simultaneously as the guidance image. Therefore, the operator can grasp the states of the joining member and the member to be joined as viewed from a plurality of directions, and it becomes easier for the operator to grasp the relative position between the joining member and the member to be joined.

[0025] In the seventh configuration, there is further provided a display control unit that simultaneously displays, on a display device, a captured image of at least one of the joining member and the member to be joined and the specific-direction view image (ninth configuration).

[0026] In the above configuration, it is possible to present to the operator a captured image of at least a part of the joining member and at least a part of the member to be joined, and a guidance image. As a result, the operator can perform the work with reference to both the actual captured image and the guidance image. Therefore, it becomes easier for the operator to further grasp the relative positions of the joining member and the member to be joined.

[0027] In the first configuration, there is further provided a display control unit that causes the guidance image to be displayed on a display device. The display control unit superimposes the guidance image on a captured image of at least one of the joining member and the member to be joined and displays it (tenth configuration).

[0028] In the above configuration, since the guidance image is superimposed on the captured image and displayed on the display device, the operator can easily grasp the joining member and the state of the joining operation of the joining member.

[0029] In the first configuration, the guidance image generation unit generates a guidance image including the outer shapes of the joining member, the member to be joined, and a guide member provided in the working machine for positioning the joining member and the member to be joined (eleventh configuration).

[0030] In the above configuration, the guidance image generation unit generates a guidance image including the outer shapes of the joining member, the member to be joined, and the guide member based on a captured image in which the joining member, the member to be joined, and the guide member are imaged. Therefore, based on the guide member, the joining operation between the joining member and the member to be joined can be advanced. Therefore, the workability of the operator can be improved.

[0031] The joining work support system according to an embodiment of the present invention includes any one of the first to eleventh configuration guidance image generation devices, a display device that displays the guidance image generated by the guidance image generation device, a working machine that joins the joining member to the joined member, and an operation input device for operating the working machine (12th configuration).

[0032] According to the above configuration, a joining work support system is obtained that presents a guidance image for assisting an operation related to joining the joining member and the joined member to an operator by a display device. Therefore, the operator of the joining work can grasp the relative position between the joining member and the joined member.

[0033] Thus, a joining work support system is obtained that can improve the efficiency of the joining work between the joining member and the joined member.

[0034] In the 12th configuration, the guidance image generation unit generates the guidance image including an image indicating a direction in which the working machine operates to join the joining member to the joined member (13th configuration).

[0035] In the above configuration, the operator can grasp what operation should be performed next based on the guidance image. Therefore, it is possible to assist the operation of the working machine for an operator who is not used to the joining work.

[0036] In the 12th configuration, the guidance image generation unit generates the guidance image including an image indicating the operation content for the operation input device to operate the working machine so as to join the joining member to the joined member (14th configuration).

[0037] Thereby, the operator can easily grasp the necessary operation content for the operation input device.

[0038] In the 12th configuration, the guidance image generation unit generates the guidance image including information regarding prediction of the relative distance and relative angle between the joining member and the member to be joined accompanying an operation on the operation input device (15th configuration).

[0039] In the above configuration, the operator can grasp how the relative distance and relative angle between the joining member and the member to be joined change by an operation using the operation input device based on the guidance image. Therefore, it is possible to assist the operation of the work machine for an operator who is not accustomed to the joining work.

Advantages of the Invention

[0040] A guidance image generation device according to an embodiment of the present invention includes a position information acquisition unit that acquires position information including a relative distance and a relative angle between a joining member held by a work machine and a member to be joined that is joined to the joining member, and a guidance image generation unit that generates a guidance image for assisting an operation for causing the work machine to perform an operation of joining the joining member to the member to be joined based on the position information.

[0041] Thereby, for example, it is possible to present a guidance image for assisting the operation to an operator who operates the work machine to perform a joining operation of joining the joining member and the member to be joined. Therefore, it is possible to realize a guidance image generation device that generates a guidance image for assisting the joining operation between the joining member and the member to be joined.

Brief Description of the Drawings

[0042]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

[0043] Hereinafter, each embodiment will be described with reference to the drawings. In each figure, the same parts are denoted by the same reference numerals, and the description of the same parts will not be repeated. Note that the dimensions of the constituent members in each figure do not faithfully represent the dimensions of the actual constituent members and the dimensional ratios of the respective constituent members.

[0044] In the following description, expressions such as "fix", "connect", and "attach" (hereinafter referred to as "fixing, etc.") include not only cases where members are directly fixed, etc., but also cases where they are fixed, etc. via other members. That is, in the following description, the expression of fixing, etc. includes the meanings of direct and indirect fixing, etc. of members to each other.

[0045] [Embodiment 1] FIG. 1 is a diagram showing a schematic configuration of a joining work support system SYS1 including a guidance image generation device 62 according to Embodiment 1. FIG. 2 is a functional block diagram showing a schematic configuration of an information processing terminal 60 having the guidance image generation device 62 according to Embodiment 1.

[0046] As shown in FIGS. 1 and 2, the joining work support system SYS1 includes a working machine 1000, an information processing terminal 60, and an imaging device 14.

[0047] An attachment can be connected to the tip of the arm portion 1003 of the working machine 1000. The working machine 1000 can perform various construction works and operations by replacing the attachment connected to the tip of the arm portion 1003. An attachment 1 for pipe joining is attached to the working machine 1000. The working machine 1000 is, for example, a mini backhoe which is a construction machine for excavation. The working machine 1000 joins a pipe W1 as a joining member to a joined pipe W2 as a member to be joined. The pipe W1 and the joined pipe W2 each extend in the axial direction and are configured to be connectable in the axial direction.

[0048] Note that the working machine 1000 may be a construction machine other than a mini backhoe, and may have any configuration as long as it is a machine that can operate the attachment 1 for pipe joining.

[0049] The information processing terminal 60 has a guidance image generation device 62 that generates a guidance image for joining work. The information processing terminal 60 is provided in the working machine main body portion 1001 of the working machine 1000.

[0050] The imaging device 14 images at least a part of each of the pipe W1 and the pipe W2 to be joined to generate an imaging image PT11. The imaging device 14 is provided on the pipe joining attachment 1 of the working machine 1000.

[0051] (Working machine) As shown in FIG. 1, the working machine 1000 is capable of traveling in the front-rear direction and is capable of turning horizontally about a rotation axis P extending in the vertical direction. The working machine 1000 can be driven by, for example, hydraulic pressure. Note that the working machine 1000 may be driven by, for example, an electric motor.

[0052] The working machine 1000 includes a working machine main body 1001, a boom portion 1002, an arm portion 1003, and a pipe joining attachment 1.

[0053] The working machine main body 1001 includes an upper swing body 1011 and a lower traveling body 1012. The upper swing body 1011 includes left and right operation levers (operation input devices) 51 and 52 and an information processing terminal 60. The operation levers 51 and 52 are levers for the driver to perform various operations on the working machine 1000. For example, according to the operations on the operation levers 51 and 52, the upper swing body 1011 can turn horizontally about the rotation axis P. Note that the working machine main body 1001 may have operation means such as levers, switches, or buttons other than the left and right operation levers 51 and 52. The information processing terminal 60 is a terminal that performs various information processes and presents various information to the driver. The information processing terminal 60 includes a guidance image generation device 62 that generates a guidance image PT25 based on the imaging image PT11 of the joining work. Details of the information processing terminal 60 will be described later.

[0054] The lower traveling body 1012 has an annular endless track. The lower traveling body 1012 can travel in the front-rear direction.

[0055] The boom part 1002 is a beam-shaped member extending forward from the upper swing body 1011. One end side of the boom part 1002 is connected to the upper swing body 1011. The boom part 1002 is swingable with respect to the work machine main body part 1001. Specifically, the boom part 1002 is rotatable in the vertical direction around the one end side with respect to the upper swing body 1011.

[0056] The arm part 1003 is a beam-shaped member with one end side connected to the front end part of the boom part 1002. The arm part 1003 is swingable with respect to the boom part 1002. Specifically, the arm part 1003 is rotatable in the vertical direction around the one end side with respect to the boom part 1002. An attachment can be connected to the tip part of the arm part 1003. In this embodiment, an attachment 1 for pipe joining is attached to the tip part of the arm part 1003.

[0057] Note that the boom part 1002 and the arm part 1003 can each be driven by a hydraulic cylinder (not shown). The boom part 1002 and the arm part 1003 may also be driven by an electric motor or the like.

[0058] The attachment 1 for pipe joining functions as a gripping device capable of holding the pipe W1. The attachment 1 for pipe joining may be connected to the other end side of the arm part 1003 via an attachment mounting member. The attachment mounting member is configured to be able to easily mount the attachment to the other end side of the arm part 1003.

[0059] Since the configurations of the work machine main body part 1001, the boom part 1002, and the arm part 1003 in the work machine 1000 are the same as the conventional configurations, detailed descriptions thereof are omitted.

[0060] (Attachment for Pipe Joining) The pipe joining attachment 1 is configured to be able to hold the pipe W1, and the held pipe W1 can be moved in the first axial direction which is the direction of the axis L of the pipe W1. The pipe joining attachment 1 includes an arm connection portion 11, a gripping portion 12, an axial movement mechanism 130, and an imaging device 14.

[0061] In the following description, the pipe W1 and the pipe W2 to be joined are preferably, for example, earthquake-resistant pipes having an earthquake-resistant joint that can be bent at a predetermined angle. Note that the pipe W1 and the pipe W2 to be joined may be pipes other than earthquake-resistant pipes.

[0062] The arm connection portion 11 is located at the upper end portion of the pipe joining attachment 1 and is connected to the other end side of the arm portion 1003 of the working machine 1000. The arm connection portion 11 may be connected to the other end side of the arm portion 1003 by bolts or pins, or may be fitted by a fitting portion. The arm connection portion 11 may be connected to the other end side of the arm portion 1003 by any configuration. Further, the arm connection portion 11 may have a rotation axis that allows the pipe joining attachment 1 to rotate in the horizontal direction. According to such a rotation axis in the horizontal direction, the pipe W1 gripped by the gripping portion 12 of the pipe joining attachment 1 can be moved into a trench in the ground or the like without moving and turning the working machine 1000.

[0063] The gripping portion 12 is supported by the axial movement mechanism 130 described later so as to be movable in the axial direction. The gripping portion 12 may have a pair of claws that sandwich and grip both sides of the outer peripheral surface of the pipe W1.

[0064] The axial movement mechanism 130 is connected to the lower portion of the arm connection portion 11. The axial movement mechanism 130 moves the gripping portion 12 in the axial direction of the pipe W1 gripped by the gripping portion 12.

[0065] The imaging device 14 images a subject located within the imaging range R1 at a predetermined frame rate. The imaging device 14 can be realized by a digital camera or the like. The imaging device 14 may be a monocular camera, a binocular stereo camera, or a multiocular camera with three or more eyes. The imaging range R1 is in one of the first axial directions with respect to the pipe joining attachment 1 and is a lower range. When the pipe W1 gripped by the gripping portion 12 is located at a position farther than a predetermined distance from the pipe W2 to be joined in the first axial direction, the imaging range R1 includes, for example, one end portion of the pipe W1 in one of the first axial directions. Also, as shown in FIG. 1, when the pipe W1 gripped by the gripping portion 12 is located within a predetermined distance from the pipe W2 to be joined in the first axial direction, the imaging range R1 includes, for example, one end portion of the pipe W1 in one of the first axial directions and one end portion of the pipe W2 to be joined in one of the second axial directions which is the direction of the axis M. The imaging device 14 has a communication unit 141 that transmits the captured image PT11 to the guidance image generation device 62 by wireless communication. According to the above-described configuration, since the imaging device 14 and the guidance image generation device 62 can communicate at a separated position even if they are not network-connected by wire, the degree of freedom in the installation position of the imaging device 14 can be improved.

[0066] Note that the imaging device 14 may incorporate a battery for obtaining power for operation. Also, the pipe joining attachment 1 may be provided with a battery for supplying power for the operation of the imaging device 14. Also, a power line may be wired from the work machine main body portion 1001 to supply power to the imaging device 14.

[0067] (Information processing terminal) The information processing terminal 60 is attached to the upper swing body 1011 of the work machine 1000 by a support such as a stay. Specifically, the information processing terminal 60 includes a guidance image generation device 62 and a display device 63.

[0068] The guidance image generation device 62 generates a guidance image PT25 based on the captured image PT11 captured by the imaging device 14. The guidance image generation device 62 includes a position information acquisition unit 621, a guidance image generation unit 622, and a display control unit 623.

[0069] The position information acquisition unit 621 acquires position information including the relative distance and relative angle between the pipe W1 held by the working machine 1000 and the pipe W2 to be joined to the pipe W1. The position information acquisition unit 621 acquires position information as follows.

[0070] First, the position information acquisition unit 621 acquires, from the imaging device 14, a captured image PT11 in which at least a part of the pipe W1 and at least a part of the pipe W2 to be joined are captured. In the captured image PT11 shown in FIG. 2, for example, a part of the pipe PTW1 and a part of the pipe W2 to be joined are captured. That is, the captured image PT11 shown in FIG. 2 is an image captured by the imaging device 14 when the pipe W1 is positioned within a predetermined distance from the pipe W2 in the first axis direction. Furthermore, in the captured image PT11, an insertion port PTW11 as a joining end of the pipe PTW1 and a receiving port PTW21 as a joined end of the pipe W2 to be joined are captured.

[0071] Next, the position information acquisition unit 621 detects the positions of the pipe W1 and the pipe W2 to be joined in the three-dimensional space based on the captured image PT11 in which at least one of the pipe W1 or the pipe W2 to be joined is captured. The positions in the three-dimensional space can be detected based on the imaging angles of the pipe PTW1 and the pipe W2 to be joined captured in the captured image PT11, the diameters and dimensions of the pipe W1 or the pipe W2 to be joined as joining targets, and the distances or angles on the three axes of the X-axis, Y-axis, and Z-axis in the three-dimensional space between the pipe W1 and the pipe W2 to be joined. That is, the positions in the three-dimensional space include at least one of the relative distance between the pipe W1 and the pipe W2 to be joined and the relative angle between the pipe W1 and the pipe W2 to be joined.

[0072] FIG. 3 is a schematic diagram for explaining an example of the relative position. The relative distance may be, for example, the distance between the insertion port W11 of the pipe W1 and the receiving port W21 of the pipe to be joined W2. As shown in FIG. 3, the relative distance in the X-axis direction of the three-dimensional space may be the interval DX1 between the end of the insertion port W11 of the pipe W1 and the end of the receiving port W21 of the pipe to be joined W2. The relative distance in the Z-axis direction of the three-dimensional space may be the interval DZ1 between the axis L of the insertion port W11 of the pipe W1 and the axis M of the receiving port W21 of the pipe to be joined W2. Although not particularly shown, the relative distance in the Y-axis direction of the three-dimensional space may be the interval between the axis L of the insertion port W11 of the pipe W1 and the axis M of the receiving port W21 of the pipe to be joined W2. The relative distance may be, for example, the distance between the reference point P1 of the camera of the imaging device 14 and the reference point P11 of the pipe W1 or the reference point P12 of the pipe to be joined W2.

[0073] Also, the relative angle may be, for example, the angle at which the axis L of the pipe W1 and the axis M of the pipe to be joined W2 intersect. The relative angle may be the angle at which the axis L of the pipe W1 or the axis M of the pipe to be joined W2 intersects with respect to a reference line such as the horizontal line or the optical axis of the camera of the imaging device 14. Note that the relative distance and the relative angle may be calculated for each of the three axes of the X-axis, Y-axis, and Z-axis.

[0074] Based on the position in the three-dimensional space detected in this way, the position information acquisition unit 621 generates position information including the relative distance and the relative angle. Note that the meaning of "acquiring position information" should be interpreted broadly, and it widely includes cases where the position information acquisition unit 621 itself generates and acquires position information, or cases where the position information acquisition unit 621 acquires position information from the outside. Also, the position information may be generated based on information other than the captured image PT11. The position information may be generated based on measurement results by a laser, ultrasonic wave, radio wave, or the like.

[0075] The guidance image generation unit 622 generates a guidance image PT25 that supports an operation for causing the working machine 1000 to perform an operation of joining the pipe W1 to the pipe W2 to be joined, based on the position information acquired by the position information acquisition unit 621, and outputs it to the display device 63. The guidance image PT25 may be, for example, an image that is displayed on the display device 63 by being superimposed on the captured image PT11, or may be an image that is displayed on the display device 63 separately from the captured image PT11. The guidance image generation unit 622 may generate, for example, a guidance image PT25 including an arrow image indicating the direction in which the working machine 1000 is operated to join the pipe W1 to the pipe W2 to be joined, as shown in FIG. 2.

[0076] The display control unit 623 generates a display image PT12 in which the guidance image PT25 is superimposed on the captured image PT11 that captures at least one of the pipe W1 or the pipe W2 to be joined, and outputs it to the display device 63. Thereby, the display control unit 623 superimposes the guidance image PT25 on the captured image PT11 that captures at least one of the pipe W1 or the pipe W2 to be joined and causes it to be displayed on the display device 63.

[0077] The display device 63 displays various types of information. The display device 63 displays, for example, the display image PT12 output from the guidance image generation device 62. The display device 63 can be realized by a liquid crystal panel, an organic EL display, or the like.

[0078] In the above-described configuration, based on the position information including the relative distance and relative angle between the pipe W1 held by the working machine 1000 and the pipe W2 to be joined to the pipe W1, a guidance image PT25 that supports an operation for causing the working machine 1000 to perform an operation of joining the pipe W1 to the pipe W2 can be generated.

[0079] Thereby, for example, it is possible to present the guidance image PT25 for assisting the operation to an operator who operates the working machine 1000 to perform a joining operation of joining the pipe W1 and the pipe W2 to be joined.

[0080] Therefore, a guidance image generation device 62 that generates a guidance image PT25 for assisting the joining operation between the pipe W1 and the pipe W2 to be joined can be realized.

[0081] Further, in the above-described configuration, it is possible to generate a guidance image PT25 in the connection operation between the pipe W1 as a pipe member extending in the axial direction and the pipe W2 to be joined. Therefore, the joining operation of the pipe member in which the joining member extends in the axial direction can be made more efficient.

[0082] Also, according to the above-described configuration, a joining operation support system SYS1 is obtained in which a guidance image PT25 for assisting an operation related to the joining operation of joining the pipe W1 and the pipe W2 to be joined is presented to the operator by the display device 63. For this reason, the operator can grasp the relative position between the pipe W1 and the pipe W2 to be joined.

[0083] Therefore, a joining operation support system that can improve the efficiency of the joining operation between the pipe W1 and the pipe W2 to be joined is obtained.

[0084] Also, in the above-described configuration, the operator can grasp what operation should be performed next based on the guidance image PT25 including an arrow image indicating the direction in which the working machine 1000 operates. Therefore, it is possible to assist the operation of the working machine 1000 for an operator who is not used to the joining operation.

[0085] Also, in the above-described configuration, since the guidance image PT25 is superimposed on the captured image PT11 and displayed on the display device 63, the operator can easily grasp the status of the joining operation of the pipe W1 and the pipe W2 to be joined.

[0086] [Embodiment 2] FIG. 4 is a diagram showing a schematic configuration of a joining operation support system SYS2 including a guidance image generation device 65 according to Embodiment 2. FIGS. 5 and 6 are diagrams showing a schematic configuration of the pipe joining attachment 2. FIG. 5 is a view of the pipe joining attachment 2 as seen from one side in the width direction. FIG. 6 is a view of the pipe joining attachment 2 as seen from one side in the first axial direction.

[0087] The guidance image generation device 65 according to Embodiment 2 is different from the guidance image generation device 62 according to Embodiment 1 in that the shape data of the three-dimensional model is used for detecting the relative position, and the relative position of the guide member 17 of the pipe joining attachment 2 is also detected. In the following description, the same components as those in Embodiment 1 are denoted by the same reference numerals and the description thereof is omitted, and only the differences from Embodiment 1 will be described.

[0088] (Pipe joining attachment) The schematic configuration of the pipe joining attachment 2 will be described with reference to FIGS. 4 to 6. The pipe joining attachment 2 includes an arm connection portion 11, a gripping portion 12, a support unit 13, an imaging device 15, and a guide member 17.

[0089] A support unit 13, which will be described later, is connected to the lower portion of the arm connection portion 11 so as to be rotatable in the horizontal direction about a rotation axis Q.

[0090] The gripping portion 12 is supported so as to be movable in the first axial direction by an axial direction movement mechanism 32, which will be described later. The gripping portion 12 has gripping claws 21 for gripping the pipe W1. The gripping claws 21 may be configured to be opened and closed by hydraulic pressure.

[0091] The support unit 13 is supported by the arm connection portion 11 so as to be rotatable in the horizontal direction about the rotation axis Q. The support unit 13 includes a support frame 31 and an axial direction movement mechanism 32.

[0092] The support frame 31 includes an upper plate 311, a stay portion 312, and a front end portion 313. The upper plate 311 is formed in a flat plate shape and is connected to the arm connection portion 11 on its upper surface. The stay portion 312 is a pair of flat plate-like members, which are connected to the lower surface of the upper plate 311 and extend downward and forward. The front end portion 313 is connected to the front end of the stay portion 312.

[0093] The axial movement mechanism 32 moves the gripping part 12 in the first axial direction of the pipe W1 gripped by the gripping part 12. The axial movement mechanism 32 includes a slide mechanism 321 and an axial drive part 322.

[0094] The slide mechanism 321 is provided on the lower surface of the upper plate 311. The slide mechanism 321 supports the gripping part 12 so as to be movable in the first axial direction with respect to the upper plate 11. The slide mechanism 321 can be realized by, for example, a guide rail or the like.

[0095] One end of the axial drive part 322 in the first axial direction is connected to the stay part 312 via the connecting part 315, and the other end of the axial drive part 322 in the first axial direction is connected to the front part of the gripping part 12. For example, the axial drive part 322 is a hydraulic cylinder. The hydraulic cylinder is supplied with, for example, hydraulic pressure output from the working machine 1000. The axial drive part 322 supplies a driving force for moving the gripping part 12 in the first axial direction according to the supplied hydraulic pressure.

[0096] The guide member 17 is a member for positioning the pipe W1 with respect to the pipe W2 to be joined. The guide member 17 is connected to the lower part of the front end part 313.

[0097] The guide member 17 is supported by the support frame 31 so as to be located on one side in the first axial direction of the insertion port W11 of the pipe W1 with respect to the gripping part 12. The guide member 17 has a plate-like part 173 and a pipe-to-be-joined contact part 175. The plate-like part 173 is connected to the lower part of the front end part 313 and extends downward. The pipe-to-be-joined contact part 175 is connected to the lower end part of the plate-like part 173. When positioning the insertion port W11 of the pipe W1 gripped by the gripping part 12 with respect to the receiving port W21 of the pipe W2 to be joined, the pipe-to-be-joined contact part 175 contacts the receiving port W21 side of the pipe W2 to be joined. Note that the receiving port W21 side of the pipe W2 to be joined includes not only the receiving port W21 of the pipe W2 to be joined but also the receiving port W21 side of the pipe body in the pipe W2 to be joined.

[0098] The joined pipe contact portion 175 has an inverted V-shaped configuration in which the distance between the joined pipe contact portions 175 expands downward. The joined pipe contact portion 175 is flat plate-shaped, with the upper end side being connected and the lower end side being spaced apart in the width direction. That is, the upper end side of the joined pipe contact portion 175 is fixed to the lower end portion of the plate-shaped portion 173.

[0099] Since the joined pipe contact portion 175 has such a configuration, even when the outer diameter of the receiving port W21 side of the joined pipe W2 is different, the joined pipe contact portion 175 can be brought into contact with the outer peripheral surface of the receiving port W21 side of the joined pipe W2.

[0100] When the joined pipe contact portion 175 is viewed in the axial direction of the receiving port W21 of the joined pipe W2, it contacts a portion of the outer peripheral surface of the receiving port W21 side of the joined pipe W2 that is located in a direction oblique to the width direction (for example, a direction of 45 degrees with respect to the width direction). Thereby, the pipe joining attachment 2 is positioned in the width direction and in a direction orthogonal to the width direction with respect to the receiving port W21 of the joined pipe W2.

[0101] Note that the joined pipe contact portion 175 may contact a portion located on one or the other side in the width direction of the outer peripheral surface of the receiving port W21 side of the joined pipe W2, or may contact a portion located in a direction orthogonal to the width direction of the outer peripheral surface of the receiving port W21 side of the joined pipe W2. That is, the joined pipe contact portion 175 may contact a portion of the outer peripheral surface of the receiving port W21 side of the joined pipe W2 that is located in at least one of the width direction and the direction orthogonal to the width direction when the receiving port W21 is viewed in the axial direction of the joined pipe W2. One of the lower end sides of the joined pipe contact portion 175 may contact the outer peripheral surface of the receiving port W21 side of the joined pipe W2.

[0102] The imaging device 15 is supported by the support frame 31 with respect to the attachment 2 for pipe joining so as to be located in one of the first axis directions with respect to the gripping portion 12 and above the gripping claws 21 of the gripping portion 12. The imaging device 15 is connected to the support frame 31 via the imaging device fixing portion 316 of the stay portion 312. The imaging device 15 has a downward imaging range R1 in one of the first axis directions. That is, the optical axis of the camera of the imaging device 15 is in one of the first axis directions and faces downward. Therefore, the imaging device 15 can image a range including at least a part of each of the pipe W1 and the pipe to be joined W2 and at least a part of the guide member 17 to generate an imaging image PT11. Further, the imaging device 15 is a monocular camera. When generating the imaging image PT11 by the imaging device 15 which is a monocular camera, the data amount of the imaging image PT11 can be reduced as compared with a multi-eye camera such as a stereo camera.

[0103] (Information processing terminal) FIG. 7 is a functional block diagram showing a schematic configuration of an information processing terminal 60 having a guidance image generation device 65 according to Embodiment 2. FIG. 8 is a schematic diagram showing a data structure of the shape-related information D10.

[0104] The guidance image generation device 65 includes a memory 625, a position information acquisition unit 626, a guidance image generation unit 627, and a display control unit 628.

[0105] The memory 625 stores shape-related information D10 which is information regarding the shapes of the pipe W1 and the pipe to be joined W2 respectively.

[0106] The shape-related information D10 includes the shape data of the three-dimensional model. The shape-related information D10 is information regarding the three-dimensional shapes of the insertion port W11 of the pipe W1, the guide member 17, and the receiving port W21 of the pipe W2 to be joined. The shape-related information D10 includes, for example, an insertion port model D11, a guide member model D12, and a receiving port model D13. The insertion port model D11 is the shape data of the three-dimensional model of the insertion port W11 as the joining end of the pipe W1. The guide member model D12 is the shape data of the three-dimensional model of the pipe W2 contact portion 175 of the guide member 17. The receiving port model D13 is the shape data of the three-dimensional model of the receiving port W21 as the joining end of the pipe W2 to be joined. As shown in FIG. 8, the insertion port model D11, the guide member model D12, and the receiving port model D13 are three-dimensional model data that can be rotated around the X-axis, Y-axis, and Z-axis, respectively, with the intersection of the X-axis, Y-axis, and Z-axis as the reference point. Note that the insertion port model D11 and the receiving port model D13 may be stored in the memory 625 for each diameter of the pipe W1 and the pipe W2 to be joined. Also, the X-axis direction in the receiving port model D13 may coincide with the first axis direction of the pipe W1. Further, the X-axis direction in the insertion port model D11 may coincide with the second axis direction of the pipe W2 to be joined.

[0107] The position information acquisition unit 626 detects the positions of the guide member 17, the pipe W1, and the pipe W2 to be joined in the three-dimensional space by performing matching processing between the shapes of the portions of the pipe W1, the guide member 17, and the pipe W2 to be joined imaged in the captured image PT11 and the shapes of the insertion port model D11 of the pipe W1, the guide member model D12 of the guide member 17, and the receiving port model D13 of the pipe W2 to be joined in the shape-related information D10 stored in the memory 625. Details of the matching processing in the position information acquisition unit 626 will be described later.

[0108] Based on the detection result of the position information acquisition unit 626, the guidance image generation unit 627 generates and outputs a guidance image PT26 including, for example, a highlight display of the pipe PTW1, the guide member PT17, and the pipe PTW2 to be joined imaged in the captured image PT11. Details of the guidance image PT26 output by the guidance image generation unit 627 will be described later.

[0109] (Matching process and guidance image) Hereinafter, with reference to FIGS. 9 to 13, the matching process by the position information acquisition unit 626 and the guidance image generated by the guidance image generation unit 627 will be described. FIG. 9 is a schematic diagram for explaining the matching process when the pipe W1 and the pipe W2 to be joined are at separated positions. FIG. 10 is a diagram showing the captured image PT11 captured in the positional relationship shown in FIG. 9. FIG. 11 is a schematic diagram showing the display image PT12 output as a result of the matching process for the captured image PT11 shown in FIG. 10. FIG. 12 is a schematic diagram for explaining the matching process when the first axial direction of the pipe W1 and the second axial direction of the pipe W2 to be joined are at the same position in the vertical direction. FIG. 13 is a schematic diagram showing the display image PT12 output as a result of the matching process for the captured image captured in the positional relationship shown in FIG. 12.

[0110] As shown in FIG. 9, in the first axial direction, when the guide member 17 is located on the other side of the receiving port W21 of the pipe W2 to be joined and, in the vertical direction, the pipe W1 held by the gripping portion 12 is located above the pipe W2 to be joined, the upper part of the insertion port W11 of the pipe W1 and the guide member 17 enter the imaging range R1. On the other hand, the receiving port W21 of the pipe W2 to be joined is located in the blind spot R2 of the camera blocked by the pipe W1. Therefore, the upper part of the receiving port W21 of the pipe W2 to be joined is imaged in a state where at least a part thereof is hidden by the pipe W1.

[0111] The position information acquisition unit 626 performs a matching process on the captured image PT1 shown in FIG. 10 using the shape-related information D10 including the shape data of the three-dimensional model. As the matching process, a pattern matching process of the prior art or the like can be adopted. The position information acquisition unit 626 performs a matching process on the captured image PT1 using the shape when the guide member model D12 is viewed in the optical axis direction of the camera of the imaging device 15. As a result, the shape of the guide member PT17 in the captured image PT1 matches. Note that the matching process of the guide member PT17 can be performed based on the feature amount related to the inverted V-shaped shape of the guide member 17. When the guide member 17 has a shape with other features such as a through hole, the matching process may be performed using the feature amount.

[0112] In addition, the position information acquisition unit 626 performs a matching process on the captured image PT1 using the shape when the insertion port model D11 of the pipe W1 is viewed in the optical axis direction of the camera of the imaging device 15. As a result, the shape of the insertion port PTW11 of the pipe PTW1 in the captured image PT1 matches.

[0113] In addition, the position information acquisition unit 626 performs a matching process on the captured image PT1 using the shape when the receiving port model D13 of the pipe W2 to be joined is viewed in the optical axis direction of the camera of the imaging device 15. However, in the captured image PT1, the receiving port PTW21 of the pipe PTW2 to be joined is partially hidden and thus does not match.

[0114] The position information acquisition unit 626 detects the relative position based on the result of the matching process. Since the position information acquisition unit 626 performs the matching process on the captured image PT1 based on the three-dimensional shape data, it can detect the three-dimensional position and angle between the insertion port W11 of the tube W1 and the guide member 17. In detecting the relative position, the position information acquisition unit 626 can use, for example, the imaging device 15 as the reference point P1. The position information acquisition unit 626 detects the relative distance of the insertion port W11 with respect to the reference point P1 on the three axes of the X-axis, Y-axis, and Z-axis based on the positional relationship between the reference point P1 and the reference point of the insertion port model D11 of the tube W1. The position information acquisition unit 626 detects the relative distance of the insertion port W11 with respect to the reference point P1 on the three axes of the X-axis, Y-axis, and Z-axis based on the positional relationship between the reference point P1 and the reference point of the guide member model D12.

[0115] The position information acquisition unit 626 detects, for example, the relative angle of the X-axis of the insertion port W11 of the tube W1 and the guide member 17 with respect to the optical axis of the camera. The relative angle means the angle formed by the optical axis of the camera and the straight line extending in the X-axis direction. The angle can be expressed by the rotation angles around the X-axis, Y-axis, and Z-axis of the straight line extending in the X-axis direction with respect to the optical axis of the camera.

[0116] Note that the expressions of the relative distance and the relative angle are not limited to the above expressions, and known expression methods can be applied.

[0117] The guidance image generation unit 627 generates and outputs a guidance image PT26 including a highlight display PT261 of the insertion port PTW11 of the tube PTW1 imaged in the captured image PT1 and a highlight display PT262 of the guide member PT17 (see FIG. 11). Note that the guidance image generation unit 627 may generate the highlight display PT261 of the insertion port PTW11 of the tube PTW1 and the highlight display PT262 of the guide member PT17 in different colors.

[0118] In addition, as shown in FIG. 11, the display control unit 623 generates and outputs a display image PT12 in which a guidance image PT26 is superimposed on the captured image PT11. Note that the display control unit 623 may draw the relative distance and relative angle between the tube PTW1 and the guide member PT17 detected by the matching process of the position information acquisition unit 626 on the display image PT12.

[0119] As shown in FIG. 12, in the first axis direction, the Z axis of the guide member 17 coincides with the Z axis of the receiving port W21 of the tube to be joined W2, and in the vertical direction, when the X axis of the tube W1 gripped by the gripping unit 12 coincides with the X axis of the tube to be joined W2, the upper part of the insertion port W11 of the tube W1, the guide member 17, and the upper part of the receiving port W21 of the tube to be joined W2 enter the imaging range R1.

[0120] The position information acquisition unit 626 performs a matching process using the insertion port model D11 of the tube W1, the guide member model D12, and the receiving port model D13 of the tube to be joined W2, thereby detecting the positions of the insertion port W11 of the tube W1, the guide member 17, and the receiving port W21 of the tube to be joined W2 in the captured image PT1.

[0121] In addition, the guidance image generation unit 627 generates and outputs a guidance image PT26 including highlight displays PT261, PT262, and PT263 of the tube PTW1, the guide member PT17, and the tube to be joined PTW2 captured in the captured image PT1. The display image PT12 is generated and output. Note that the guidance image generation unit 627 may generate the highlight displays PT261, PT262, and 263 in different colors. PT263 to generate and output. Generate and output the display image PT12. Note that the guidance image generation unit 627 may generate the highlight displays PT261, PT262, and 263 in different colors.

[0122] Further, when the Z-axis of the guide member 17 coincides with the Z-axis of the receiving port W21 of the pipe to be joined W2 and the X-axis of the pipe W1 coincides with the X-axis of the pipe to be joined W2, the guidance image generation unit 627 may generate a guide display PT27 indicating the width of the pipe W1 as a guidance image PT26. Specifically, the guidance image generation unit 627 calculates a virtual straight line passing through the outer edge in the radial direction of the pipe PTW1 and parallel to the first axial direction in the captured image PT1. The guidance image generation unit 627 generates a guidance image PT26 including the guide display PT27 based on the virtual straight line. In the above configuration, the operator can be presented with the virtual straight line of the pipe PTW1 in the captured image PT1. Therefore, it becomes easier for the operator to grasp the trajectory when the pipes W1 and the pipe to be joined W2 are moved in the axial direction. Note that the guidance image generation unit 627 may also calculate the virtual straight line for the pipe to be joined PTW2.

[0123] Also, as shown in FIG. 13, the display control unit 623 generates and outputs a display image PT12 in which the guidance image PT26 is superimposed on the captured image PT11. Note that, as shown in FIG. 13, the display control unit 623 may output the relative distance and relative angle of each of the pipe PTW1, the guide member PT17, and the pipe to be joined PTW2 detected by the matching process.

[0124] In the above configuration, the guidance image generation unit 627 detects the relative distance and relative angle between the pipe W1 and the pipe to be joined W2 by pattern matching processing between at least a part of the shapes of the pipe PTW1 and the pipe to be joined PTW2 captured in the captured image PT11 and the three-dimensional shapes of the pipes W1 and the pipe to be joined W2 shown in the shape-related information D10.

[0125] For this reason, the detection accuracy of the relative position between the pipe W1 and the pipe to be joined W2 is improved. Therefore, since the guidance image generation unit 627 generates the guidance image PT26 based on the accurately detected relative position, the accuracy of operation support by the guidance image PT26 can be improved.

[0126] In addition, in the above-described configuration, the guidance image generation unit 627 generates a guidance image PT26 including the outer shapes of the pipe W1, the pipe to be joined W2, and the guide member 17 based on the captured image PT11 in which the pipe W1, the pipe to be joined W2, and the guide member 17 are captured. Therefore, based on the guide member 17, the joining operation between the pipe W1 and the pipe to be joined W2 can be advanced. Therefore, the workability of the operator can be improved.

[0127] Furthermore, since the imaging device 15 and the guide member 17 are supported with respect to the pipe joining attachment 2 by the support frame 31, the position of the guide member PT17 in the captured image PT1 is basically the same. For this reason, in the matching process of the shape of the guide member PT17 in the captured image PT1, the search range for matching can be limited to the range where the guide member PT17 should be reflected. Thereby, the processing load of the matching process of the guide member PT17 can be reduced.

[0128] The position of the insertion port PTW11 of the pipe PTW1 in the captured image PT1 varies depending on the position where the gripping portion 12 grips the pipe W1. However, the variation in the position of the insertion port PTW11 of the pipe PTW1 in the captured image PT1 is basically assumed to be a variation in the first axial direction. Therefore, the search range for matching can be limited to the range of the variation. Thereby, the processing load of the matching process of the insertion port PTW11 of the pipe PTW1 can be reduced. Also, the matching process of the insertion port PTW11 of the pipe PTW1 may be performed based on the detected position of the guide member PT17. Thereby, the insertion port PTW11 of the pipe PTW1 can be detected with higher accuracy.

[0129] The search range of the receiving port PTW21 of the pipe to be joined PTW2 in the captured image PT1 may be determined based on at least one of the detected positions of the insertion port PTW11 of the pipe PTW1 and the guide member PT17. For example, the search range of the receiving port PTW21 of the pipe to be joined PTW2 in the captured image PT1 may be limited between the insertion port PTW11 of the pipe PTW1 and the guide member PT17.

[0130] [Other Embodiments] Although the embodiments of the present invention have been described above, the above-described embodiments are merely examples for implementing the present invention. Therefore, without being limited to the above-described embodiments, it is possible to appropriately modify and implement the above-described embodiments within the scope not departing from the gist thereof.

[0131] In each of the above embodiments, the joining member is a pipe, and the member to be joined is a pipe to be joined. However, the joining member and the member to be joined may be other than pipes. The joining member and the member to be joined may be a column, a beam, a wall, a ceiling, a floor, a frame body, or the like.

[0132] In each of the above embodiments, the imaging devices 14 and 15 image a subject located within the imaging range R1 at a predetermined frame rate. In each of the above embodiments, although not particularly described, the guidance image generation device may generate a guidance image at the same frequency as the frame rate of the imaging device or at a frequency lower than the frame rate.

[0133] In each of the above embodiments, the transport device is the work machine 1000. However, the transport device may be a machine other than the work machine.

[0134] In each of the above embodiments, the imaging devices 14 and 15 are provided on the pipe joining attachments 1 and 2 of the work machine 1000. However, the imaging device may be provided on the arm portion of the work machine, or the imaging device may be provided on the boom portion of the work machine. Further, the imaging device may be realized as a device separate from the work machine. The information processing terminal may have the imaging device.

[0135] In each of the above embodiments, the information processing terminal 60 is provided in the working machine main body 1001 of the working machine 1000. However, the information processing terminal may be a different information terminal from the working machine. The information processing terminal may be a portable terminal such as a notebook PC (personal computer), a tablet terminal, a wearable terminal, or a smartphone. Further, the working machine may be provided with a holder for fixing the portable terminal.

[0136] In each of the above embodiments, the information processing terminal 60 has the guidance image generation devices 62 and 65. However, the guidance image generation device may be a device different from the information processing terminal. The guidance image generation device may be included in the imaging device. Further, the guidance image generation device may be a server device connected via a communication network to the imaging device.

[0137] In each of the above embodiments, the position information acquisition units 621 and 626 detect the relative position based on the insertion port PTW11 of the pipe PTW1 imaged in the captured image PT1 and the receiving port PTW21 of the pipe to be joined PTW2. However, the position information acquisition unit may detect the relative position based on parts other than the insertion port of the pipe imaged in the captured image and parts other than the receiving port of the pipe to be joined. Further, the pipe may have a receiving port, and the pipe to be joined may have an insertion port.

[0138] In each of the above embodiments, the guidance image generation devices 62 and 65 and the imaging devices 14 and 15 are wirelessly communicably connected. However, the guidance image generation device and the imaging device may be communicably connected by wire.

[0139] In Embodiment 2 above, the pipe contact portion 175 of the guide member 17 has an inverted V-shaped configuration. However, the pipe contact portion may have a different configuration such as an inverted U-shaped configuration or an inverted T-shaped configuration instead of the inverted V-shaped configuration. Further, the guide member may not have a pipe contact portion.

[0140] In the second embodiment, the shape-related information D10 includes the shape data of the three-dimensional model. However, the shape-related information may include two-dimensional shape data. The position information acquisition unit may perform matching processing on the captured image based on the two-dimensional shape.

[0141] In the second embodiment, the shape-related information D10 includes the insertion port model D11, the guide member model D12, and the receiving port model D13. Further, the position information acquisition unit 626 detects the relative positions of the guide member 17, the pipe W1, and the pipe W2 to be joined by performing matching processing between the shapes of the pipe W1, the guide member 17, and the pipe W2 to be joined captured in the captured image PT11 and the shapes of the insertion port model D11 of the pipe W1, the guide member model D12 of the guide member 17, and the receiving port model D13 of the pipe W2 to be joined in the shape-related information D10. However, the guide member data may not be included in the shape-related information. Further, the position information acquisition unit may detect the relative positions of the pipe and the pipe to be joined by performing matching processing between the shapes of the pipe and the pipe to be joined captured in the captured image and the shapes of the pipe and the pipe to be joined in the shape-related information.

[0142] In the second embodiment, the position information acquisition unit 626 performs matching processing on the captured image PT11 using the insertion port model D11, the guide member model D12, and the receiving port model D13. However, the presence or absence of the matching processing of the guide member model may be switched according to the operation of the operator in the information processing terminal or the like. Further, in the information processing terminal, the shape used for the matching processing may be selectable. For example, the diameters of the insertion port model and the receiving port model may be selectable in the information processing terminal.

[0143] In the second embodiment, the guidance image generation unit 627 generates a guidance image PT26 including a guide display PT27. However, the guidance image generation unit may generate and output a guidance image including an image indicating the operation content for the operation lever (operation input device) for operating the work machine so as to join the pipe to the pipe to be joined. FIG. 14 is a schematic diagram showing an example of a display image PT3 including a guidance image PT30.

[0144] When the position information acquisition unit 626 detects a relative position indicating that in the first axial direction, the guide member 17 is located on the other side of the receiving port W21 of the pipe W2 to be joined, and in the vertical direction, the pipe W1 gripped by the gripping unit 12 is located above the pipe W2 to be joined, the guidance image generation unit 627 determines the operations required for the joining operation. To join the pipe W1 to the pipe W2 to be joined, the following operations are necessary. First, in the first axial direction, move the pipe joining attachment 2 to one side in the first axial direction. Next, lower the pipe joining attachment 2 until the guide member 17 contacts the upper part of the receiving port W21 of the pipe W2 to be joined. Next, move the gripping unit 12 in one direction in the first axial direction by the axial movement mechanism 32.

[0145] The guidance image generation unit 627 generates an operation guidance display PT31 (see FIG. 14). The operation guidance display PT31 includes a lever operation display PT311 corresponding to the left operation lever 51 and a lever operation display PT312 corresponding to the right operation lever 52.

[0146] For example, in the work machine 1000, the forward and backward operations FR1, RR1 of the left operation lever 51 correspond to pushing and pulling of the arm, and the left and right operations LF1, RG1 correspond to left and right turning. The forward and backward operations FR2, RR2 of the right operation lever 52 correspond to lowering and raising of the boom, and the left and right operations LF2, RG2 correspond to excavation and dumping of the bucket. Note that these operation patterns are examples, and in the work machine 1000, other operation patterns may be adopted.

[0147] For example, in the first axis direction, the guidance image generation unit 627 determines that operations such as pushing the arm and lowering the boom are necessary to move the pipe joining attachment 2 in one direction of the first axis. The guidance image generation unit 627 generates an operation guidance display PT31 indicating that the front operation FR1 of the left operation lever 51 and the front operation FR2 of the right operation lever 52 are necessary as a guidance image. Thereby, the operator can easily grasp the necessary operation contents for the left and right operation levers 51 and 52.

[0148] Further, the guidance image generation unit 627 generates an axis display PT33 indicating the first axis direction of the pipe W1.

[0149] Further, the guidance image generation unit 627 generates and outputs a guidance image PT30 including the operation guidance display PT31 and the axis display PT33. Specifically, the axis display PT33 is superimposed on the captured image PT11, while the operation guidance display PT31 is arranged next to the captured image PT11 on which the axis display PT33 is superimposed.

[0150] In this way, since the guidance image PT30 includes the axis display PT33 as a symbol, the operator can be presented with the first axis direction of the pipe W1 by the axis display PT33. Therefore, the axis alignment work between the pipe W1 as a pipe member and the pipe W2 to be joined can be made more efficient.

[0151] In the second embodiment, the guidance image generation unit 627 generates a guide display PT27 for superimposing on the captured image PT11. However, the guidance image generation unit may generate a specific direction perspective image showing a state of a virtual three-dimensional space in which a pipe model, a pipe to be joined model, etc. are arranged as seen from at least one direction as a guidance image according to the detection result of the position information acquisition unit. FIG. 15 is a diagram showing an example of the perspective image PT4.

[0152] First, the guidance image generation unit 627 arranges the outer shape of the portion including the insertion port W11 which is the joining end of the pipe W1, the outer shape of the guide member 17, and the outer shape of the portion including the receiving port W12 which is the joined end of the pipe W2 to be joined, based on the relative distance and the relative angle. That is, the guidance image generation unit 627 arranges the insertion port model D11, the guide member model D12, and the receiving port model D13 in a virtual three-dimensional space based on the relative distances and relative angles of the insertion port W11, the guide member 17, and the receiving port W21 respectively. As shown in FIG. 15, the guidance image generation unit 627 generates a specific direction viewpoint image PT4 including a plan view image (first direction view image) PT41 showing a state of viewing the virtual three-dimensional space in which each model is arranged in one direction of the Z-axis direction (first direction), a side view image (second direction view image) PT42 showing a state of viewing in one direction of the Y-axis direction (second direction), and a front view image PT43 showing a state of viewing in one direction of the X-axis direction.

[0153] In the above-described configuration, as the guidance image, a specific direction view image PT4 showing a state of viewing at least one of the pipe W1 and the pipe W2 to be joined from one direction is generated, so that an operator can intuitively generate a guidance image that makes it easy to grasp the relative positions of the pipe W1 and the pipe W2 to be joined.

[0154] Also, the insertion port W11 as the joining end of the pipe W1 and the receiving port W21 as the joined end of the pipe W2 to be joined are important parts that are the objects of connection in the joining operation. In the above-described configuration, a specific direction image PT4 that enables an operator to easily grasp the relative positions of the insertion port W11 of the pipe W1 and the receiving port W21 of the pipe W2 to be joined, which are important in the joining operation of the pipe members, can be generated.

[0155] Note that the guidance image generation unit 627 may add an axis line PT411 indicating the X-axis direction of the insertion port model D11 and an axis line PT412 indicating the X-axis direction of the receiving port model D13 to the plan view image PT41 and the side view image PT42. Also, the guidance image generation unit 627 may add a virtual straight line PT413 that passes through the outer edge in the radial direction of the receiving port model D13 and is parallel to the X-axis direction of the model of the receiving port model D13 to the plan view image PT41 and the side view image PT42.

[0156] The guidance image generation unit 627 may add the center point P21 where the X axis of the insertion port model D11 is located and the center point P22 where the X axis of the receiving port model D13 is located to the front view image PT43.

[0157] In addition, the display control unit 623 generates and outputs a display image PT12 including the captured image PT11 and the specific direction viewpoint image PT4. That is, the display control unit 623 causes the captured image PT11 and the specific direction viewpoint image PT4 to be simultaneously displayed on the display device 63.

[0158] In the above configuration, the captured image PT11 and the specific direction viewpoint image PT4 as the guidance image can be presented to the operator according to the situation. Thereby, the operator can perform the work with reference to both the actual captured image PT11 and the guidance image. Therefore, it becomes easier for the operator to grasp the relative positions of the pipe W1 and the pipe to be joined W2. Note that the display control unit 623 may generate a display image PT12 in which another guidance image is superimposed on the captured image PT11 and the specific direction viewpoint image PT4 is arranged.

[0159] In addition, in the above configuration, the display control unit 623 causes the plan view image PT41, the side view image PT42, and the front view image PT43 to be simultaneously displayed on the display device 63 as the guidance image.

[0160] Thereby, a plurality of specific direction view images can be presented to the operator simultaneously as the guidance image. Therefore, the operator can grasp the state of viewing the pipe W1 and the pipe to be joined W2 from a plurality of directions, and it becomes easier for the operator to grasp the relative positions of the pipe W1 and the pipe to be joined W2. Note that the display control unit 623 does not necessarily have to display any one of the plan view image PT41, the side view image PT42, and the front view image PT43.

[0161] In the above-described Embodiment 2, when the pipe W1 is positioned at a predetermined position with respect to the pipe W2 to be joined, the guidance image generation unit 627 generates a guide display PT27 indicating the width of the pipe W1. However, the guidance image generation unit may generate a guide display even when the pipe is not positioned at a predetermined position with respect to the pipe to be joined. Further, the guidance image generation unit may generate a guidance image including information regarding prediction of the relative distance and relative angle between the pipe and the pipe to be joined associated with an operation on the operation input device. Specifically, the guidance image generation unit predicts the operation of the working machine due to an operation being performed on the operation input device. The guidance image generation unit generates information regarding the prediction based on the predicted operation. The information regarding the prediction may be, for example, a marker indicating the position of the pipe after a predetermined time has elapsed from the current time. Further, the information regarding the prediction may be a locus or an arrow indicating the movement of the pipe until a predetermined time has elapsed from the current time.

[0162] In the above-described configuration, by means of the guidance image, an operator can grasp how the relative distance and relative angle between the pipe and the pipe to be joined change due to an operation by the operation input device. Therefore, it is possible to assist the operation of the working machine for an operator who is not accustomed to the joining work.

Industrial Applicability

[0163] The present invention can be used in a guidance image generation device that generates a guidance image.

Explanation of Signs

[0164] 1, 2 Pipe joining attachment 11 Arm connection part 12 Gripping part 13 Support unit 14, 15 Imaging device 62, 65 Guidance image generation device 63 Display device 621, 626 Position information acquisition unit 622, 627 Guidance image generation unit 623, 628 Display control unit 625 Memory 1000 working machine SYS1 and SYS2 joint operation support system Pipe W1 Pipe W2 to be joined

Claims

1. A position information acquisition unit that acquires position information including a relative distance and a relative angle between a joining member held by a working machine and a joined member joined to the joining member, A guidance image generation unit that generates a guidance image for assisting an operation for causing the working machine to perform an operation of joining the joining member to the joined member based on the position information, having a guidance image generation device.

2. In the guidance image generation device according to Claim 1, the guidance image generation unit generates a guidance image in which an outer shape of a portion including a joining end portion of the joining member and an outer shape of a portion including a joined end portion of the joined member are arranged based on the position information. a guidance image generation device.

3. In the guidance image generation device according to Claim 1, the guidance image generation unit generates the guidance image including a symbol indicating at least one axial direction of the joining member and the joined member. a guidance image generation device.

4. In the guidance image generation device according to Claim 1, the guidance image generation unit generates the guidance image including a virtual straight line passing through an outer edge in a radial direction of at least one of the joining member and the joined member and parallel to the axial direction in the captured image. a guidance image generation device.

5. In the guidance image generation device according to Claim 1, the position information acquisition unit detects the relative distance and the relative angle based on a captured image in which at least one of the joining member and the joined member is imaged. a guidance image generation device.

6. In the guidance image generation device according to Claim 5, further having a memory that stores shape-related information which is information regarding three-dimensional shapes of the joining member and the joined member, the position information acquisition unit detects the relative distance and the relative angle based on a pattern matching process between a shape of at least a part of the joining member and the joined member imaged in the captured image and the shapes of the joining member and the joined member shown in the shape-related information. a guidance image generation device.

7. In the guidance image generation device according to Claim 6, the guidance image generation unit generates, as the guidance image, a specific direction view image showing a state of the three-dimensional shapes of the joining member and the joined member as viewed from at least one direction. a guidance image generation device.

8. In the guidance image generation device according to claim 7, further comprising a display control unit that causes the guidance image to be displayed on a display device, the guidance image generation unit generates, as the specific direction view image, a first direction view image showing a state of the three-dimensional shapes of the joining member and the member to be joined as viewed from a first direction and a second direction view image showing a state as viewed from a second direction different from the first direction, the display control unit causes the first direction view image and the second direction view image to be simultaneously displayed on the display device as the guidance image, a guidance image generation device.

9. In the guidance image generation device according to claim 7, further comprising a display control unit that causes an imaging image obtained by imaging at least one of the joining member or the member to be joined and the specific direction view image to be simultaneously displayed on a display device, a guidance image generation device.

10. In the guidance image generation device according to claim 1, further comprising a display control unit that causes the guidance image to be displayed on a display device, the display control unit causes the guidance image to be displayed by superimposing it on an imaging image obtained by imaging at least one of the joining member or the member to be joined, a guidance image generation device.

11. In the guidance image generation device according to claim 1, the guidance image generation unit generates a guidance image including the outer shapes of the joining member, the member to be joined, and a guide member provided in the working machine for positioning the joining member and the member to be joined, a guidance image generation device.

12. A joining work support system comprising: the guidance image generation device according to any one of claims 1 to 11; a display device that displays the guidance image generated by the guidance image generation device; a working machine that joins the joining member to the member to be joined; and an operation input device for operating the working machine. a joining work support system.

13. In the joining work support system according to claim 12, the guidance image generation unit generates the guidance image including an image indicating a direction in which the working machine is operated to join the joining member to the member to be joined, a joining work support system.

14. In the joining work support system according to claim 12, The guidance image generation unit generates the guidance image including an image indicating an operation content for the operation input device for operating the work machine so as to join the joining member to the joined member. Joining work support system.

15. In the joining work support system according to claim 12, the guidance image generation unit generates the guidance image including information regarding prediction of a relative distance and a relative angle between the joining member and the joined member associated with an operation on the operation input device. Joining work support system.

Images