Joint work image processing device, joint work support system, and joint work image processing program

Abstract

To provide a joint work image processing device capable of generating positional information for efficiently joining a joining material and a joined material.SOLUTION: A joint work image processing device 62 includes: a pick-up image acquisition unit 621 that acquires a pick-up image PT1 including insertion port W11 of a pipe W1 and an insertion port W21 of a pipe W2 to be joined from an imaging unit 14; and a position detection unit 622 that detects the relative position between the pipe W1 and the pipe W2 to be joined based on the pick-up image PT1. The position detection unit 622 detects the shapes of the pipe W1 and the pipe W2 to be joined together picked up in the pick-up image PT1 and the relative position between the pipe W1 and the pipe W2 to be joined by matching the shapes of the pipe W1 and pipe W2 to be joined that are stored in advance.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a joining work image processing apparatus, a joining work support system, and a joining work image processing program for processing an imaging image captured by an imaging device.

Background Art

[0002] Conventionally, a technique for joining a pipe such as a water pipe to a pipe to be joined using an attachment for pipe joining attached to a working machine has been known (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique of Patent Document 1, it may be difficult to accurately grasp the relative position between the joining member and the member to be joined from the driver's seat of the working machine.

[0005] An object of the present invention is to realize a joining work image processing apparatus that generates position information capable of improving the efficiency of the joining work between a joining member and a member to be joined.

Means for Solving the Problems

[0006] An image processing apparatus for joining work according to an embodiment of the present invention includes an imaging image acquisition unit that acquires an imaging image including at least a part of a joining member and at least a part of a joined member joined to the joining member, and a position detection unit that detects a relative position between the joining member and the joined member based on the imaging image. The position detection unit detects the relative position between the joining member and the joined member by matching the shapes of the joining member and the joined member imaged in the imaging image with the shapes of the joining member and the joined member stored in advance. (First configuration).

[0007] According to the above configuration, when joining the joining member and the joined member, the relative position between the joining member and the joined member can be detected based on the imaging image. Further, the position detection unit detects the relative position between the joining member and the joined member by the matching process as described above. Therefore, the relative position can be detected with high accuracy.

[0008] Therefore, when the operator relatively moves the joining member and the joined member, the operator can more accurately grasp the relative position between the joining member and the joined member based on the relative position.

[0009] Thus, an image processing apparatus for joining work that generates position information that can improve the efficiency of the joining work between the joining member and the joined member based on the imaging image can be realized.

[0010] In the first configuration, the joining member and the joined member are each a pipe member extending in the axial direction, and are configured to be connectable in the axial direction. The position detection unit detects, as the relative position, a relative distance between a joining end portion of the joining member and a joined end portion of the joined member joined to the joining end portion of the joining member, and a relative angle between the axial direction of the joining member and the axial direction of the joined member. (Second configuration).

[0011] In the above configuration, when performing a joining operation for axially connecting the pipe members, the position detection unit detects the relative position between one side end of the joining member, which is one of the pipe members, and one side end of the member to be joined, which is the other of the pipe members, as the relative position. Since the relative position related information output unit generates and outputs relative position related information based on the relative position, the operator can easily grasp the relative position of the portion where the joining member and the member to be joined are joined.

[0012] Moreover, since the position detection unit detects a part of the pipe member, the processing load can be reduced as compared with the case of detecting the entire pipe member.

[0013] Therefore, when generating the relative position related information, the processing load of the joining operation image processing apparatus can be reduced.

[0014] In the first configuration, it further has a memory for storing shape related information, which is information regarding the shapes of the joining member and the member to be joined respectively. The shape related information includes shape data of a three-dimensional model. The position detection unit detects the relative position between the joining member and the member to be joined by performing pattern matching processing on the captured image using the shape related information (third configuration).

[0015] In the above configuration, the position detection unit detects the relative position between the joining member and the member to be joined by performing pattern matching processing on the captured image using the shape related information including the shape data of the three-dimensional model. Thereby, the relative position can be detected with higher accuracy. Therefore, it is possible to generate relative position related information that can make the joining operation by the operator more efficient.

[0016] In the first configuration, it includes a relative position related information generation unit that generates relative position related information including at least one of information indicating the relative position and guidance information for assisting the joining operation of joining the joining member to the member to be joined based on the detection result of the position detection unit (fourth configuration).

[0017] In the above configuration, the relative position related information output unit generates relative position related information including at least one of information indicating the relative position and guidance information. The relative position between the joining member and the member to be joined can be grasped based on the information indicating the relative position. The guidance information can assist the operator in operating the transfer device. Therefore, the joining operation between the joining member and the member to be joined can be made more efficient.

[0018] In the fourth configuration, the relative position related information generation unit generates the relative position related information including a guidance image to be displayed on a display device to assist the joining operation (fifth configuration).

[0019] In the above configuration, the guidance information can be visually transmitted to the operator. Therefore, the operator can easily grasp the content of the support for the joining operation.

[0020] The joining operation support system according to an embodiment of the present invention includes any one of the joining operation image processing apparatuses of the first to fifth configurations, a transfer device that relatively moves the joining member with respect to the member to be joined, and an imaging device that images at least a part of each of the joining member and the member to be joined to generate the captured image (sixth configuration).

[0021] According to the above configuration, when the operator performs the joining operation of relatively moving the joining member with respect to the member to be joined by the transfer device, a joining operation support system is obtained in which the operator can grasp the relative position between the joining member and the member to be joined.

[0022] Therefore, a joining operation support system that can make the joining operation between the joining member and the member to be joined more efficient is obtained.

[0023] In the sixth configuration, the transfer device has a guide member for positioning the joining member with respect to the member to be joined, the imaging device acquires the captured image including at least a part of the guide member in addition to at least a part of each of the joining member and the member to be joined, and the position detection unit detects the relative position between the joining member and the member to be joined based on the relative position between the guide member and the joining member or the member to be joined (sixth configuration).

[0024] In the above configuration, since the relative position between the joining member and the member to be joined is detected based on the relative position between the joining member and the guide member or the relative position between the member to be joined and the guide member, the accuracy of the relative position-related information can be further improved.

[0025] In the sixth configuration, the transfer device includes a main body portion, a boom portion swingable with respect to the main body portion, an arm portion swingable with respect to the boom portion, and a gripping device attached to the tip of the arm portion and capable of gripping the joining member. The imaging device is provided on the gripping device or the arm portion (eighth configuration).

[0026] According to the above configuration, by imaging the joining member and the like with the imaging device provided on the gripping device or the arm portion of the transfer device, the joining member and the like can be imaged at a position closer to the joining member. Based on the captured image generated by imaging the joining member at a position closer to the joining member, the joining operation image processing device generates relative position-related information. Thereby, the accuracy of the relative position-related information can be further improved.

[0027] In the sixth configuration, the imaging device is a monocular camera (ninth configuration).

[0028] According to the above configuration, since the imaging device generates a captured image by imaging with a monocular camera, the data amount of the captured image can be reduced compared to a multi-eye camera such as a stereo camera. Thereby, the computational load when generating the relative position-related information can be reduced.

[0029] In the sixth configuration, the imaging device includes a communication unit that transmits the captured image to the joining operation image processing device by wireless communication (tenth configuration).

[0030] According to the above configuration, the degree of freedom in the installation position of the imaging device can be improved.

[0031] In the sixth configuration, the apparatus further includes an operation control device that controls the operation of the transport device so that the transport device joins the joining member to the member to be joined based on the relative position (eleventh configuration).

[0032] According to the above configuration, the transport device can be driven without an operator's operation. Therefore, the joining operation between the joining member and the member to be joined can be made more efficient.

[0033] A joining operation image processing program according to an embodiment of the present invention is a program for controlling the joining operation image processing device having the first configuration. The joining operation image processing program causes a computer to execute the processing of the captured image acquisition unit and the position detection unit in the joining operation image processing device (twelfth configuration).

[0034] Thereby, when joining the joining member and the member to be joined, a program can be realized that detects the relative position between the joining member and the member to be joined based on the captured image and outputs it as relative position related information.

[0035] Therefore, when the operator relatively moves the joining member and the member to be joined, the operator can grasp the relative position between the joining member and the member to be joined based on the relative position related information.

[0036] Therefore, by executing the program, position information that can improve the efficiency of the joining operation between the joining member and the member to be joined based on the captured image can be generated.

Effects of the Invention

[0037] An image pickup image acquisition unit that acquires an image pickup image including at least a part of a joining member and at least a part of a member to be joined that is joined to the joining member, and based on the image pickup image, a position detection unit that detects the relative position between the joining member and the member to be joined imaged in the image pickup image. The position detection unit detects the relative position between the joining member and the member to be joined by performing matching processing between the shapes of the joining member and the member to be joined imaged in the image pickup image and the shapes of the joining member and the member to be joined stored in advance.

[0038] Thereby, when the operator relatively moves the joining member and the member to be joined, based on the relative position, the operator can more accurately grasp the relative position between the joining member and the member to be joined. Therefore, an image processing apparatus for joining work that generates position information that can streamline the joining work between the joining member and the member to be joined can be realized.

Brief Description of the Drawings

[0039]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

DETAILED DESCRIPTION OF THE INVENTION

[0040] Hereinafter, each embodiment will be described with reference to the drawings. In each figure, the same parts are denoted by the same reference numerals, and the description of the same parts will not be repeated. Note that the dimensions of the constituent members in each figure do not faithfully represent the actual dimensions of the constituent members and the dimensional ratios of the constituent members.

[0041] In the following description, expressions such as "fix", "connect", and "attach" (hereinafter, "fix", etc.) include not only the case where members are directly fixed, etc., but also the case where they are fixed, etc. via other members. That is, in the following description, the expressions such as "fix", etc. include the meanings of direct and indirect fixing, etc. of members to each other.

[0042] [Embodiment 1] FIG. 1 is a diagram showing a schematic configuration of a joining work support system SYS1 including a joining work image processing apparatus 62 according to Embodiment 1. FIG. 2 is a functional block diagram showing a schematic configuration of an information processing terminal 60 having the joining work image processing apparatus 62 according to Embodiment 1.

[0043] As shown in FIGS. 1 and 2, the joining work support system SYS1 includes a work machine (transport device) 1000, an information processing terminal 60, and an imaging device 14.

[0044] An attachment can be connected to the tip of the arm portion 1003 of the work machine 1000. The work machine 1000 can perform various construction works and operations by replacing the attachment connected to the tip of the arm portion 1003. An attachment 1 for pipe joining is attached to the work machine 1000. The work machine 1000 is, for example, a mini backhoe which is a construction machine for excavation. The work machine 1000 functions as a transport device that relatively moves a pipe W1 as a joining member with respect to a pipe W2 to be joined as a member to be joined. The pipe W1 and the pipe W2 to be joined each extend in the axial direction and are configured to be connectable in the axial direction.

[0045] Note that the work machine 1000 may be a construction machine other than a mini backhoe, and may be a machine having any configuration as long as it can operate the attachment 1 for pipe joining.

[0046] The information processing terminal 60 has a joining work image processing apparatus 62 that processes a captured image PT1 of the joining work. The information processing terminal 60 is provided in the work machine main body portion 1001 of the work machine 1000.

[0047] The imaging device 14 captures at least a part of each of the pipe W1 and the pipe W2 to be joined and generates a captured image PT1. The imaging device 14 is provided on the attachment 1 for pipe joining of the work machine 1000.

[0048] (Work machine) As shown in FIG. 1, the work machine 1000 is capable of traveling in the front - rear direction and is capable of horizontally turning about a rotation axis P extending in the vertical direction. The work machine 1000 can be driven, for example, by hydraulic pressure. Note that the work machine 1000 may be driven by, for example, an electric motor.

[0049] The work machine 1000 includes a work machine main body 1001, a boom part 1002, an arm part 1003, and a pipe - joining attachment 1.

[0050] The work machine main body 1001 includes an upper swing body 1011 and a lower traveling body 1012. The upper swing body 1011 includes left and right operation levers 51, 52 and an information processing terminal 60. The operation levers 51, 52 are levers for the driver to perform various operations on the work machine 1000. For example, according to the operations on the operation levers 51, 52, the upper swing body 1011 can horizontally turn about the rotation axis P. Note that the work machine main body 1001 may have operation means such as levers, switches, or buttons other than the left and right operation levers 51, 52. The information processing terminal 60 is a terminal that performs various information processing and presents various information to the driver. The information processing terminal 60 includes a joining - work image processing device 62 that processes the captured image PT1 of the joining work. Details of the information processing terminal 60 will be described later.

[0051] The lower traveling body 1012 has an annular endless track. The lower traveling body 1012 can travel in the front - rear direction.

[0052] The boom part 1002 is a beam - shaped member extending forward from the upper swing body 1011. One end side of the boom part 1002 is connected to the upper swing body 1011. The boom part 1002 is swingable with respect to the work machine main body 1001. Specifically, the boom part 1002 can rotate in the vertical direction about the one end side with respect to the upper swing body 1011.

[0053] The arm part 1003 is a beam-shaped member with one end connected to the other end of the boom part 1002. The arm part 1003 is swingable with respect to the boom part 1002. Specifically, the arm part 1003 is rotatable in the vertical direction about the one end with respect to the boom part 1002. An attachment can be connected to the other end side of the arm part 1003. In the present embodiment, a pipe joining attachment 1 is attached to the other end side of the arm part 1003.

[0054] Note that the boom part 1002 and the arm part 1003 can each be driven by a hydraulic cylinder (not shown). The boom part 1002 and the arm part 1003 may also be driven by an electric motor or the like.

[0055] The pipe joining attachment 1 functions as a gripping device capable of gripping the pipe W1. The pipe joining attachment 1 may be connected to the other end side of the arm part 1003 via an attachment mounting member. The attachment mounting member is configured to be able to easily attach the attachment to the other end side of the arm part 1003.

[0056] Since the configurations of the work machine main body part 1001, the boom part 1002, and the arm part 1003 in the work machine 1000 are the same as the conventional configurations, detailed descriptions thereof are omitted.

[0057] (Pipe joining attachment) The pipe joining attachment 1 is configured to be able to grip the pipe W1 and to be able to move the gripped pipe W1 in a first axial direction which is the direction of the axis L of the pipe W1. The pipe joining attachment 1 has an arm connection part 11, a gripping part 12, an axial movement mechanism 130, and an imaging device 14.

[0058] In the following description, the pipe W1 and the pipe to be joined W2 are preferably, for example, earthquake-resistant pipes having an earthquake-resistant joint that can be bent at a predetermined angle. Note that the pipe W1 and the pipe to be joined W2 may be pipes other than earthquake-resistant pipes.

[0059] The arm connection part 11 is located at the upper end part of the pipe joining attachment 1 and is connected to the other end side of the arm part 1003 of the working machine 1000. The arm connection part 11 may be connected to the other end side of the arm part 1003 by bolts or pins, or may be fitted by a fitting part. The arm connection part 11 may be connected to the other end side of the arm part 1003 by any configuration. Further, the arm connection part 11 may have a rotation axis that enables the pipe joining attachment 1 to rotate in the horizontal direction. According to such a rotation axis in the horizontal direction, the pipe W1 held by the gripping part 12 of the pipe joining attachment 1 can be moved into a trench in the ground or the like without moving and turning the working machine 1000.

[0060] The gripping part 12 is supported so as to be movable in the axial direction by an axial movement mechanism 130 described later. The gripping part 12 may have a pair of claws that sandwich and grip both sides of the outer peripheral surface of the pipe W1.

[0061] The axial movement mechanism 130 is connected to the lower part of the arm connection part 11. The axial movement mechanism 130 moves the gripping part 12 in the axial direction of the pipe W1 gripped by the gripping part 12.

[0062] The imaging device 14 captures a subject located within the imaging range R1 at a predetermined frame rate. The imaging device 14 can be realized by a digital camera or the like. The imaging device 14 may be a monocular camera, a binocular stereo camera, or a multiocular camera with three or more eyes. The imaging range R1 is in one direction of the first axis with respect to the pipe joining attachment 1 and is a lower range. When the pipe W1 gripped by the gripping portion 12 is located at a position farther than a predetermined distance from the pipe W2 to be joined in the first axis direction, the imaging range R1 includes, for example, one end portion of the pipe W1 in one direction of the first axis. Also, as shown in FIG. 1, when the pipe W1 gripped by the gripping portion 12 is located within a predetermined distance from the pipe W2 to be joined in the first axis direction, the imaging range R1 includes, for example, one end portion (joining end portion) of the pipe W1 in one direction of the first axis and one end portion (end portion to be joined) of the pipe W2 to be joined in one direction of the second axis which is the direction of the axis M of the pipe W2 to be joined. The imaging device 14 has a communication unit 141 that transmits the captured image PT1 to the joining operation image processing device 62 by wireless communication. According to the above configuration, since the imaging device 14 and the joining operation image processing device 62 can communicate at a separated position even if they are not network-connected by wire, the degree of freedom in the installation position of the imaging device 14 can be improved.

[0063] Note that the imaging device 14 may incorporate a battery for obtaining power for operation. Also, the pipe joining attachment 1 may be provided with a battery for supplying power for the operation of the imaging device 14. Further, a power line may be wired from the work machine main body portion 1001 to supply power to the imaging device 14.

[0064] (Information processing terminal) The information processing terminal 60 is attached to the upper swing body 1011 of the work machine 1000 by a support such as a stay. Specifically, the information processing terminal 60 has a joining operation image processing device 62 and a display device 63.

[0065] The joining operation image processing device 62 processes the captured image PT1 captured by the imaging device 14. The joining operation image processing device 62 has a captured image acquisition unit 621, a position detection unit 622, and a relative position relationship information generation unit 623.

[0066] The imaging image acquisition unit 621 acquires an imaging image PT1 including at least a part of the pipe W1 and at least a part of the pipe W2 to be joined from the imaging device 14. In the imaging image PT1 shown in FIG. 2, for example, a part of the pipe PTW1 and a part of the pipe W2 to be joined are imaged. That is, the imaging image PT1 shown in FIG. 2 is an image captured by the imaging device 14 when the pipe W1 is located within a predetermined distance from the pipe W2 to be joined in the first axis direction. Furthermore, in the imaging image PT1, an insertion port PTW11 as a joining end of the pipe PTW1 and a receiving port PTW21 as a joined end of the pipe W2 to be joined are imaged.

[0067] The position detection unit 622 detects the relative position of the pipe W1 and the pipe W2 to be joined in the three-dimensional space based on the imaging image PT1. The relative position can be detected based on the imaging angles of the pipes PTW1 and W2 to be joined imaged in the imaging image PT1, the diameters and dimensions of the pipe W1 or the pipe W2 to be joined as joining targets, and the distances or angles on the three axes of the X-axis, Y-axis, and Z-axis in the three-dimensional space between the pipe W1 and the pipe W2 to be joined. That is, the relative position includes at least one of the relative distance between the pipe W1 and the pipe W2 to be joined and the relative angle between the pipe W1 and the pipe W2 to be joined. The position detection unit 622 detects the relative position between the pipe W1 and the pipe W2 to be joined, for example, by performing a matching process between the shapes of the pipes W1 and W2 to be joined imaged in the imaging image PT1 and the shapes of the pipes W1 and W2 to be joined stored in advance.

[0068] FIG. 3 is a schematic diagram for explaining an example of the relative position. The relative distance may be, for example, the distance between the insertion port W11 of the pipe W1 and the receiving port W21 of the pipe W2 to be joined. As shown in FIG. 3, the relative distance in the X-axis direction of the three-dimensional space may be the interval DX1 between the end of the insertion port W11 of the pipe W1 and the end of the receiving port W21 of the pipe W2 to be joined. The relative distance in the Z-axis direction of the three-dimensional space may be the interval DZ1 between the axis L of the insertion port W11 of the pipe W1 and the axis M of the receiving port W21 of the pipe W2 to be joined. Although not particularly shown, the relative distance in the Y-axis direction of the three-dimensional space may be the interval between the axis L of the insertion port W11 of the pipe W1 and the axis M of the receiving port W21 of the pipe W2 to be joined. The relative distance may be, for example, the distance between the reference point P1 of the camera of the imaging device 14 and the reference point P11 of the pipe W1 or the reference point P12 of the pipe W2 to be joined.

[0069] Further, the relative angle may be, for example, the angle at which the axis L of the pipe W1 and the axis M of the pipe W2 to be joined intersect. The relative angle may be the angle at which the axis L of the pipe W1 or the axis M of the pipe W2 to be joined intersects with respect to a reference line such as a horizontal line or the optical axis of the camera of the imaging device 14. Note that the relative distance and the relative angle may be calculated for each of the three axes of the X-axis, Y-axis, and Z-axis.

[0070] The relative position related information generation unit 623 generates relative position related information D1 regarding the relative position based on the detection result of the position detection unit 622 and outputs it to the display device 63. The relative position related information D1 includes at least one of information indicating the relative position and guidance information for assisting the joining operation of joining the pipe W1 to the pipe W2 to be joined.

[0071] The display device 63 displays various types of information. The display device 63 displays, for example, the relative position related information D1 output from the joining operation image processing device 62. The display device 63 can be realized by a liquid crystal panel, an organic EL display, or the like.

[0072] According to the above configuration, when joining the pipe W1 as the joining member and the joined pipe W2 as the member to be joined, the relative position between the pipe W1 and the joined pipe W2 can be detected based on the captured image PT1 and output as relative position related information D1. Further, the position detection unit 622 detects the relative position between the pipe W1 and the joined pipe W2 by the matching process as described above. Therefore, the relative position can be detected with high accuracy.

[0073] Therefore, even when the situation inside the structure is difficult for the operator operating in the driver's seat of the work machine 1000 to visually recognize, the operator can easily grasp the situation inside the structure. That is, when the operator relatively moves the pipe W1 and the joined pipe W2, the operator can more accurately grasp the relative position between the pipe W1 and the joined pipe W2 based on the relative position related information D1.

[0074] Thus, the joining work image processing device 62 that generates position information that can streamline the joining work between the pipe W1 and the joined pipe W2 based on the captured image can be realized.

[0075] According to the above-described joining work support system SYS1, when the operator performs the joining work of relatively moving the pipe W1 with respect to the joined pipe W2 by the work machine 1000, the operator can grasp the relative position between the pipe W1 and the joined pipe W2.

[0076] Thus, a joining work support system SYS1 that can streamline the joining work between the pipe W1 and the joined pipe W2 is obtained.

[0077] Further, in the above configuration, the imaging device 14 is provided on the pipe joining attachment 1 as the gripping device. By imaging the pipe W1 and the like with the imaging device 14 provided on the pipe joining attachment 1, the pipe W1 and the like can be imaged at a position closer to the pipe W1 and the like. Based on the captured image PT1 generated by imaging the pipe W1 at a position closer to the pipe W1, the joining work image processing device 62 generates the relative position related information D1. Thereby, the accuracy of the relative position related information D1 can be further improved.

[0078] (Cooperation Work Image Processing Program) The various functions of the cooperation work image processing apparatus 62 can be realized by a computer program. The cooperation work image processing program may be a program that causes a computer to execute the processes of the imaging image acquisition unit 621 and the position detection unit 622 in the cooperation work image processing apparatus 62. From another perspective, the cooperation work image processing program causes the cooperation work image processing apparatus 62 to perform an imaging image acquisition step of acquiring the imaging image PT1 from the imaging apparatus 14, a position detection step of detecting the relative position between the pipe W1 and the pipe W2 to be joined based on the imaging image PT1, and a relative position related information output step of generating and outputting relative position related information D1 regarding the relative position based on the detection result of the position detection step. Any program may be used as long as it meets these requirements.

[0079] Thus, a program can be realized that, when joining the pipe W1 and the pipe W2 to be joined, detects the relative position between the pipe W1 and the pipe W2 to be joined based on the imaging image PT1 and outputs it as relative position related information D1. Therefore, by executing the above program, a cooperation work image processing apparatus 62 can be realized that generates position information that can improve the efficiency of the cooperation work between the pipe W1 and the pipe W2 to be joined based on the imaging image.

[0080] [Embodiment 2] FIG. 4 is a diagram showing a schematic configuration of a cooperation work support system SYS2 including a cooperation work image processing apparatus 65 according to Embodiment 2. FIGS. 5 and 6 are diagrams showing a schematic configuration of the pipe joining attachment 2. FIG. 5 is a view of the pipe joining attachment 2 as seen from one side in the width direction. FIG. 6 is a view of the pipe joining attachment 2 as seen from one side in the first axis direction.

[0081] The cooperation work image processing apparatus 65 according to Embodiment 2 is different from the cooperation work image processing apparatus 62 according to Embodiment 1 in that shape data of a three-dimensional model is used for detecting the relative position and the relative position of the guide member 17 of the pipe joining attachment 2 is also detected. In the following description, the same reference numerals are given to the same configurations as those in Embodiment 1, and the description thereof is omitted, and only the differences from Embodiment 1 are described.

[0082] (Attachment for pipe connection) The schematic configuration of the attachment 2 for pipe connection will be described with reference to FIGS. 4 to 6. The attachment 2 for pipe connection includes an arm connection part 11, a gripping part 12, a support unit 13, an imaging device 15, and a guide member 17.

[0083] A support unit 13, which will be described later, is connected to the lower part of the arm connection part 11 so as to be rotatable in the horizontal direction about the rotation axis Q.

[0084] The gripping part 12 is supported so as to be movable in the first axial direction by an axial movement mechanism 32, which will be described later. The gripping part 12 has gripping claws 21 for gripping the pipe W1. The gripping claws 21 may be configured to be opened and closed by hydraulic pressure.

[0085] The support unit 13 is supported by the arm connection part 11 so as to be rotatable in the horizontal direction about the rotation axis Q. The support unit 13 includes a support frame 31 and an axial movement mechanism 32.

[0086] The support frame 31 includes an upper plate 311, a stay part 312, and a front end part 313. The upper plate 311 is formed in a flat plate shape and is connected to the arm connection part 11 on its upper surface. The stay part 312 is a pair of flat plate-like members, which are connected to the lower surface of the upper plate 311 and extend downward and forward. The front end part 313 is connected to the front end of the stay part 312.

[0087] The axial movement mechanism 32 moves the gripping part 12 in the first axial direction of the pipe W1 gripped by the gripping part 12. The axial movement mechanism 32 includes a slide mechanism 321 and an axial drive part 322.

[0088] The slide mechanism 321 is provided on the lower surface of the upper plate 311. The slide mechanism 321 supports the gripping portion 12 so as to be movable in the first axial direction with respect to the upper plate 311. The slide mechanism 321 can be realized by, for example, a guide rail or the like.

[0089] One end portion of the axial drive portion 322 in the first axial direction is connected to the stay portion 312 via the connecting portion 315, and the other end portion of the axial drive portion 322 in the first axial direction is connected to the front portion of the gripping portion 12. For example, the axial drive portion 322 is a hydraulic cylinder. This hydraulic cylinder is supplied with, for example, hydraulic pressure output from the work machine 1000. The axial drive portion 322 supplies a driving force for moving the gripping portion 12 in the first axial direction according to the supplied hydraulic pressure.

[0090] The guide member 17 is a member for positioning the pipe W1 with respect to the pipe to be joined W2. The guide member 17 is connected to the lower portion of the front end portion 313.

[0091] The guide member 17 is supported by the support frame 31 so as to be located on one side in the first axial direction of the insertion port W11 of the pipe W1 with respect to the gripping portion 12. The guide member 17 has a plate-like portion 173 and a pipe-to-be-joined contact portion 175. The plate-like portion 173 is connected to the lower portion of the front end portion 313 and extends downward. The pipe-to-be-joined contact portion 175 is connected to the lower end portion of the plate-like portion 173. When positioning the insertion port W11 of the pipe W1 gripped by the gripping portion 12 with respect to the receiving port W21 of the pipe to be joined W2, the pipe-to-be-joined contact portion 175 contacts the receiving port W21 side of the pipe to be joined W2. Note that the receiving port W21 side of the pipe to be joined W2 includes not only the receiving port W21 of the pipe to be joined W2 but also the receiving port W21 side of the pipe body in the pipe to be joined W2.

[0092] The pipe-to-be-joined contact portion 175 has an inverted V-shaped shape in which the interval between the pipe-to-be-joined contact portions 175 expands as it goes downward. The pipe-to-be-joined contact portion 175 is flat, with the upper end side connected and the lower end side spaced apart in the width direction. That is, the upper end side of the pipe-to-be-joined contact portion 175 is fixed to the lower end portion of the plate-like portion 173.

[0093] Since the joint pipe contact portion 175 has such a configuration, even when the outer diameter on the receiving port W21 side of the joint pipe W2 is different, the joint pipe contact portion 175 can be brought into contact with the outer peripheral surface on the receiving port W21 side of the joint pipe W2.

[0094] The joint pipe contact portion 175 contacts a portion of the outer peripheral surface on the receiving port W21 side of the joint pipe W2 that is located in an oblique direction (for example, a direction of 45 degrees with respect to the width direction) with respect to the width direction when the receiving port W21 of the joint pipe W2 is viewed in the axial direction. Thereby, the pipe joining attachment 2 is positioned in the width direction and in a direction orthogonal to the width direction with respect to the receiving port W21 of the joint pipe W2.

[0095] Note that the joint pipe contact portion 175 may contact a portion located on one or the other side in the width direction of the outer peripheral surface on the receiving port W21 side of the joint pipe W2, or may contact a portion located in a direction orthogonal to the width direction of the outer peripheral surface on the receiving port W21 side of the joint pipe W2. That is, the joint pipe contact portion 175 only needs to contact a portion located on at least one of the width direction and the direction orthogonal to the width direction of the outer peripheral surface on the receiving port W21 side of the joint pipe W2 when the receiving port W21 is viewed in the axial direction of the joint pipe W2. One of the lower end sides of the joint pipe contact portion 175 may contact the outer peripheral surface on the receiving port W21 side of the joint pipe W2.

[0096] The imaging device 15 is supported by the support frame 31 with respect to the pipe joint attachment 2 so as to be located in one of the first axis directions with respect to the holding portion 12 and above the holding claws 21 of the holding portion 12. The imaging device 15 is connected to the support frame 31 via the imaging device fixing portion 316 of the stay portion 312. The imaging device 15 has a lower imaging range R1 in one of the first axis directions. That is, the optical axis of the camera of the imaging device 15 is in one of the first axis directions and faces downward. Therefore, the imaging device 15 can image a range including at least a part of each of the pipe W1 and the pipe to be joined W2 and at least a part of the guide member 17 to generate an imaging image PT1. Further, the imaging device 15 is a monocular camera. When the imaging image PT1 is generated by the imaging device 15 which is a monocular camera, the data amount of the imaging image PT1 can be reduced as compared with a multi-eye camera such as a stereo camera. Thereby, the calculation load when generating the relative position related information D1 in the joining work image processing device 65 can be reduced. The details of the generation process of the relative position related information D1 in the joining work image processing device 65 will be described later.

[0097] (Information processing terminal) FIG. 7 is a functional block diagram showing a schematic configuration of an information processing terminal 60 having a joining work image processing device 65 according to Embodiment 2. FIG. 8 is a schematic diagram showing the data structure of the shape related information D10.

[0098] The joining work image processing device 65 includes a memory 625, an imaging image acquisition unit 621, a position detection unit 626, and a relative position related information generation unit 627.

[0099] The memory 625 stores shape related information D10 which is information regarding the shapes of the pipe W1 and the pipe to be joined W2 respectively.

[0100] The shape-related information D10 includes the shape data of the three-dimensional model. The shape-related information D10 includes, for example, an insertion port model D11, a guide member model D12, and a receiving port model D13. The insertion port model D11 is the shape data of the three-dimensional model of the insertion port W11 as the joining end of the pipe W1. The guide member model D12 is the shape data of the three-dimensional model of the joined pipe contact portion 175 of the guide member 17. The receiving port model D13 is the shape data of the three-dimensional model of the receiving port W21 as the joined end of the joined pipe W2. As shown in FIG. 8, the insertion port model D11, the guide member model D12, and the receiving port model D13 are three-dimensional model data that can be rotated around the X-axis, Y-axis, and Z-axis with the intersection of the X-axis, Y-axis, and Z-axis as the reference point. Note that the insertion port model D11 and the receiving port model D13 may be stored in the memory 625 for each diameter of the pipe W1 and the joined pipe W2. Also, the X-axis direction in the receiving port model D13 may coincide with the first axis direction of the pipe W1. Further, the X-axis direction in the insertion port model D11 may coincide with the second axis direction of the joined pipe W2.

[0101] The position detection unit 626 detects the relative positions of the guide member 17, the pipe W1, and the joined pipe W2 by performing a matching process between the shapes of the pipe W1, the guide member 17, and the joined pipe W2 imaged in the captured image PT1 and the shapes of the insertion port model D11 of the pipe W1, the guide member model D12 of the guide member 17, and the receiving port model D13 of the joined pipe W2 in the shape-related information D10 stored in the memory 625. Details of the matching process in the position detection unit 626 will be described later.

[0102] Based on the detection result of the position detection unit 626, the relative position-related information generation unit 627 generates and outputs, as the relative position-related information D1, for example, the relative distances and relative angles of the pipe PTW1, the guide member PT17, and the joined pipe PTW2, and a display image PT2 in which the pipe PTW1, the guide member PT17, and the joined pipe PTW2 imaged in the captured image PT1 are highlighted. Details of the relative position-related information D1 output by the relative position-related information generation unit 627 will be described later.

[0103] (Matching Process and Relative Position - Related Information) Hereinafter, with reference to FIGS. 9 to 13, the matching process by the position detection unit 626 and the relative position - related information D1 output by the relative position - related information generation unit 627 will be described. FIG. 9 is a schematic diagram for explaining the matching process when the pipe W1 and the pipe W2 to be joined are in separated positions. FIG. 10 is a diagram showing the captured image PT1 captured in the positional relationship shown in FIG. 9. FIG. 11 is a schematic diagram showing the relative position - related information D1 output as a result of the matching process for the captured image PT1 shown in FIG. 10. FIG. 12 is a schematic diagram for explaining the matching process when the first axial direction of the pipe W1 and the second axial direction of the pipe W2 to be joined are in the same position in the vertical direction. FIG. 13 is a schematic diagram showing the relative position - related information D1 output as a result of the matching process for the captured image captured in the positional relationship shown in FIG. 12.

[0104] As shown in FIG. 9, in the first axial direction, the guide member 17 is located on the other side of the receiving port W21 of the pipe W2 to be joined, and in the vertical direction, when the pipe W1 gripped by the gripping portion 12 is located above the pipe W2 to be joined, the upper part of the insertion port W11 of the pipe W1 and the guide member 17 enter the imaging range R1. On the other hand, the receiving port W21 of the pipe W2 to be joined is located in the blind spot R2 of the camera blocked by the pipe W1. Therefore, the upper part of the receiving port W21 of the pipe W2 to be joined is imaged in a state where at least a part of it is hidden by the pipe W1.

[0105] The position detection unit 626 performs a matching process on the captured image PT1 shown in FIG. 10 using the shape-related information D10 including the shape data of the three-dimensional model. As the matching process, a pattern matching process of the prior art or the like can be adopted. The position detection unit 626 performs a matching process on the captured image PT1 using the shape when the guide member model D12 is viewed in the optical axis direction of the camera of the imaging device 15. As a result, the shape of the guide member PT17 in the captured image PT1 matches. Note that the matching process of the guide member PT17 can be performed based on the feature amount related to the inverted V-shaped shape of the guide member 17. When the guide member 17 has a shape with other features such as a through hole, the matching process may be performed using the feature amount.

[0106] Further, the position detection unit 626 performs a matching process on the captured image PT1 using the shape when the insertion port model D11 of the pipe W1 is viewed in the optical axis direction of the camera of the imaging device 15. As a result, the shape of the insertion port PTW11 of the pipe PTW1 in the captured image PT1 matches.

[0107] Further, the position detection unit 626 performs a matching process on the captured image PT1 using the shape when the receiving port model D13 of the pipe W2 to be joined is viewed in the optical axis direction of the camera of the imaging device 15. However, in the captured image PT1, the receiving port PTW21 of the pipe PTW2 to be joined is partially hidden and thus does not match.

[0108] The position detection unit 626 detects the relative position based on the result of the matching process. Since the position detection unit 626 performs a matching process on the captured image PT1 based on the three-dimensional shape data, it can detect the three-dimensional position and angle between the insertion port W11 of the tube W1 and the guide member 17. In detecting the relative position, the position detection unit 626 can use, for example, the imaging device 15 as the reference point P1. The position detection unit 626 detects the relative position of the insertion port W11 with respect to the reference point P1 on the three axes of the X-axis, Y-axis, and Z-axis based on the positional relationship between the reference point P1 and the reference point of the insertion port model D11 of the tube W1. The position detection unit 626 detects the relative position of the insertion port W11 with respect to the reference point P1 on the three axes of the X-axis, Y-axis, and Z-axis based on the positional relationship between the reference point P1 and the reference point of the guide member model D12.

[0109] The position detection unit 626 detects, for example, the relative angle of the X-axis of the insertion port W11 of the tube W1 and the guide member 17 with respect to the optical axis of the camera. The relative angle means the angle formed between the optical axis of the camera and the straight line extending in the X-axis direction. The angle can be expressed by the rotation angles around the X-axis, Y-axis, and Z-axis of the straight line extending in the X-axis direction with respect to the optical axis of the camera. Note that the position detection unit 626 may detect the relative angle between the axial direction of the tube W1 and the axial direction of the tube W2 to be joined as the relative position.

[0110] Note that the expression of the relative position and relative angle is not limited to the above expression, and known expression methods can be applied.

[0111] As shown in FIG. 11, the relative position-related information generation unit 627 outputs the relative distance and relative angle between the tube PTW1 and the guide member PT17 detected by the matching process of the position detection unit 626. Further, the relative position-related information generation unit 627 generates and outputs a display image PT2 in which the insertion port PTW11 of the tube PTW1 and the guide member PT17 captured in the captured image PT1 are highlighted. The relative position-related information generation unit 627 may highlight the insertion port PTW11 of the tube PTW1 and the guide member PT17 in different colors.

[0112] As shown in FIG. 12, in the first axis direction, the Z axis of the guide member 17 coincides with the Z axis of the receiving port W21 of the pipe to be joined W2, and in the vertical direction, when the X axis of the pipe W1 gripped by the gripping portion 12 coincides with the X axis of the pipe to be joined W2, the upper portion of the insertion port W11 of the pipe W1, the guide member 17, and the upper portion of the receiving port W21 of the pipe to be joined W2 enter the imaging range R1.

[0113] The position detection unit 626 detects the positions of the insertion port W11 of the pipe W1, the guide member 17, and the receiving port W21 of the pipe to be joined W2 in the captured image PT1 by performing a matching process using the insertion port model D11 of the pipe W1, the guide member model D12, and the receiving port model D13 of the pipe to be joined W2.

[0114] According to the detection result of the position detection unit 626, as shown in FIG. 13, the relative position relationship information generation unit 627 outputs the relative distances and relative angles of the pipe PTW1, the guide member PT17, and the pipe to be joined PTW2 detected by the matching process. Further, the relative position relationship information generation unit 627 generates and outputs a display image PT2 in which the pipe PTW1, the guide member PT17, and the pipe to be joined PTW2 imaged in the captured image PT1 are highlighted. The relative position relationship information generation unit 627 may highlight the insertion port PTW11 of the pipe PTW1, the guide member PT17, and the receiving port PTW21 of the pipe to be joined PTW2 in different colors. Further, when the Z axis of the guide member 17 coincides with the Z axis of the receiving port W21 of the pipe to be joined W2 and the X axis of the pipe W1 coincides with the X axis of the pipe to be joined W2, the relative position relationship information generation unit 627 may add a guide display PT21 indicating the width of the pipe W1 to the display image PT2.

[0115] In the above configuration, when performing the joining operation of the pipe members joined in the axial direction, the position detection unit 626 detects the relative position between the insertion port W11 as the joining end of one of the pipe members, the pipe W1, and the receiving port W21 as the joined end of the other pipe member, the pipe W2 to be joined. The position detection unit 626 may detect the relative distance between the insertion port W11 of the pipe W1 and the receiving port W21 of the pipe W2 to be joined as the relative position. Further, the position detection unit 626 may detect the relative angle between the axial direction of the pipe W1 and the axial direction of the pipe W2 to be joined as the relative position. Since the relative position related information generation unit 627 generates and outputs the relative position related information D1 based on the relative position, the operator can easily grasp the relative position of the portion where the pipe W1 and the pipe W2 to be joined are joined.

[0116] Moreover, since the position detection unit 626 detects a part of the pipe W1 and the pipe W2 to be joined, the processing load on the joining operation image processing apparatus 65 can be reduced as compared with the case of detecting the entire pipe W1 and the pipe W2 to be joined.

[0117] Therefore, when generating the relative position related information D1, the processing load on the joining operation image processing apparatus 65 can be reduced.

[0118] According to the above configuration, the relative position between the pipe W1 and the pipe W2 to be joined can be detected by the matching process between the captured image PT1 and the model included in the shape related information D10. Therefore, the relative position can be detected with high accuracy.

[0119] In the above configuration, the position detection unit 626 performs pattern matching processing on the captured image PT1 shown in FIG. 10 using the shape related information D10 including the shape data of the three-dimensional model, thereby detecting the relative position between the pipe W1 and the pipe W2 to be joined. Thereby, the relative position can be detected more accurately. Therefore, it is possible to generate relative position related information that can further improve the working efficiency of the joining operation by the operator.

[0120] In the above configuration, the pipe joining attachment 2 of the work machine 1000 has a guide member 17 for positioning the pipe W1 with respect to the pipe W2 to be joined. The imaging device 15 acquires an imaging image PT1 including at least a part of each of the pipe W1 and the pipe W2 to be joined and at least a part of the guide member 17. The position detection unit 626 detects the relative position between the pipe W1 and the pipe W2 to be joined based on the relative position between the guide member 17 and the pipe W1 or the pipe W2 to be joined.

[0121] Since the imaging device 15 and the guide member 17 are supported with respect to the pipe joining attachment 2 by the support frame 31, the position of the guide member PT17 in the imaging image PT1 is basically the same. Therefore, in the matching process of the shape of the guide member PT17 in the imaging image PT1, the search range for matching can be limited to the range where the guide member PT17 should be reflected. Thereby, the processing load of the matching process of the guide member PT17 can be reduced.

[0122] The position of the insertion port PTW11 of the pipe PTW1 in the imaging image PT1 varies depending on the position where the gripping part 12 grips the pipe W1. However, the variation in the position of the insertion port PTW11 of the pipe PTW1 in the imaging image PT1 is basically assumed to be a variation in the first axis direction. Therefore, the search range for matching can be limited to the range of the variation. Thereby, the processing load of the matching process of the insertion port PTW11 of the pipe PTW1 can be reduced. Further, the matching process of the insertion port PTW11 of the pipe PTW1 may be performed based on the detected position of the guide member PT17. Thereby, the insertion port PTW11 of the pipe PTW1 can be detected more accurately.

[0123] The search range of the receiving port PTW21 of the pipe PTW2 to be joined in the imaging image PT1 may be determined based on the detected position of at least one of the insertion port PTW11 of the pipe PTW1 and the guide member PT17. For example, the search range of the receiving port PTW21 of the pipe PTW2 to be joined in the imaging image PT1 may be limited between the insertion port PTW11 of the pipe PTW1 and the guide member PT17.

[0124] In the above configuration, since the relative position between the pipe W1 and the guide member 17 or the relative position between the pipe to be joined W2 and the guide member 17 is detected, the accuracy of the relative position-related information D1 can be further improved.

[0125] (Joining operation image processing program) The various functions of the joining operation image processing apparatus 65 can be realized by a computer program. The joining operation image processing program may be a program that causes the joining operation image processing apparatus 65 to execute the following steps.

[0126] That is, the joining operation image processing program causes the joining operation image processing apparatus 65 to execute an imaging image acquisition step of acquiring the captured image PT1 from the imaging apparatus 15, a shape-related information acquisition step of the position detection unit 626 of the joining operation image processing apparatus 65 acquiring shape-related information D10 including the shape data of the three-dimensional models of the pipe W1 and the pipe to be joined W2, respectively, a position detection step of the position detection unit 626 detecting the relative position between the pipe W1 and the pipe to be joined W2 by pattern matching processing between the shapes of the pipe W1 and the pipe to be joined W2 imaged in the captured image PT1 and the shapes of the pipe W1 and the pipe to be joined W2 in the shape-related information D10, and a relative position-related information output step of generating and outputting relative position-related information D1 regarding the relative position based on the detection result of the position detection step. Any program that can achieve this is acceptable.

[0127] Thereby, the relative position between the pipe W1 and the pipe to be joined W2 can be detected by matching processing between the captured image PT1 and the shape-related information D10 of the pipe W1 and the pipe to be joined W2. Therefore, the relative position can be detected with high accuracy.

[0128] Further, in the above-described configuration, in the position detection step, the relative position between the pipe W1 and the pipe to be joined W2 is detected by performing pattern matching processing on the captured image using the shape-related information D10 including the shape data of the three-dimensional model. Thereby, the relative position can be detected with higher accuracy. Therefore, it is possible to generate the relative position-related information D1 that can make the joining work by the operator more efficient.

[0129] [Other Embodiments] As described above, the embodiments of the present invention have been described. However, the above-described embodiments are merely examples for implementing the present invention. Therefore, the present invention is not limited to the above-described embodiments, and the above-described embodiments can be appropriately modified and implemented without departing from the gist thereof.

[0130] In each of the above embodiments, the joining member is a pipe, and the member to be joined is a pipe to be joined. However, the joining member and the member to be joined may be other than pipes. The joining member and the member to be joined may be a column, a beam, a wall, a ceiling, a floor, a frame body, or the like.

[0131] In each of the above embodiments, the imaging devices 14 and 15 image a subject located within the imaging range R1 at a predetermined frame rate. In each of the above embodiments, although not particularly described, the guidance image generation device may generate a guidance image at the same frequency as the frame rate of the imaging device or at a frequency lower than the frame rate.

[0132] In each of the above embodiments, the transfer device is the working machine 1000. However, the transfer device may be a machine other than the working machine.

[0133] In each of the above embodiments, the imaging devices 14 and 15 are provided on the pipe joining attachments 1 and 2 of the working machine 1000. However, the imaging device may be provided on the arm portion of the working machine, or the imaging device may be provided on the boom portion of the working machine. Further, the imaging device may be realized as a device separate from the working machine. The information processing terminal may have the imaging device.

[0134] In each of the above embodiments, the information processing terminal 60 is provided in the working machine main body 1001 of the working machine 1000. However, the information processing terminal may be a separate information terminal from the working machine. The information processing terminal may be a portable terminal such as a notebook PC (personal computer), a tablet terminal, a wearable terminal, or a smartphone. Further, the working machine may be provided with a holder for fixing the portable terminal.

[0135] In each of the above embodiments, the joining operation image processing devices 62 and 65 are included in the information processing terminal 60. However, the joining operation image processing device may be a device separate from the information processing terminal. The joining operation image processing device may be included in the imaging device. Further, the joining operation image processing device may be a server device connected via a communication network to the imaging device.

[0136] In each of the above embodiments, the position detection units 622 and 626 detect the relative position based on the insertion port PTW11 of the pipe PTW1 imaged in the captured image PT1 and the receiving port PTW21 of the pipe to be joined PTW2. However, the position detection unit may detect the relative position based on a portion other than the insertion port of the pipe imaged in the captured image and a portion other than the receiving port of the pipe to be joined. Further, the pipe may have a receiving port, and the pipe to be joined may have an insertion port.

[0137] In each of the above embodiments, the joining operation image processing devices 62 and 65 and the imaging devices 14 and 15 are connected wirelessly so as to be communicable. However, the joining operation image processing device and the imaging device may be connected communicably by wire.

[0138] In Embodiment 2 above, the pipe-to-be-joined contact portion 175 of the guide member 17 has an inverted V-shaped configuration. However, the pipe-to-be-joined contact portion may have a different configuration such as an inverted U-shaped configuration or an inverted T-shaped configuration instead of the inverted V-shaped configuration. Further, the guide member may not have a pipe-to-be-joined contact portion.

[0139] In the second embodiment, the shape-related information D10 includes the shape data of the three-dimensional model. However, the shape-related information may include two-dimensional shape data. The position detection unit may perform matching processing on the captured image based on the two-dimensional shape.

[0140] In the second embodiment, the shape-related information D10 includes the insertion port model D11, the guide member model D12, and the receiving port model D13. Further, the position detection unit 626 performs matching processing on the shapes of the pipe W1, the guide member 17, and the pipe W2 to be joined captured in the captured image PT1 and the shapes of the insertion port model D11 of the pipe W1, the guide member model D12 of the guide member 17, and the receiving port model D13 of the pipe W2 to be joined in the shape-related information D10, thereby detecting the relative positions of the guide member 17, the pipe W1, and the pipe W2 to be joined. However, the guide member data may not be included in the shape-related information. Further, the position detection unit may detect the relative positions of the pipe and the pipe to be joined by performing matching processing on the shapes of the pipe and the pipe to be joined captured in the captured image and the shapes of the pipe and the pipe to be joined in the shape-related information.

[0141] In the second embodiment, the position detection unit 626 performs matching processing on the captured image PT1 using the insertion port model D11, the guide member model D12, and the receiving port model D13. However, the presence or absence of the matching processing of the guide member model D may be switched according to the operation of the operator in the information processing terminal or the like. Further, in the information processing terminal, the shape used for the matching processing may be selectable. For example, the diameters of the insertion port model and the receiving port model may be selectable in the information processing terminal.

[0142] In the second embodiment, the relative position related information generation unit 627 generates the display image PT2 including the guide display PT21. However, the relative position related information generation unit may generate and output a display image (relative position related information) including a guidance image to be displayed on the display device 63 in order to assist an operation related to the joining operation of joining the pipe and the pipe to be joined by the work machine based on the relative position. FIG. 14 is a schematic diagram showing an example of the display image PT3 output as the relative position related information D1.

[0143] In the first axial direction, when the position detection unit 626 detects a relative position indicating that the guide member 17 is located on the other side of the receiving port W21 of the pipe to be joined W2 and, in the vertical direction, the pipe W1 held by the gripping unit 12 is located above the pipe to be joined W2, the relative position related information generation unit 627 determines the operations necessary for the joining operation. To join the pipe W1 to the pipe to be joined W2, the following operations are necessary. First, in the first axial direction, move the pipe joining attachment 2 to one side in the first axial direction. Next, lower the pipe joining attachment 2 until the guide member 17 contacts the upper part of the receiving port W21 of the pipe to be joined W2. Next, move the gripping unit 12 in one direction in the first axial direction by the axial movement mechanism 32.

[0144] As shown in FIG. 14, the relative position related information generation unit 627 outputs the operation guidance display PT31. The operation guidance display PT31 includes a lever operation display PT311 corresponding to the left operation lever 51 and a lever operation display PT312 corresponding to the right operation lever 52.

[0145] For example, in the work machine 1000, the forward and backward operations FR1, RR1 of the left operation lever 51 correspond to pushing and pulling of the arm, and the left and right operations LF1, RG1 correspond to left and right turning. The forward and backward operations FR2, RR2 of the right operation lever 52 correspond to raising and lowering of the boom, and the left and right operations LF2, RG2 correspond to excavation and dumping of the bucket. Note that these operation patterns are examples, and another operation pattern may be adopted in the work machine 1000.

[0146] For example, in the first axis direction, the relative position related information generation unit 627 determines that operations such as pushing the arm and lowering the boom are necessary to move the pipe joint attachment 2 in one direction of the first axis. As shown in FIG. 14, the relative position related information generation unit 627 outputs an operation guidance display PT31 indicating that the front operation FR1 of the left operation lever 51 and the front operation FR2 of the right operation lever 52 are necessary as a guidance image. Further, the relative position related information generation unit 627 outputs a display image PT3 including an arrow PT32 indicating the direction in which the pipe W1 should be moved as a guidance image. Further, the relative position related information generation unit 627 may add an axis display PT33 indicating the first axis direction of the pipe W1 to the display image PT3.

[0147] In the above configuration, the relative position related information generation unit 627 generates and outputs relative position related information D1 including a guidance image. The operation of the working machine 1000 by the operator can be visually assisted by the guidance image representing the guidance information. Therefore, the operator can easily grasp the support content of the joining operation. Thus, the joining operation can be made more efficient.

[0148] In the second embodiment, the relative position related information generation unit 627 generates a display image PT2 including a highlight display as the relative position related information D1. However, the relative position related information generation unit 627 may generate a perspective image of a virtual three-dimensional space viewed from at least one direction according to the detection result of the position detection unit 626. FIG. 15 is a diagram showing an example of the perspective image PT4.

[0149] First, the relative position related information generation unit 627 arranges an insertion port model D11, a guide member model D12, and a receiving port model D13 in a virtual three-dimensional space based on the relative positions and relative angles of the insertion port W11, the guide member 17, and the receiving port W21, respectively. As shown in FIG. 15, the relative position related information generation unit 627 generates a perspective image PT4 including a plan perspective image PT41 of the virtual three-dimensional space in which each model is arranged viewed from one direction in the Z-axis direction, a side perspective image PT42 viewed from one direction in the Y-axis direction, and a front perspective image PT43 viewed from one direction in the X-axis direction.

[0150] The relative position related information generation unit 627 may add an axis PT411 indicating the X-axis direction of the insertion port model D11 and an axis PT412 indicating the X-axis direction of the receiving port model D13 to the plan view image PT41 and the side view image PT42. Further, the relative position related information generation unit 627 may add a virtual straight line PT413 that passes through the outer edge in the radial direction of the receiving port model D13 and is parallel to the X-axis direction of the model of the receiving port model D13 to the plan view image PT41 and the side view image PT42.

[0151] The relative position related information generation unit 627 may add a center point P21 where the X-axis of the insertion port model D11 is located and a center point P22 where the X-axis of the receiving port model D13 is located to the front view image PT43.

[0152] In each of the above embodiments, although not particularly described, in the joining work support system, an operation control device for controlling the operation of the working machine so that the working machine as a conveying device joins the joining member to the member to be joined based on the relative position may be further provided. Note that the relative position related information generation unit may output a control signal to the operation control device based on the detection result of the relative position.

[0153] According to the above configuration, the working machine can be driven without the operator operating it. Therefore, the joining work between the joining member and the member to be joined can be made more efficient.

Industrial Applicability

[0154] The present invention can be used for a joining work image processing device that processes a captured image captured by an imaging device.

Explanation of Signs

[0155] 1, 2 Attachment for pipe joining 11 Arm connection part 12 Gripping part 13 Support unit 14, 15 Imaging device 62, 65 Joining work image processing device 63 indicating device 621 imaging image acquisition unit 622, 626 position detection units 623, 627 relative position related information generation units 625 memory 1000 working machine SYS1, SYS2 joining operation support systems W1 pipe W2 pipe to be joined

Claims

1. An imaging image acquisition unit that acquires an imaging image including at least a part of a joining member and at least a part of a joined member joined to the joining member; A position detection unit that detects a relative position between the joining member and the joined member based on the imaging image; The position detection unit detects the relative position between the joining member and the joined member by a matching process between the shapes of the joining member and the joined member imaged in the imaging image and the shapes of the joining member and the joined member stored in advance; A joining operation image processing apparatus.

2. In the joining operation image processing apparatus according to Claim 1, The joining member and the joined member are each a pipe member extending in the axial direction, and are configured to be connectable in the axial direction, The position detection unit detects, as the relative position, a relative distance between a joining end portion of the joining member and a joined end portion of the joined member joined to the joining end portion of the joining member, and a relative angle between the axial direction of the joining member and the axial direction of the joined member; A joining operation image processing apparatus.

3. In the joining operation image processing apparatus according to Claim 1, It further has a memory that stores shape-related information, which is information regarding the shapes of the joining member and the joined member respectively, The shape-related information includes shape data of a three-dimensional model, The position detection unit detects the relative position between the joining member and the joined member by performing pattern matching processing on the imaging image using the shape-related information; A joining operation image processing apparatus.

4. In the joining operation image processing apparatus according to Claim 1, A relative position-related information generation unit that generates relative position-related information including at least one of information indicating the relative position and guidance information for assisting a joining operation of joining the joining member to the joined member based on a detection result of the position detection unit; A joining operation image processing apparatus.

5. In the joining operation image processing apparatus according to Claim 4, The relative position-related information generation unit generates the relative position-related information including a guidance image to be displayed on a display device to assist the joining operation; A joining operation image processing apparatus.

6. A joining operation image processing apparatus according to any one of Claims 1 to 5, A conveying device that relatively moves the joining member with respect to the joined member; An imaging device that images at least a part of each of the joining member and the member to be joined to generate the captured image, A joining work support system having the same.

7. In the joining work support system according to claim 6, The transfer device has a guide member for positioning the joining member with respect to the member to be joined, The imaging device acquires the captured image including at least a part of the guide member in addition to at least a part of each of the joining member and the member to be joined, The position detection unit detects the relative position between the joining member and the member to be joined based on the relative position between the guide member and the joining member or the member to be joined. Joining work support system.

8. In the joining work support system according to claim 6, The transfer device includes A main body part, A boom part swingable with respect to the main body part, An arm part swingable with respect to the boom part, A gripping device attached to the tip of the arm part and capable of gripping the joining member, Having The imaging device is provided on the gripping device or the arm part. Joining work support system.

9. In the joining work support system according to claim 6, The imaging device is a monocular camera. Joining work support system.

10. In the joining work support system according to claim 6, The imaging device has a communication unit that transmits the captured image to the joining work image processing device by wireless communication. Joining work support system.

11. In the joining work support system according to claim 6, Based on the relative position, it further has an operation control device that controls the operation of the transfer device so that the transfer device joins the joining member to the member to be joined. Joining work support system.

12. A joining work image processing program that causes a computer to execute the processing of the captured image acquisition unit and the position detection unit in the joining work image processing device according to claim 1. Joining work image processing program.

Images