Bonding operation support device, bonding operation support system, bonding operation support method, and program

Abstract

To provide a joining work support device with which an instructor can instruct an operator during joining work.SOLUTION: A joining work support device 10 includes: an image acquisition unit 11 which acquires taken visible taken images and thermal taken images; a temperature detection unit 12 which detects a temperature of a joined portion on the basis of the visible taken images and the thermal taken images; a temperature determination unit 13 which determines propriety of the detected temperature of the joined portion; a 3D data acquisition unit 14 which acquires 3D data showing a 3D shape and a dimension of each of piping members 71, 72 and 73 and a tool 61; and a display control unit 15 which displays joining work support information on display units 23 and 33 on the basis of a determination result from determination of the temperature determination unit 13 and three-dimensionally displays joining work shown through one display device 20 on the display unit 33 of the other display device 30 on the basis of positions and orientations of AR markers 51, 52 and 53 included in one of the acquired visible taken images and the 3D data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a joining operation support device, a joining operation support system, a joining operation support method, and a program for supporting joining operations such as brazing or welding.

Background Art

[0002] Conventionally, a technique of supporting joining operations by presenting joining operation support information to workers during joining operations for supporting joining operations such as brazing or welding has become widespread. For example, Patent Document 1 discloses a technique of supporting a joining operation by presenting an abnormal portion of a joined portion to a worker during the joining operation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, in joining techniques for supporting joining operations such as brazing or welding, there is a shortage of workers with rich work experience. Workers with rich work experience are required to guide workers with little work experience. When guiding a worker in a joining operation, if the instructor can guide the worker during the joining operation, it is considered that the joining technique of the worker will improve quickly. On the other hand, the technique disclosed in Patent Document 1 cannot be used by an instructor to guide a worker during the joining operation.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to provide a joining operation support device that enables an instructor to guide a worker during the joining operation.

Means for Solving the Problems

[0006] The joining work support device according to the present disclosure is provided on each of two display devices, and each visible captured image captured by a visible imaging unit that captures a joining work area including a joining target member and a marker, and each provided on each of the two display devices and a thermal imaging unit that captures the temperature distribution of the joining work area. An image acquisition unit that acquires each thermal captured image, and the image acquisition unit acquires the By overlapping each visible captured image and each thermal captured image respectively, and three-dimensionally connecting the overlapped images with each other, A temperature detection unit that detects the temperature of the joining portion in the joining target member, a temperature determination unit that determines whether or not the temperature of the joining portion detected by the temperature detection unit is acceptable, and from the storage unit, the joining target member, the base material to which the joining target member is joined, and the jig that fixes the joining target member and the base material. A three-dimensional data acquisition unit that acquires three-dimensional data indicating each three-dimensional shape and each dimension, and based on the determination result determined by the temperature determination unit, on the display units respectively included in the two display devices, the joining work support information for supporting the joining work is displayed, and the position and orientation of the marker included in one visible captured image acquired by the image acquisition unit, and based on the three-dimensional data acquired by the three-dimensional data acquisition unit, the joining work visible through one display device is three-dimensionally displayed on the display unit of the other display device. It is provided with a display control unit.

Effects of the Invention

[0007] According to the present disclosure, an instructor can instruct an operator during joining work. An operator and an instructor who instructs this operator can quantitatively observe the joining work that could only be imitated by observing the actions and action timings based on the intuition of skilled workers until now, together with the actual work of the operator. For this reason, the instructor can accurately and in a short time instruct the operator on the joining work. In addition, by quantitatively displaying the joining work support information for the joining work based on the intuition of the operator, it becomes possible to stabilize the joining quality and manage the joining quality.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0010] Embodiment 1. The joining operation support device 10 according to Embodiment 1 will be described with reference to FIGS. 1 to 10.

[0011] Figure 1 is a block diagram showing the configuration of a joining operation support system to which the joining operation support device 10 according to Embodiment 1 is applied. Figure 2 is a diagram showing the configuration of a jig 61 used when joining piping members 71, 72, 73. Figure 3 is a diagram associating the configuration of the display devices 20, 30 with the joining operation.

[0012] The joining work support system shown in FIG. 1 supports a worker performing joining work such as brazing or welding by presenting joining work support information for supporting the joining work to the worker. In the present embodiment, the case where the joining work support system supports brazing work will be described, but it is not limited to the brazing work.

[0013] Here, as shown in FIG. 2, the worker uses the jig 61 to braze the pipe member 71 to be joined to the pipe members 72 and 73 serving as base materials to assemble the pipe 70. At this time, as shown in the upper figure in FIG. 3, the worker supplies the brazing material 63 to the joint portion of the pipe member 71 and heats it with the flame ejected from the gas burner 62.

[0014] Specifically, first, the worker fixes the pipe members 71 and 72 in a vertical state at one end side of the jig 61 and brazes one end of the pipe member 71 to one end of the pipe member 72 (portion I in FIG. 2). Next, the worker fixes the pipe member 71 brazed to the pipe member 72 and the pipe member 73 in a horizontal state at the other end side of the jig 61 and brazes the other end of the pipe member 71 to one end of the pipe member 73 (portion II in FIG. 2).

[0015] Further, a plurality of AR markers 51, 52, and 53 are provided at different positions on the jig 61. FIG. 2 shows an example in which three AR markers 51, 52, and 53 are provided on the jig 61. The AR markers 51, 52, and 53 are composed of, for example, a square black frame portion and a rectangular portion provided inside the frame portion. Different shapes, characters, colors, etc. are drawn on each rectangular portion. Therefore, the AR markers 51, 52, and 53 are distinguishable from each other.

[0016] As shown in FIG. 1, the joining work support system according to Embodiment 1 includes a joining work support device 10, display devices 20 and 30, and a CAD device 40. The joining work support device 10 is a computer connected to the display devices 20 and 30 by wire or wirelessly. Further, the joining work support device 10 is not electrically connected to the CAD device 40. The CAD data created by this CAD device 40 is transferred to the joining work support device 10 using a storage medium or the like, for example, when necessary.

[0017] As shown in FIG. 3, the display devices 20 and 30 are, for example, transmissive smart glasses of the head-mounted display type, and display digital information superimposed on the real image seen through transmission. Such display devices 20 and 30 may display the digital information on the display surface (reflective surface) or directly project it onto the retina of a person. One display device 20 is worn by the worker who performs the joining work. The other display 30 device is worn by the instructor who guides the worker. Note that the display devices 20 and 30 shown in FIG. 3 are configured as seen when facing the brazing work. Also, the instructor who guides the worker is, for example, a skilled worker with rich work experience.

[0018] As shown in FIGS. 1 and 3, the display device 20 has a visible imaging unit 21, a thermal imaging unit 22, and a display unit 23.

[0019] The imaging range of the visible imaging unit 21 overlaps most of the field of view of the worker. The visible imaging unit 21 is, for example, a visible image camera. Therefore, during the joining work of the worker, the visible imaging unit 21 can image the joining work area including at least the piping member 71 fixed to the jig 61 and the AR markers 51, 52, and 53 provided on the jig 61 from the worker's side. Further, the visible imaging unit 21 outputs the captured visible imaging image to the joining work support device 10.

[0020] The thermal imaging unit 22 has a shooting range that overlaps most of the operator's field of view. This thermal imaging unit 22 is, for example, a thermal camera. Therefore, the thermal imaging unit 22 can shoot the temperature distribution of the above-mentioned joining work area from the operator's side. In addition, the thermal imaging unit 22 outputs the captured thermal imaging image to the joining work support device 10.

[0021] The display unit 23 displays joining work support information in the operator's visual field area. The joining work support information is information for supporting the joining work for the operator who performs the joining work. This joining work support information is, for example, the temperature of the joint part in the piping member 71, the insertion time and extraction time (end time of insertion) of the brazing material 63, an alarm due to the temperature rise of the piping member 71, etc. The display unit 23 is controlled by a display control unit 15 described later to display or not display the joining work support information.

[0022] As shown in FIGS. 1 and 3, the display device 30 includes a visible imaging unit 31, a thermal imaging unit 32, and a display unit 33.

[0023] The visible imaging unit 31 has a shooting range that overlaps most of the instructor's field of view. This visible imaging unit 31 is, for example, a visible image camera. Therefore, the visible imaging unit 31 can shoot the joining work area including at least the piping member 71 fixed to the jig 61 and the AR markers 51, 52, 53 provided on the jig 61 from the instructor's side during the operator's joining work. In addition, the visible imaging unit 31 outputs the captured visible imaging image to the joining work support device 10.

[0024] The thermal imaging unit 32 has a shooting range that overlaps most of the instructor's field of view. This thermal imaging unit 32 is, for example, a thermal camera. Therefore, the thermal imaging unit 32 can shoot the above-mentioned joining work area from the instructor's side. In addition, the thermal imaging unit 32 outputs the captured thermal imaging image to the joining work support device 10.

[0025] The display unit 33 displays the joining operation support information in the field of view of the instructor. The same information as the joining operation support information displayed on the display unit 23 is simultaneously displayed on this display unit 33. Also, although details will be described later, the display unit 33 can three-dimensionally display the joining operations of the operators visible through the display device 20. The display or non-display of the display unit 33 is controlled by a display control unit 15, which will be described later.

[0026] The CAD device 40 designs the jig 61, the piping 70, and the piping members 71, 72, 73. This CAD device 40 creates CAD data related to them individually and stores it in the storage unit 41. The CAD data related to the jig 61, the piping 70, and the piping members 71, 72, 73 is three-dimensional data showing the three-dimensional shapes and dimensions of the jig 61, the three-dimensional shapes and dimensions of the piping 70, and the three-dimensional shapes and dimensions of the piping members 71, 72, 73. Note that the storage unit 41 may be provided in the joining operation support device 10.

[0027] On the other hand, the joining operation support device 10 includes an image acquisition unit 11, a temperature detection unit 12, a temperature determination unit 13, a three-dimensional data acquisition unit 14, and a display control unit 15.

[0028] The image acquisition unit 11 acquires the visible captured image captured by the visible imaging unit 21 and the thermal captured image captured by the thermal imaging unit 22. Also, the image acquisition unit 11 acquires the visible captured image captured by the visible imaging unit 31 and the thermal captured image captured by the thermal imaging unit 32. Such an image acquisition unit 11 outputs each visible captured image and each thermal captured image to the temperature detection unit 12.

[0029] The temperature detection unit 12 detects the temperature of the joint portion in the piping member 71 based on each visible captured image and each thermal captured image acquired by the image acquisition unit 11. Also, the temperature detection unit 12 outputs the detected temperature of the joint portion to the temperature determination unit 13.

[0030] Specifically, the temperature detection unit 12 superimposes the visible captured image captured by the visible imaging unit 21 and the thermal captured image captured by the thermal imaging unit 22. Further, the temperature detection unit 12 superimposes the visible captured image captured by the visible imaging unit 31 and the thermal captured image captured by the thermal imaging unit 32. Then, the temperature detection unit 12 three-dimensionally connects the superimposed images and detects the temperature of the joint portion in the pipe member 71 from the three-dimensionally connected image.

[0031] The temperature determination unit 13 determines whether the temperature of the joint portion in the pipe member 71 detected by the temperature detection unit 12 is acceptable. Further, the temperature determination unit 13 outputs the determination result to the display control unit 15. Specifically, the temperature determination unit 13 includes a brazing material temperature determination unit 13a and a base material temperature determination unit 13b.

[0032] The brazing material temperature determination unit 13a determines whether the temperature of the joint portion in the pipe member 71 detected by the temperature detection unit 12 exceeds the melting temperature of the brazing material 63. Further, the base material temperature determination unit 13b determines whether the temperature of the joint portion in the pipe member 71 detected by the temperature detection unit 12 does not exceed the melting temperature of the pipe members 72 and 73 serving as the base material.

[0033] The three-dimensional data acquisition unit 14 acquires three-dimensional data indicating the three-dimensional shape and dimensions of the pipe member 71, the three-dimensional shape and dimensions of the pipe members 72 and 73 to which the pipe member 71 is joined, and the three-dimensional shape and dimensions of the jig 61 that fixes the pipe members 71, 72, and 73 from the storage unit 41 of the CAD device 40. Further, the three-dimensional data acquisition unit 14 outputs the acquired three-dimensional data to the display control unit 15.

[0034] Based on the determination result determined by the temperature determination unit 13, the display control unit 15 displays joint operation support information for assisting the joint operation on the display units 23 and 33 respectively included in the two display devices 20 and 30.

[0035] Specifically, when the brazing material temperature determination unit 13a determines that the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 exceeds the melting temperature of the brazing material 63 and reaches a preset optimal melting temperature, the display control unit 15 displays joining operation support information on each of the display units 23, 33. That is, since the brazing material 63 melts appropriately and good brazing is being performed, the display control unit 15 presents joining operation support information for assisting the operator in the brazing operation.

[0036] Also, when the base material temperature determination unit 13b determines that the temperature of the joint in the piping member 71 detected by the temperature detection unit 12 is a temperature immediately before the melting temperature of the piping members 72, 73 serving as the base material, the display control unit 15 displays an alarm on each of the display units 23, 33. That is, since good brazing cannot be performed if the base material melts, the display control unit 15 alerts the operator before the base material melts.

[0037] Furthermore, based on the positions and orientations of the AR markers 51, 52, 53 included in one of the visible captured images acquired by the image acquisition unit 11 and the 3D data acquired by the 3D data acquisition unit 14, the display control unit 15 three-dimensionally displays the joining operation visible through one display device 20 on the display unit 33 of the other display device 30.

[0038] Specifically, the display control unit 15 calculates the 3D coordinates and orientation of the display device 20 based on the positions and orientations of the AR markers 51, 52, 53 included in the visible captured image captured by the visible imaging unit 21 and the 3D data. Next, based on the calculation result and the visible captured image captured by the visible imaging unit 21, the display control unit 15 three-dimensionally displays the joining operation visible to the operator through the display device 20 on the display unit 33 of the display device 30 worn by the instructor.

[0039] Therefore, the instructor can pseudo-view the operator's joining operation from the same direction as the operator is looking at the joining operation area. As a result, the instructor can provide detailed guidance on the operator's joining operation. In addition, the joining operation support device 10 can project the joining operation visible to the operator via the display device 20 in real time onto a display device different from the display devices 20 and 30, so that the instructor can guide the operator. Furthermore, the joining operation support device 10 can record the joining operation visible to the operator via the display device 20 and project the recorded joining operation onto a display device different from the display devices 20 and 30, so that the instructor can guide the operator.

[0040] On the other hand, the operator can quantitatively confirm the joining operation, which was previously only imitable by observing the actions of skilled workers based on intuition, together with the actual joining operation. Therefore, the operator can acquire a high joining technique equivalent to that of skilled workers with accurate and short-time guidance.

[0041] FIG. 4 is a diagram showing the display state of the display device 30 worn by the instructor. FIGS. 4A to 4D show a state in which a three-dimensional image of the joining operation visible to the operator via the display device 20 and joining operation support information or an alarm are displayed on the display unit 33 of the display device 30. As described above, the joining operation support information or alarm displayed on the display unit 33 is the same as that displayed on the display unit 23. In addition, in FIGS. 4A to 4D, the visible imaging unit 31 and the thermal imaging unit 32 are omitted.

[0042] The joining operation support information shown in FIG. 4A is the temperature of the joint in the piping member 71 and the heating instruction for the corresponding joint. The temperature is displayed in a graph and numbers. The joining operation support information shown in FIG. 4B is the temperature of the joint in the piping member 71 and the brazing material insertion start instruction corresponding thereto. The joining operation support information shown in FIG. 4C is the temperature of the joint in the piping member 71 and the brazing material insertion end instruction corresponding thereto. The joining operation support information shown in FIG. 4D is the temperature of the joint in the piping member 71. Correspondingly, an alarm indicating that the temperature of the joint is rising is displayed on the display unit 33.

[0043] FIG. 5 is a diagram showing the brazing process in order. FIG. 5D is a longitudinal sectional view of FIG. 5C.

[0044] The operator brazes the piping member 71 to the piping member 72 while looking at the joining operation support information displayed on the display unit 23 of the display device 20. As shown in FIGS. 5A to 5D, for example, the brazing operation performed by the operator is to melt the brazing material 63 into the gap formed between them in a state where one end of the piping member 72 is inserted into one end of the piping member 71. Supply to do. Therefore, the brazing material 63 that protrudes from the gap forms a fillet. This fillet is formed over the entire circumferential direction of the piping members 71 and 72. Such an operator's brazing operation is displayed as a three-dimensional image on the display unit 33 of the display device 30 worn by the instructor.

[0045] At this time, when the display control unit 15 determines that the fillet of the brazing material 63 is properly formed based on the two visible captured images acquired by the image acquisition unit 11, the display units 23 and 33 may display that the brazing operation has been completed.

[0046] FIG. 6 is a diagram showing a state where brazing is not properly performed. FIG. 6A shows a state where the brazing material 63 is not sufficiently supplied to the gap between the piping members 71 and 72 and the fillet is not continuous. FIG. 6B shows a state where the brazing material 63 is excessively supplied to the gap between the piping members 71 and 72 and the fillet is not properly formed in an annular shape.

[0047] At this time, when the display control unit 15 determines that a brazing defect has occurred as shown in FIGS. 6A and 6B using the two visible captured images acquired by the image acquisition unit 11, the display control unit 15 may display on the displays 23 and 33 that a brazing defect has occurred.

[0048] FIG. 7 is a diagram showing the configuration of a pipe 70 having a valve portion 74. As shown in this FIG. 7, the pipe 70 may have a valve portion 74. This valve portion 74 incorporates a pressure sensor, a solenoid valve, etc., and is a part that may be damaged when kept at a high temperature. In particular, when the piping members 71, 72, and 73 are formed of aluminum, copper, or the like, since those materials have good heat conduction, the heat generated during the brazing of the piping member 71 may be transmitted to the valve portion 74.

[0049] Therefore, as shown in FIG. 7, the joining operation support device 10 is shown in a case where it includes a coolant supply pipe 81. This coolant supply pipe 81 supplies coolant to the outside of the valve portion 74. The coolant is, for example, water or oil. When the joining operation by the operator is started, the joining operation support device 10 starts to supply coolant to the outside of the valve portion 74 using the coolant supply pipe 81. The supply of this coolant continues until the joining operation is completed. At this time, the joining operation support device 10 determines the start and completion of the joining operation by the operator based on the visible captured image and the thermal captured image acquired by the image acquisition unit 11.

[0050] Then, when the joining operation support device 10 determines based on the visible captured image and the thermal captured image acquired by the image acquisition unit 11 that the temperature of the valve portion 74 has exceeded a preset temperature, the display unit 23, 33 74Displays that the temperature is rising. Therefore, the joining work support device 10 can prevent the valve portion 74 from being damaged by the heat generated during the brazing of the piping member 71.

[0051] FIG. 8 is a diagram showing the temperature change of the joint portion. The vertical axis of FIG. 8 indicates the temperature of the joint portion. The horizontal axis of FIG. 8 indicates the passage of time. The solid line shown in FIG. 8 is the temperature ideal curve of the temperature change of the joint portion. The two-dot chain line shown in FIG. 8 is the temperature upper limit curve of the temperature change of the joint portion. The one-dot chain line shown in FIG. 8 is the temperature lower limit curve of the temperature change of the joint portion.

[0052] The joining work support device 10 calculates in advance a temperature ideal curve, a temperature upper limit curve, and a temperature lower limit curve corresponding to the joint portion of the piping member 71 based on the diameters and thicknesses of the piping members 71, 72, and 73. Then, the joining work support device 10 detects the temperature change of the joint portion based on the two visible captured images and the two thermal captured images acquired by the image acquisition unit 11, and before the detected temperature exceeds the temperature upper limit curve or the temperature lower limit curve, the display units 23 and 33 display that the temperature of the joint portion is rising or that the temperature of the joint portion is falling.

[0053] Next, the joining work support method according to Embodiment 1 will be described with reference to FIG. 9. FIG. 9 is a flowchart showing the joining work support method according to Embodiment 1.

[0054] In step ST11, the image acquisition unit 11 acquires each visible captured image captured by the visible imaging units 21 and 31 and each thermal captured image captured by the thermal imaging units 22 and 32.

[0055] In step ST12, the temperature detection unit 12 detects the temperature of the joint portion in the piping member 71 based on each visible captured image and each thermal captured image acquired by the image acquisition unit 11.

[0056] In step ST13, the temperature determination unit 13 determines the acceptability of the temperature of the joint portion detected by the temperature detection unit 12.

[0057] In step ST14, the three-dimensional data acquisition unit 14 acquires three-dimensional data indicating the three-dimensional shape and dimensions of the piping member 71, the three-dimensional shapes and dimensions of the piping members 72 and 73 to which the piping member 71 is joined, and the three-dimensional shape and dimensions of the jig 61 that fixes the piping members 71, 72, and 72 from the storage unit 41 of the CAD device 40.

[0058] In step ST15, the display control unit 15 displays joining operation support information on the display units 23 and 33 of the display devices 20 and 30 based on the determination result determined by the temperature determination unit 13. Further, the display control unit 15 three-dimensionally displays the joining operation visible from the display device 20 on the display unit 33 of the display device 30. Then, the joining operation support method ends.

[0059] Next, FIG. 10A is a block diagram showing a hardware configuration for realizing the functions of the joining operation support device 10 according to the first embodiment. FIG. 10B is a block diagram showing a hardware configuration for executing software for realizing the functions of the joining operation support device 10 according to the first embodiment.

[0060] In FIGS. 10A and 10B, the input interface 104 is an interface that relays the visible captured image and the thermal captured image output from the display devices 20 and 30 to the joining operation support device 10, and the three-dimensional data output from the CAD device 40 to the joining operation support device 10. Further, the output interface 105 is an interface that relays the display control signal output from the joining operation support device 10 to the display devices 20 and 30.

[0061] When the processing circuit is the dedicated hardware processing circuit 101 shown in FIG. 10A, the processing circuit 101 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15 provided in the bonding operation support device 10 may be realized by separate processing circuits, or these functions may be realized by a single processing circuit.

[0062] When the processing circuit is the processor 102 shown in FIG. 10B, the functions of the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15 provided in the bonding operation support device 10 are realized by software, firmware, or a combination of software and firmware. Note that the software or firmware is described as a program and stored in the memory 103.

[0063] The processor 102 reads and executes the program stored in the memory 103 to realize the functions of the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15 provided in the bonding operation support device 10. For example, the bonding operation support device 10 includes a memory 103 for storing a program that, when executed by the processor 102, results in the execution of the processing from step ST11 to step ST15 shown in FIG. 9. These programs cause a computer to execute the procedures or methods of the processing performed by the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15. The memory 103 may be a computer-readable storage medium storing a program for causing a computer to function as the image acquisition unit 11, the temperature detection unit 12, the temperature determination unit 13, the three-dimensional data acquisition unit 14, and the display control unit 15.

[0064] The memory 103 corresponds to, for example, non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), magnetic disks, flexible disks, optical disks, compact disks, mini-disks, DVDs, etc.

[0065] Some of the functions of the image acquisition unit 11, temperature detection unit 12, temperature determination unit 13, three-dimensional data acquisition unit 14, and display control unit 15 included in the bonding operation support device 10 may be realized by dedicated hardware and some may be realized by software or firmware. For example, the image acquisition unit 11, temperature detection unit 12, temperature determination unit 13, and three-dimensional data acquisition unit 14 realize their functions by a processing circuit 101 which is dedicated hardware, and the display control unit 15 realizes its function by the processor 102 reading and executing a program stored in the memory 103. Thus, the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.

[0066] As described above, the joining work support device 10 according to Embodiment 1 is provided on each of the two display devices 20 and 30, and each visible captured image captured by the visible imaging units 21 and 21 that capture the joining work area including the piping member 71 and the AR markers 51, 52, and 53, and each thermal captured image captured by the thermal imaging units 22 and 32 that are provided on each of the two display devices 20 and 30 and capture the temperature distribution of the joining work area. An image acquisition unit 11 that acquires the images; a temperature detection unit 12 that detects the temperature of the joint portion in the piping member 71 based on each visible captured image and each thermal captured image acquired by the image acquisition unit 11; a temperature determination unit 13 that determines whether the temperature of the joint portion detected by the temperature detection unit 12 is acceptable; a three-dimensional data acquisition unit 14 that acquires three-dimensional data indicating each three-dimensional shape and each dimension of the piping member 71, the piping members 72 and 73 to which the piping member 71 is joined, and the jig 61 that fixes the piping members 71, 72, and 73 from the storage unit 41; and based on the determination result determined by the temperature determination unit 13, the display units 23 and 33 respectively included in the two display devices 20 and 30 display joining work support information for supporting the joining work, and based on the positions and orientations of the AR markers 51, 52, and 53 included in one of the visible captured images acquired by the image acquisition unit 11 and the three-dimensional data acquired by the three-dimensional data acquisition unit 14, the joining work visible through one of the display devices 20 is three-dimensionally displayed on the display unit 33 of the other display device 30. A display control unit 15. Therefore, the joining work support device 10 enables the instructor to guide the operator during the joining work.

[0067] In the joining work support device 10, the temperature determination unit 13 has a solder temperature determination unit 13a that determines whether the temperature of the joint portion detected by the temperature detection unit 12 exceeds the melting temperature of the solder 63. When the solder temperature determination unit 13a determines that the temperature of the joint portion detected by the temperature detection unit 12 exceeds the melting temperature of the solder 63, the display control unit 15 displays the joining work support information on each of the display units 23 and 33. Therefore, when the joining work is being properly performed, the joining work support device 10 can immediately present the joining work support information to the operator.

[0068] In the joining operation support device 10, the temperature determination unit 13 includes a base material temperature determination unit 13b that determines whether or not the temperature of the joint portion detected by the temperature detection unit 12 exceeds the melting temperature of the pipe members 72 and 73 serving as the base materials. When the display control unit 15 determines, based on the base material temperature determination unit 13b, that the temperature of the joint portion detected by the temperature detection unit 12 is the temperature immediately before the melting temperature of the pipe members 72 and 72 serving as the base materials, the display control unit 15 displays an alarm on each of the display units 23 and 33. Therefore, when the joining operation is not properly performed, the joining operation support device 10 can immediately present an alarm to the operator.

[0069] Note that within the scope of the present disclosure, any component of the embodiment can be modified or any component of the embodiment can be omitted.

Explanation of Reference Numerals

[0070] 10 Joining operation support device, 11 Image acquisition unit, 12 Temperature detection unit, 13 Temperature determination unit, 13a Brazing material temperature determination unit, 13b Base material temperature determination unit, 14 Three-dimensional data acquisition unit, 15 Display control unit, 20, 30 Display device, 21, 31 Visible imaging unit, 22, 32 Thermal imaging unit, 23, 33 Display unit, 40 CAD device 41 Storage unit, 51 to 53 AR marker, 61 Fixture, 62 Gas burner, 63 Brazing material, 70 Pipe, 71, 72, 73 Pipe member, 74 Valve portion, 81 Coolant supply pipe, 101 Processing circuit, 102 Processor, 103 Memory, 104 Input interface, 105 Output interface.

Claims

1. An image acquisition unit that is provided in each of two display devices and acquires each visible captured image captured by a visible imaging unit that captures an assembly work area including an assembly target member and a marker, and each thermal captured image captured by a thermal imaging unit that is provided in each of the two display devices and captures the temperature distribution of the assembly work area; A temperature detection unit that overlaps each visible captured image and each thermal captured image acquired by the image acquisition unit, and three-dimensionally connects the overlapped images to detect the temperature of the joint portion of the assembly target member; A temperature determination unit that determines whether or not the temperature of the joint portion detected by the temperature detection unit is acceptable; A three-dimensional data acquisition unit that acquires three-dimensional data indicating each three-dimensional shape and each dimension of the assembly target member, a base material to which the assembly target member is joined, and a jig that fixes the assembly target member and the base material from a storage unit; Based on the determination result determined by the temperature determination unit, joint work support information for assisting the joint work is displayed on the display units respectively included in the two display devices, and based on the position and orientation of the marker included in one visible captured image acquired by the image acquisition unit and the three-dimensional data acquired by the three-dimensional data acquisition unit, the joint work visible through one display device is three-dimensionally displayed on the display unit of the other display device, and a display control unit; An assembly work support device characterized by the above.

2. The temperature determination unit: Has a brazing material temperature determination unit that determines whether or not the temperature of the joint portion detected by the temperature detection unit exceeds the melting temperature of the brazing material; The display control unit: When the brazing material temperature determination unit determines that the temperature of the joint portion detected by the temperature detection unit exceeds the melting temperature of the brazing material, the joint work support information is displayed on each display unit. The assembly work support device according to claim 1, characterized by the above.

3. The temperature determination unit: Has a base material temperature determination unit that determines whether or not the temperature of the joint portion detected by the temperature detection unit does not exceed the melting temperature of the base material; The display control unit: When the base material temperature determination unit determines that the temperature of the joint portion detected by the temperature detection unit is the temperature immediately before the melting temperature of the base material, an alarm is displayed on each display unit. The assembly work support device according to claim 1 or 2, characterized by the above.

4. The joining operation support device according to any one of claims 1 to 3, One display device of an operator who performs the joining operation, The other display device of an instructor who guides the operator, And the storage unit that stores the three-dimensional data. A joining operation support system characterized by the above.

5. The image acquisition unit is provided in each of the two display devices, and each visible captured image captured by the visible imaging unit that captures the joining operation area including the joining target member and the marker, and each provided in each of the two display devices, and the heat imaging unit that captures the temperature distribution of the joining operation area Acquires each thermal captured image, The temperature detection unit overlays each visible captured image and each thermal captured image acquired by the image acquisition unit, and three-dimensionally connects the overlaid images to detect the temperature of the joint portion in the joining target member. The temperature determination unit determines whether the temperature of the joint portion detected by the temperature detection unit is acceptable. The three-dimensional data acquisition unit acquires, from the storage unit, three-dimensional data indicating each three-dimensional shape and each dimension of the joining target member, the base material to which the joining target member is joined, and the jig that fixes the joining target member and the base material. Based on the determination result determined by the temperature determination unit, the display control unit displays joining operation support information for supporting the joining operation on the display units respectively included in the two display devices, and the position and orientation of the marker included in one visible captured image acquired by the image acquisition unit, and based on the three-dimensional data acquired by the three-dimensional data acquisition unit, the joining operation visible through one display device is three-dimensionally displayed on the display unit of the other display device. A joining operation support method characterized by the above.

6. A computer, An image acquisition unit that is provided in each of the two display devices and acquires each visible captured image captured by the visible imaging unit that captures the joining operation area including the joining target member and the marker, and each thermal captured image captured by the thermal imaging unit that is provided in each of the two display devices and captures the temperature distribution of the joining operation area, A temperature detection unit that overlays each visible captured image and each thermal captured image acquired by the image acquisition unit, and three-dimensionally connects the overlaid images to detect the temperature of the joint portion in the joining target member. A temperature determination unit that determines whether the temperature of the joint portion detected by the temperature detection unit is acceptable; A three-dimensional data acquisition unit that acquires three-dimensional data indicating the three-dimensional shapes and dimensions of the joining target member, the base material to which the joining target member is joined, and the jig that fixes the joining target member and the base material from a storage unit; Based on the determination result determined by the temperature determination unit, joining operation support information for supporting the joining operation is displayed on the display units respectively included in the two display devices, and based on the position and orientation of the marker included in one visible captured image acquired by the image acquisition unit and the three-dimensional data acquired by the three-dimensional data acquisition unit, a program for causing a display control unit to function to three-dimensionally display a joining operation visible through one display device on the display unit of the other display device.

Images