Learning model generation method for welding information, learning model, program product, and welding system

Abstract

A learning model generation method uses image data from a visual sensor and welding information extracted from the image data as teaching data for learning. Multiple welding conditions used in generating the learning model correspond to differences in settings regarding at least one item. Image data corresponding to each welding condition includes at least one of a molten pool, a welding wire, and an arc. The welding information includes at least one of information related to changes in the molten pool, information related to the position of the welding wire, and information related to the arc. A learning model is generated that uses image data as input and outputs welding information.

Description

Technical Field

[0001] The present invention relates to a method for generating a learning model of welding information, a learning model, a program and a welding system. Background Art

[0002] In recent years, in the field of welding operations, technologies for automatically creating or adjusting optimal welding conditions have been realized. For example, in Patent Document 1, for the purpose of automatically creating optimal welding conditions, "a machine learning device (30) is disclosed, which learns to determine at least one arc welding condition, and comprises: a state observation unit (31), which observes a state variable consisting of at least one physical quantity related to arc welding and at least one arc welding condition during or after the execution of arc welding; and a learning unit (35), which associates the change of at least one physical quantity observed by the state observation unit with at least one arc welding condition to perform learning."

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-30014 Summary of the invention

[0006] Problems to be solved by the invention

[0007] Patent Document 1 uses as physical quantities the waveform of welding current and welding voltage obtained from a current and voltage measuring unit, image data of a welding portion captured by an image capturing unit, and information extracted from the image data. The information extracted from the image data refers to the appearance of the weld bead, the residual height output of the weld bead, and the welding voltage waveform.

[0008] Height, bead width, and amount of spatter generated.

[0009] However, welding phenomena vary depending on the combination of welding current, arc voltage, tip-base material distance (hereinafter also referred to as "protruding length"), shape of the object to be welded (hereinafter also referred to as "workpiece"), welding posture, position of the visual sensor, and other arc welding conditions (hereinafter referred to as "welding conditions").

[0010] Furthermore, changes in welding phenomena occur not only due to differences in combination but also due to differences in the settings of individual welding conditions.

[0011] However, Patent Document 1 does not mention the influence of the difference in setting of each welding condition on the welding phenomenon.

[0012] For this reason, in the case of Patent Document 1, when the settings of the welding conditions used in learning the learning model are different from the settings of the welding conditions used in actual welding, the accuracy of the physical quantity automatically extracted from the welding phenomenon (hereinafter also referred to as "welding information") is unstable, and there is a possibility of performing erroneous processing based on the erroneously extracted welding information.

[0013] An object of the present invention is to generate a learning model that can stably extract welding information from captured image data with high accuracy, without depending on various welding conditions set in actual welding.

[0014] Means for solving problems

[0015] According to the relevant purpose, as an invention, a method for generating a learning model of welding information is provided, which generates a learning model that outputs welding information corresponding to the image data when the image data output from the visual sensor that photographs the welding situation is input. In the learning model generation method, the image data used in the generation of the learning model is photographed for multiple settings of at least one welding condition, the image data includes at least one of the molten pool, the welding wire and the arc, the welding information includes at least one of information related to the change of the molten pool, information related to the position of the welding wire and information related to the arc, and the image data and the welding information corresponding to the image data are used in the teaching data to generate the learning model.

[0016] In addition, preferably, the image data corresponds to two or more settings related to at least one of the protruding length of the welding wire, the shape of the welding tip, the brightness of the image data, the shape of the object to be welded, the shape of the conductive tip, the welding current, the arc voltage, the welding speed, the swing conditions, and the welding posture.

[0017] Furthermore, preferably, at least one welding condition is a protruding length of the welding wire, and the setting of the protruding length is selected from a plurality of settings at least in the range of 10 to 40 mm.

[0018] The information on the movement of the molten pool includes information on one or more positions of the leading end of the molten pool appearing in the direction in which welding is advanced, and information on the positions of the left and right ends of the molten pool relative to the direction in which welding is advanced.

[0019] Furthermore, preferably, the information on the position of the welding wire includes information on the position of the leading end of the welding wire.

[0020] Furthermore, preferably, the information related to the arc includes information on the center point of the arc.

[0021] As another invention, a learning model is provided, which outputs welding information corresponding to the image data when image data output from a visual sensor that photographs a welding situation is input. The image data used in generating the learning model is photographed for multiple settings of at least one welding condition, the image data includes at least one of a molten pool, a welding wire, and an arc, the welding information includes at least one of information related to changes in the molten pool, information related to the position of the welding wire, and information related to the arc, and the image data and the welding information corresponding to the image data are used in teaching data to generate the learning model.

[0022] As yet another invention, a program is provided that enables a computer to realize a function of generating a learning model that, if inputted with image data outputted from a visual sensor that photographs a welding situation, outputs welding information corresponding to the image data, wherein the function uses the image data and the welding information corresponding to the image data in teaching data, and the image data used in the generation of the learning model is photographed for multiple settings of at least one welding condition, the image data includes at least one of a molten pool, a welding wire, and an arc, and the welding information includes at least one of information related to changes in the molten pool, information related to the position of the welding wire, and information related to the arc.

[0023] As yet another invention, a welding system is provided, comprising: a welding device that welds an object to be welded; a visual sensor that captures images of the welding situation; a learning model that, when image data output from the visual sensor is input, outputs welding information corresponding to the image data; and a control device that uses the welding information output when the image data captured by the visual sensor is input into the learning model to control the activity of the welding device, wherein the image data used in generating the learning model is captured for multiple settings of at least one welding condition, the image data includes at least one of a molten pool, a welding wire, and an arc, the welding information includes at least one of information related to changes in the molten pool, information related to the position of the welding wire, and information related to the arc, and the control device uses the image data and the welding information corresponding to the image data in the teaching data to generate the learning model.

[0024] Preferably, the welding system includes, in addition to the learning model that outputs welding information corresponding to the image data, another learning model that inputs the welding information output from the learning model and outputs a correction signal for the welding information.

[0025] Effects of the Invention

[0026] According to the present invention, it is possible to generate a learning model that can stably extract welding information with high accuracy from captured image data without depending on various welding conditions set in actual welding. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a diagram showing a configuration example of an automatic welding system used in the description of the embodiment.

[0028] Figure 2 This is a perspective view illustrating the configuration position of the vision sensor.

[0029] Figure 3 This is a diagram for explaining a configuration example of a robot control device that controls the operation of a welding robot.

[0030] Figure 4 This is a diagram for explaining a configuration example of a data processing device.

[0031] Figure 5 This is a diagram that conceptually explains the process of generating a learning model.

[0032] Figure 6 This is a diagram for explaining an example of a teaching operation and a screen used therein.

[0033] Figure 7 is a flow chart illustrating supervised learning of the learning model.

[0034] Figure 8 This is a flowchart for explaining the processing operations of the robot control device and the power supply device.

[0035] Fig. 9 This is a flowchart for explaining the correction information generation process performed by the data processing device.

[0036] Fig.10 This is a diagram for explaining an example of welding conditions used for generating teaching data.

[0037] Fig.11 1 and 2 are diagrams for explaining an output example of welding information corresponding to Specific Example 1 and an output example of welding information corresponding to Comparative Example 1.

[0038] Fig.12 This is a diagram for explaining another example of welding conditions used for generating teaching data.

[0039] Fig.13 This is a diagram for explaining an output example of specific example 2 when the aperture value of the camera is F=8.

[0040] Fig.14 This is a diagram showing an output example of Specific Example 2 when the aperture value of a camera used in an actual welding environment is F=15.

[0041] Fig.15 This is a diagram for explaining an output example of Comparative Example 2 when the aperture value of the camera is F=8.

[0042] Fig.16This is a diagram for explaining an output example of Comparative Example 2 when the aperture value of a camera used in an actual welding environment is F=15. DETAILED DESCRIPTION

[0043] Embodiments of a method for generating a learning model of welding information, a learning model, a program, and a welding system according to the present invention will be described below with reference to the accompanying drawings. In addition, each drawing is provided for the purpose of explaining the present invention, and the embodiments of the present invention are not limited to the contents shown in the drawings.

[0044] <Automatic welding system configuration>

[0045] Figure 1 This is a diagram showing a configuration example of an automatic welding system 1 used in the description of the embodiment.

[0046] Figure 1 The illustrated automatic welding system 1 includes a welding robot 10 , a robot control device 20 , a power supply device 30 , a vision sensor 40 , and a data processing device 50 .

[0047] The automatic welding system 1 in this embodiment has the function of generating a learning model by using an image of a welding position and welding information taught by an operator about the corresponding image as teaching data, and the function of automatically controlling arc welding using the generated learning model. In addition, deep learning is used to generate the learning model.

[0048] Figure 1 The welding robot 10 shown is a 6-axis multi-joint robot, and a welding torch 11 for arc welding is mounted on its front end. Arc welding includes, for example, MIG (Metal Inert Gas) welding and MAG (Metal Active Gas) welding, and in this embodiment, MAG welding is selected.

[0049] Furthermore, the welding robot 10 may be, for example, a small portable robot.

[0050] Welding wire 13 is supplied to welding torch 11 from wire feeder 12. Welding wire 13 is fed out from the front end of welding torch 11.

[0051] Power supply device 30 is a power supply device that supplies power to welding wire 13. This power applies arc voltage between welding wire 13 and workpiece W to generate an arc. Power supply device 30 is provided with a current sensor (not shown) that detects welding current flowing from welding wire 13 to workpiece W during welding.

[0052] Power supply device 30 includes a CPU (Central Processing Unit) and a memory (not shown). The CPU executes a computer program for power supply control to control the power applied to welding wire 13 .

[0053] The power supply device 30 is also connected to the welding wire feeding device 12 , and the CPU controls the feeding speed of the welding wire 13 .

[0054] The visual sensor 40 is, for example, a CCD (Charge Coupled Device) camera.

[0055] Figure 2 1 is a perspective view for explaining the configuration position of the visual sensor 40. In the case of this embodiment, the workpiece W is a butt joint. The workpiece W is two metal plates butted across a groove. In addition, a ceramic backing member 14 is installed on the back side of the two butted metal plates. In addition, a metal backing member may be used on the back side, or there may be no backing member.

[0056] In a butt joint, arc welding is performed in one direction along the groove.

[0057] The direction of welding advancement is referred to as "forward" below. Figure 2 In the figure, the direction of welding advancement is shown by arrows. For this reason, the welding torch 11 is located at the rear of the visual sensor 40.

[0058] In the present embodiment, the side where the right hand is located when viewing the visual sensor 40 from behind is referred to as the “right side”, and the side where the left hand is located when viewing the front is referred to as the “left side”.

[0059] The workpiece W in the present embodiment is placed horizontally so that the surface on the side to be welded faces vertically upward. Therefore, the welding robot 10 welds the workpiece W from the upper side of the workpiece W.

[0060] like Figure 2 As shown, the vision sensor 40 is disposed obliquely above the welding position of the workpiece W. The imaging range of the vision sensor 40 includes the welding position of the workpiece W, and an image of the welding position during arc welding is captured. The image includes the molten pool, welding wire 13, and arc.

[0061] The visual sensor 40 in this embodiment can continuously capture still images of, for example, 1024×768 pixels. In other words, the visual sensor 40 can capture welding images as moving images.

[0062] <Structure of the robot control device>

[0063] Figure 31 is a diagram for explaining a configuration example of the robot control device 20 that controls the operation of the welding robot 10 .

[0064] The robot control device 20 includes: a CPU 201 for controlling the entire device; a memory 202 for storing data; an operation panel 203 including a plurality of switches; a teaching pendant 204 used in teaching work; an input / output unit 205; and a communication unit 206.

[0065] The memory 202 stores a control program 202A used for controlling the welding robot 10. The CPU 201 controls the welding operation of the welding robot 10 by executing the control program 202A.

[0066] The operation panel 203 and the teaching pendant 204 are used to input instructions to the robot control device 20. The operator can use the teaching pendant 204 to input a teaching program.

[0067] The robot control device 20 controls the welding operation of the welding robot 10 according to the teaching program input from the teaching pendant 204. In principle, the teaching program can also be created using a computer (not shown).

[0068] The power supply device 30 and the driving circuit of the welding robot 10 are connected to the input / output unit 205. The current value of the welding current detected by, for example, a current sensor (not shown) is provided from the power supply device 30 to the CPU 201 through the input / output unit 205. In addition, the CPU 201 outputs a control signal based on the control program 202A to the driving circuit of the welding robot 10.

[0069] The communication unit 206 is a communication module for wired or wireless communication and is used for data communication with the power supply device 30 and the data processing device 50 .

[0070] The robot control device 20 also controls the speed of the welding torch 11 by controlling each axis of the welding robot 10. In the present embodiment, the robot control device 20 uses the result of image recognition to control the speed of the welding torch 11. However, the speed of the welding torch 11 may be controlled to be constant.

[0071] The robot control device 20 also controls the swinging motion of the welding robot 10. The so-called swinging motion means that the welding torch 11 is alternately shaken in a direction intersecting the welding advancing direction. The robot control device 20 controls the swinging motion according to the set cycle, amplitude, and welding speed.

[0072] The robot control device 20 performs welding line imitation control together with the swinging operation. The welding line imitation control is an operation to control the left and right positions of the welding torch 11 so that a weld bead is formed along the welding line.

[0073] <Structure of data processing device>

[0074] Figure 4 It is a diagram for explaining a configuration example of the data processing device 50 .

[0075] The data processing device 50 is constituted by, for example, a computer. The computer includes a main body 510 , an input unit 520 , and a display unit 530 .

[0076] The main body 510 includes a CPU 511 , a ROM (Read Only Memory) 512 , a RAM (Random Access Memory) 513 , a nonvolatile storage device 514 , an input / output interface 515 , a communication interface 516 , and a video output interface 517 .

[0077] The CPU 511 , the ROM 512 , the RAM 513 , the nonvolatile storage device 514 , the input / output interface 515 , the communication interface 516 , and the video output interface 517 are connected to one another via buses and signal lines.

[0078] The non-volatile storage device 514 stores: a learning program 514A that performs deep learning using image data obtained by photographing welding phenomena and welding information taught by an operator with respect to the image data as teaching data; a learning model 514B generated by the execution of the learning program 514A; a welding information generation program 514C that uses the learning model 514B to generate welding information related to arc welding; and image data 514D of the welding position.

[0079] The learning model 514B in this embodiment is composed of a convolutional neural network. In addition, an operating system and an application program are installed in the nonvolatile storage device 514 .

[0080] The data processing device 50 realizes various functions by executing the program through the CPU 511 .

[0081] In the case of this embodiment, the data processing device 50 realizes: a function of generating a learning model; a function of inputting image data and outputting welding information when actually performing welding. The contents of these functions will be described later. In addition, the data processing device 50 can be divided according to the function of generating a learning model and the function of inputting image data and outputting welding information when actually performing welding. From the perspective of versatility, it is preferable to divide the data processing device 50 according to the respective functions.

[0082] The ROM 512 stores a BIOS (Basic Input Output System) and the like executed by the CPU 511 . The RAM 513 is used as a work area for programs read from the nonvolatile storage device 514 .

[0083] The input unit 520 including a keyboard, a mouse, etc. is connected to the input / output interface 515. The visual sensor 40 is also connected to the input / output interface 515. Image data output from the visual sensor 40 is supplied to the CPU 511 via the input / output interface 515.

[0084] The communication interface 516 is a communication module for wired or wireless communication. The image output interface 517 is connected to the display unit 530 composed of, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and outputs an image signal corresponding to the image data provided by the CPU 511 to the display unit 530 .

[0085] <Automatic welding system operation>

[0086] Next, the operation of the automatic welding system 1 will be described.

[0087] <Generation of learning model>

[0088] Here, the generation of the learning model 514B will be described. The learning model 514B is generated by executing the learning program 514A.

[0089] Figure 5 This is a diagram conceptually illustrating a generation process of the learning model 514B. The learning model 514B in this embodiment is composed of a convolutional neural network, including a plurality of convolutional layers, a plurality of pooling layers, and a plurality of fully connected layers.

[0090] The learning model 514B receives the image data output from the vision sensor 40 as input, and outputs various welding information appearing in the image data.

[0091] The image data input to the learning model 514B includes the molten pool, the welding wire 13 (reference Figure 1 ) and arc. This image data is hereinafter referred to as a "welding image".

[0092] In the case of the present embodiment, welding is performed with the settings of welding conditions changed, and welding images obtained by imaging each welding situation are prepared.

[0093] The welding conditions in this embodiment include, for example, the protruding length of the welding wire 13, the brightness of the image data (F value), the shape of the workpiece W, the shape of the conductive tip, the welding current, the arc voltage, the welding speed, the swing conditions, and the posture of the welding wire 13 relative to the workpiece W (welding posture).

[0094] For example, the shape of the conductive tip includes a straight welding tip and a tapered welding tip. In addition, the information that specifies the welding posture includes, for example, a retreat angle, an advance angle, and a deviation of the welding line relative to the groove.

[0095] In this embodiment, a plurality of welding processes are performed with different settings of the "protrusion length" among welding conditions, and welding images obtained by photographing the situations during each welding are prepared as teaching data. For example, welding images are prepared when the protrusion length is 22 mm, welding images are prepared when the protrusion length is 27 mm, and welding images are prepared when the protrusion length is 32 mm. In addition, welding information taught by the operator about these welding images is prepared as teaching data.

[0096] Furthermore, welding may be performed while changing the settings of welding conditions other than the protrusion length, and welding images obtained by photographing the situations during each welding and welding information taught by the operator based on the welding images may be prepared.

[0097] In this embodiment, as welding information, the tip of the welding wire 13 (wire tip), the center point of the arc (arc center), the positions of the left and right tips of the molten pool, and the positions of the left and right ends of the molten pool are used.

[0098] The operator inputs welding information used as teaching data by designating a specific position on the welding image in accordance with instructions on the operation screen supporting the teaching work.

[0099] Figure 6 This is a diagram illustrating an example of a screen used in teaching operations. Figure 6 The welding image shown includes a molten pool 15, a welding wire 13, and an arc 16. Figure 6 In FIG. 1 , the molten pool 15 is indicated by shadow.

[0100] In the case of this embodiment, the operator is taught as welding information the coordinate position of the arc center (ArcX, ArcY), the coordinate position of the front end of the welding wire (WireX, WireY), the coordinate position of the left end of the front end of the molten pool (Pool_Lead_Lx, Pool_Lead_Ly), the coordinate position of the right end of the front end of the molten pool (Pool_Lead_Rx, Pool_Lead_Ry), the coordinate position of the left end of the molten pool (Pool_Lx), and the coordinate position of the right end of the molten pool (Pool_Rx).

[0101] The operator inputs welding information by pointing to a specific position on the screen.

[0102] The coordinates of the boundary between welding wire 13 and the arc are given as an example of the position coordinates of the tip of the welding wire.

[0103] The molten pool front left end, the molten pool front right end, the molten pool left end, and the molten pool right end are examples of information related to the change of the molten pool 15. In addition, the molten pool left end and the molten pool right end can calculate the width of the molten pool 15.

[0104] Figure 7 This is a flowchart for explaining supervised learning of the learning model 514B. The symbol S shown in the figure indicates a step.

[0105] When supervised learning is performed, the data processing device 50 (refer to Figure 1 ) CPU511 (reference Figure 4 ) receiving image data obtained by imaging welding for changing settings of at least one of welding conditions and welding information taught with respect to the image data as teaching data (step 1).

[0106] As described above, the welding information is prepared in advance by the operator. In the case of the present embodiment, a grayscale image reduced in size to, for example, 120×100 pixels is used as the welding image serving as teaching data.

[0107] Next, CPU 511 inputs the shaded image as teaching data to learning model 514B, and performs supervised learning so that the output welding information is consistent with the welding information as teaching data (step 2). Specifically, back propagation processing is performed as error feedback to adjust the parameters of each layer and between layers of learning model 514B.

[0108] Next, the CPU 511 determines whether or not a termination condition is satisfied (step 3). The termination condition is, for example, that the average error in the most recent given teaching data is less than a given value.

[0109] If the termination condition is not satisfied (a negative result in step 3), CPU 511 returns to step 2 and continues supervised learning. On the other hand, if the termination condition is satisfied (a positive result in step 3), CPU 511 terminates the generation of learning model 514B.

[0110] <Actual welding using the generated learning model>

[0111] The welding operation of automatic welding system 1 during actual welding will be described below.

[0112] When performing arc welding, the operator activates the robot control device 20 , the power supply device 30 , and the data processing device 50 , respectively.

[0113] The robot control device 20 controls the movement of the welding robot 10, and the power supply device 30 performs welding. In addition, the data processing device 50 monitors the welding image captured by the visual sensor 40, and sequentially calculates welding information related to arc welding.

[0114] In this embodiment, the welding information is the position of the tip of the welding wire, the arc center, the right and left end positions of the tip of the molten pool, and the right and left end positions of the molten pool.

[0115] Figure 8 This is a flowchart for explaining the processing operations of the robot control device 20 and the power supply device 30. The symbol S shown in the figure means a step.

[0116] When starting arc welding, the operator operates the teaching pendant 204 (refer to Figure 3 ), to input the teaching program, various setting values ​​and welding start instructions to the robot control device 20. The teaching program here specifies the welding robot 10 (reference Figure 1 ) activities, etc.

[0117] The CPU 201 of the robot control device 20 (refer to Figure 3 ) accepts the teaching program, various setting values ​​and welding start instructions (step 11).

[0118] CPU 201 instructs power supply device 30 to start welding (start supplying welding power) (step 12). Power supply device 30 receives the instruction (step 13), and controls the power supply circuit built into power supply device 30 to start welding (step 14). As a result, welding wire 13 (reference Figure 1 ) and workpiece W (reference Figure 1 ) to generate an arc by applying welding voltage between them.

[0119] Next, the CPU 201 sends a control signal to the welding robot 10 to execute welding control (step 15 ). The welding control includes automatic welding control (step 151 ), swing motion control (step 152 ), and welding line imitation control (step 153 ).

[0120] In the automatic welding control, the CPU 511 automatically moves the welding torch 11 in the welding direction, and sends a correction signal for controlling at least one of the welding speed, welding current or arc voltage to the power supply device 30, and the power supply device 30 performs welding according to the correction signal. In addition, from the perspective of ease of control, it is preferred that the automatic welding control includes the control of the welding speed, but in this embodiment, only the control of the welding speed is performed.

[0121] CPU 201 determines whether the welding operation needs to be stopped (step 16). If the welding operation does not need to be stopped (no result in step 16), robot control device 20 receives welding information from data processing device 50 (step 17), and returns to step 15. Returning to step 15, CPU 201 performs welding control based on the received welding information.

[0122] For example, when an instruction to stop welding is received from an operator, a sensor provided in the welding robot 10 detects a welding end position or a welding abnormality, or both (affirmative result in step 16 ), the CPU 201 determines that the welding operation needs to be stopped.

[0123] In this case, CPU 201 stops welding control (step 18 ), and instructs power supply device 30 to stop welding (step 19 ). Welding is stopped by stopping the supply of welding power.

[0124] Furthermore, the CPU 201 instructs the data processing device 50 to stop generating the welding information (step 20 ).

[0125] On the other hand, when the power supply device 30 receives the welding stop command (step 21), the CPU (not shown) controls the power supply circuit to stop welding (step 22). Thus, the operation of the robot control device 20 and the power supply device 30 ends.

[0126] The welding information received by the CPU 201 of the robot controller 20 in step 17 is generated as follows.

[0127] Fig. 9 This is a flowchart for explaining the generation process of welding information performed by the data processing device 50, and the execution of welding control based on the welding information and the transmission of correction signals of each control factor to the power supply device 30. In addition, each control factor refers to a welding condition, and examples thereof include welding speed, welding current, arc voltage, oscillation motion, and welding line simulation. The symbol S shown in the figure refers to a step.

[0128] The CPU 511 of the data processing device 50 receives a welding image obtained by imaging the welding position from the vision sensor 40 during the execution of the automatic welding (step 31 ).

[0129] Next, CPU 511 reduces the size of the received welding image and converts it into a grayscale image (step 32 ). CPU 511 also reduces the size of the welding image to 120×100 pixels.

[0130] Next, CPU 511 inputs the reduced welding image to learning model 514B, and acquires welding information output from learning model 514B (step 33 ).

[0131] From the learning model 514B, the coordinate position of the arc center (ArcX, ArcY), the coordinate position of the front end of the welding wire (WireX, WireY), the coordinate position of the left end of the front end of the molten pool (Pool_Lead_Lx, Pool_Lead_Ly), the coordinate position of the right end of the front end of the molten pool (Pool_Lead_Rx, Pool_Lead_Ry), the coordinate position of the left end of the molten pool (Pool_Lx), and the coordinate position of the right end of the molten pool (Pool_Rx) are output.

[0132] The CPU 511 performs welding control based on the acquired welding information and calculates a correction signal for each control factor (step 34 ).

[0133] In the present embodiment, as correction signals, the left-right position correction amount ΔX of the welding torch 11 , the speed correction amount ΔV of the welding torch 11 , and the setting value w of the width of the swing operation (hereinafter referred to as “swing width”) are calculated.

[0134] The left-right position correction amount ΔX of the welding torch 11 is calculated by the following equation (1).

[0135] PoolCenX=(Pool_Lx+Pool_Rx) / 2

[0136] ΔX=PoolCenX-X0 (1)

[0137] Here, X0 is a reference value of the left-right position of the center of the molten pool, and is a preset value.

[0138] The calculated left-right position correction amount ΔX is used to move the welding torch 11 in the left-right direction so that the center PoolCenX of the molten pool image coincides with X0.

[0139] When calculating the speed correction amount ΔV, first, the distance ΔL is calculated using the following equation (2).

[0140] LeadY=(Pool_Lead_Ly+Pool_Lead_Ry) / 2-ArcY

[0141] ΔL=LeadY-L0 (2)

[0142] Here, L0 is the distance between the front end of the molten pool and the center of the arc, and is preset to a value that can form a penetration weld with the best quality.

[0143] LeadY is also the distance between the front end of the molten pool and the center of the arc. LeadY is calculated by the above formula (2) using Pool_Lead_Ly, Pool_Lead_Ry, and ArcY as image feature information extracted from the welding image.

[0144] The distance ΔL is the difference between LeadY and L0. When the distance ΔL is calculated, the speed correction amount ΔV is calculated by the following equation (3) using the conversion coefficient β.

[0145] ΔV=ΔL×β (3)

[0146] The set value w of the swing width is calculated by the following formula (5) or (6).

[0147] Leadw=Pool_Lead_Rx-Pool_Lead_Lx

[0148] IF Leadw<w0THEN w=0 (4)

[0149] IF Leadw≥w0THEN w=(Leadw-α)(5)

[0150] Here, w0 is the gap width of the groove, which is preset to the minimum value required for swinging.

[0151] Leadw also represents the gap width, and is a value calculated as described above because the coordinate positions (Pool_Lead_Lx, Pool_Lead_Rx) of the image feature information extracted from the welding image correspond to the left and right positions of the groove, respectively.

[0152] According to the above-mentioned formula (4), when the gap width Leadw is less than w0, the set value w of the swing width becomes 0, and the swing operation is not performed.

[0153] According to the above formula (5), when the gap width Leadw is equal to or larger than w0, the set value w of the swing width is a value smaller than the gap width Leadw by α. α is a preset value.

[0154] The process of step 34 may be further modeled as a learning model. For example, the learning model may be generated by using information that associates welding information with welding result information in the teaching data, and the welding information is input to the learning model, and a correction signal of each control factor is output to obtain the best welding result. Here, the welding result information includes waveform data of welding current, waveform data of arc voltage, arc sound data during welding, weld bead shape data, spatter generation data, smoke generation data, welding defect data, etc. It is preferred to treat at least one of these data as welding result information.

[0155] When calculation of the correction signal in step 34 is completed, the CPU 511 transmits the correction signal to the robot control device 20 (step 35 ).

[0156] Next, the CPU 511 determines whether or not a command to stop the generation of the correction signal has been received from the robot controller 20 (step 36 ).

[0157] When the stop instruction is not received (a negative result in step 36 ), the CPU 511 returns to step 31 .

[0158] On the other hand, when a stop instruction is received (affirmative result in step 36 ), the CPU 511 ends the correction signal generation process.

[0159] As described above, in the automatic welding system 1 according to the present embodiment, since the welding image obtained by changing the setting of at least one of the welding conditions and the welding information taught by the operator with respect to the welding image are used as teaching data to perform deep learning to generate the learning model 514B, it is possible to stably output highly reliable welding information without depending on various welding conditions set in actual welding. That is, a learning model with a jump in robustness is generated.

[0160] <Specific examples>

[0161] Hereinafter, it will be described how the recognition accuracy of welding information is improved by using the learning model 514B generated by the method described in the embodiment.

[0162] (1) Specific example 1

[0163] Fig.10 FIG. 1 is a diagram for explaining an example of welding conditions used in generating teaching data. Fig.10 In the description, the teaching data corresponding to the method described in the embodiment is referred to as specific example 1, and the teaching data corresponding to the conventional method is referred to as comparative example 1.

[0164] In the case of Specific Example 1, a total of 5400 welding images are used as teaching data. This number is twice the number of welding images used in Comparative Example 1.

[0165] The welding image of Specific Example 1 is different from the welding image of Comparative Example 1 in the following two points.

[0166] The first difference is that two types of settings are used for the protrusion length as one of the welding conditions. In the case of Comparative Example 1, only the welding image captured when the protrusion length is 25 mm is used, and in the case of Specific Example 1, the welding image captured when the protrusion length is 22 mm is added.

[0167] The second difference is that the welding wire 13 (reference Figure 1 ) Welding images of the case where the weld pool is on the right and the case where it is on the left.

[0168] However, in the case of Comparative Example 1, since the position of the welding wire 13 in the welding image used as teaching data is not determined, both welding images captured in a state where the welding wire 13 is located on the right side of the molten pool and welding images captured in a state where the welding wire 13 is located on the left side of the molten pool may be included. However, the number of images of the corresponding positions is significantly smaller than that of Specific Example 1.

[0169] Fig.11 1 and 2 are diagrams for explaining an output example of welding information corresponding to Specific Example 1 and an output example of welding information corresponding to Comparative Example 1.

[0170] exist Fig.11 Six welding images are shown. The three welding images in the upper layer correspond to the state where the welding wire 13 is located on the right side of the molten pool, and the three welding images in the lower layer correspond to the state where the welding wire 13 is located on the left side of the molten pool. In addition, the two columns of welding images on the left correspond to Comparative Example 1, and the one column of welding images on the right corresponds to Specific Example 1.

[0171] In each welding image, circular marks are used to indicate the positions of the arc center, the tip of the welding wire, the left end of the front end of the molten pool, the right end of the front end of the molten pool, the left end of the molten pool, and the right end of the molten pool recognized using the learning model 514B.

[0172] In the case of Comparative Example 1, the circular mark indicating the recognized position of the tip of the welding wire appears at a position different from the actual tip of the welding wire 13 in any welding image. This means that there is an error in the recognition of the tip of the welding wire. Fig.11 In the column of Comparative Example 1, NG1 and NG2 are indicated.

[0173] On the other hand, in the case of Specific Example 1, the circular mark indicating the recognized position of the tip of the welding wire overlaps with the tip of the welding wire 13 in any welding image. That is, in the case of Specific Example 1, the correct position is recognized. In addition, in the case of Specific Example 1, regardless of the position of the welding wire 13 relative to the molten pool, it is confirmed that the accuracy of recognizing the tip of the welding wire is improved.

[0174] In addition, in the case of Comparative Example 1, if the protrusion length used in the generation of the learning model is different from the protrusion length used in the actual welding environment, it is confirmed that the recognition accuracy of the front end of the welding wire is reduced, but in the case of the specific example, high-precision recognition can be performed without depending on the protrusion length used in the welding environment.

[0175] (2) Specific example 2

[0176] Fig.12 FIG. 2 is a diagram for explaining another example of welding conditions used in generating teaching data. Fig.12In the present invention, the teaching data corresponding to the method described in the embodiment is referred to as specific example 2, and the teaching data corresponding to the conventional method is referred to as comparative example 2.

[0177] In the welding image of the specific example 2, a part of the corresponding welding conditions is different from the welding image of the specific example 1. In the specific example 2, the welding wire 13 (reference Figure 1 ) The welding images of the case where the weld is located on the right side of the molten pool and the case where the weld is located on the left side are used as teaching data, but multiple settings are also considered for the exposure time and groove gap as welding conditions.

[0178] For example, the exposure time of Comparative Example 2 is one type of exposure time, 1100 μs, but the exposure time of Specific Example 2 is three types of exposure time, 2000 μs, 1100 μs, and 800 μs.

[0179] In addition, the groove gap of Comparative Example 2 is only 5 mm, but the groove gap of 100 types in the range of 3 to 10 mm is considered for Specific Example 2. In addition, the value of the groove gap is randomly determined.

[0180] In addition, the aperture value of the camera used for the welding image in Specific Example 2 and the camera used in Comparative Example 2 is F=8.

[0181] Fig.13 This is a diagram for explaining an output example of specific example 2 when the aperture value of the camera is F=8.

[0182] exist Fig.13 , 6 welding images are shown. The 3 welding images in the upper layer are when the groove gap is narrow, and the 3 welding images in the lower layer are when the groove gap is wide. In addition, the column at the left end is when the exposure time is 2000μs, the column in the middle is when the exposure time is 1100μs, and the column at the right end is when the exposure time is 800μs.

[0183] Regardless of which welding image is concerned, the positions of the arc center, the front end of the welding wire, the left end of the front end of the molten pool, the right end of the front end of the molten pool, the left end of the molten pool, and the right end of the molten pool are correctly recognized.

[0184] Fig.14 This is a diagram for explaining an output example of specific example 2 when the aperture value F of a camera used in an actual welding environment is 15. Fig.14 In the case of , the leftmost column is the case where the exposure time is 6000 μs, the middle column is the case where the exposure time is 5000 μs, and the rightmost column is the case where the exposure time is 4000 μs.

[0185] exist Fig.14 In the case of , the exposure time and the aperture value F of the camera are different from the settings of the welding conditions used in the learning of the learning model 514B. Fig.14In any welding image shown, all welding information is correctly recognized.

[0186] Fig.15 This is a diagram for explaining an output example of Comparative Example 2 when the aperture value of the camera is F=8. Fig.15 In, with Fig.13 The corresponding parts are marked with corresponding reference numerals. That is, the three welding images in the upper layer are when the groove gap is narrow, and the three welding images in the lower layer are when the groove gap is wide. In addition, the column at the left end is when the exposure time is 2000μs, the column in the middle is when the exposure time is 1100μs, and the column at the right end is when the exposure time is 800μs.

[0187] In the case of Comparative Example 2, in any welding image, deviations can be seen in the recognition of the positions of the arc center, the welding wire tip, the left end of the molten pool tip, the right end of the molten pool tip, the left end of the molten pool, and the right end of the molten pool. For example, when the exposure time is 2000 μs, the recognized arc center and welding wire tip deviate greatly from the correct positions regardless of the width of the groove gap. In addition, the right end and left end of the molten pool tip also deviate greatly from the correct positions.

[0188] Fig.16 This is an output example of Comparative Example 2 for explaining the case where the aperture value F=15 of a camera used in an actual welding environment. Fig.16 In, with Fig.14 The corresponding parts are marked with corresponding reference numerals. That is, the three welding images in the upper layer are when the groove gap is narrow, and the three welding images in the lower layer are when the groove gap is wide. In addition, the column at the left end is when the exposure time is 6000μs, the column in the middle is when the exposure time is 5000μs, and the column at the right end is when the exposure time is 4000μs.

[0189] exist Fig.16 In the case of , deviations can be seen in the recognition of any of the positions of the arc center, the tip of the welding wire, the left end of the front end of the molten pool, the right end of the front end of the molten pool, the left end of the molten pool, and the right end of the molten pool. For example, when the exposure time is 6000μs and the groove gap is narrow, the recognized right end and left end of the molten pool, and the right end and left end of the molten pool deviate greatly from the correct position. In addition, when the groove gap is wide even when the exposure time is 6000μs, the recognized arc center and the tip of the welding wire deviate greatly from the correct position.

[0190] As described above, in the case of Comparative Example 2, when the welding conditions used in the actual welding environment are different from the welding conditions used during learning, the recognition accuracy is significantly reduced.

[0191] (3) Summary

[0192] As described above using two specific examples, welding images corresponding to multiple settings of at least one welding condition are prepared as teaching data used in learning the learning model 514B. As a result, compared with a case where a welding image corresponding to one setting of one welding condition is used in the teaching data, changes in exposure conditions in the use environment have less impact on the accuracy of recognizing welding information.

[0193] As a result, welding control using the recognized results becomes appropriate, and the quality of automatic welding is improved.

[0194] <Other embodiments>

[0195] The embodiments of the present invention are described above, but the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements made to the above embodiments are also included in the technical scope of the present invention, as is clear from the description of the claims.

[0196] (1) For example, in the aforementioned embodiment, MIG welding and MAG welding of a molten-electrode type are exemplified as an example of an arc welding machine, but TIG (Tungsten Inert Gas) welding in which an arc is generated in an inert gas may also be used. In this case, a welding image obtained by taking a picture in a state where the shape and posture of the filler metal are changed as welding conditions may be additionally prepared. In addition, the present invention can also be applied to a welding machine that does not generate an arc. For example, the present invention can also be used in welding based on electroslag welding.

[0197] (2) In the above-described embodiment, the welding robot 10 is described as an example of a multi-joint robot, but may be a single-joint robot.

[0198] (3) In the aforementioned embodiment, the left end and the right end of the molten pool are output as welding information, but the width of the molten pool calculated based on this information may be output. In addition, the front end width of the molten pool calculated based on the front end left end and the front end right end of the molten pool may be included in the welding information. In addition, the distance between the front end position of the welding pool and the center of the arc, i.e., L0 and LeadY, may be included in the welding information.

[0199] (4) In the above-mentioned embodiment, the protrusion length as a welding condition is changed in steps of 5 mm to 22 mm, 27 mm, and 32 mm, and welding images obtained by photographing the welding conditions at each protrusion length are used as teaching data. However, the protrusion length step may be 3 mm as in Specific Example 1. However, the protrusion length step used in photographing the welding image used as teaching data is arbitrary.

[0200] (5) In the above-mentioned embodiments and specific examples, the case where the protrusion length of the welding condition is made variable during welding by photographing the welding image used as teaching data is illustrated. However, in the welding image used as teaching data, it does not necessarily mean that the protrusion length must be made variable. In other words, with respect to any one or more of the above-mentioned welding conditions, the learning model 514B may be generated by using a welding image obtained by photographing the teaching data with each setting made variable.

[0201] Various embodiments have been described above with reference to the accompanying drawings, but it is self-evident that the present invention is not limited to the relevant examples. It is clear that a person skilled in the art can think of various variations or modifications within the scope of the claims, and these are of course also within the technical scope of the present invention. In addition, the constituent elements in the above embodiments can be arbitrarily combined within the scope of the subject matter of the invention.

[0202] In addition, this application is based on the Japanese patent application (Japanese Patent Application No. 2020-080006) for which it applied on April 30, 2020, and the content is used in this application as a reference.

[0203] Description of Reference Numerals

[0204] 1…Automatic welding system, 10…Welding robot, 11…Welding torch, 12…Wire feeding device, 13…Welding wire, 14…Backing member, 20…Robot control device, 30…Power supply device, 40…Visual sensor, 50…Data processing device, 514A…Learning program, 514B…Learning model, 514C…Welding information generation program, 514D…Image data

Claims

1. A method for generating a learning model of welding information, the method comprising generating a learning model which, when image data outputted from a visual sensor for capturing a welding situation is inputted, outputs welding information corresponding to the image data, The method for generating a learning model of welding information is characterized in that: The image data used in generating the learning model is captured for a plurality of settings related to at least a protruding length of the welding wire and a positional relationship between the welding wire and the molten pool. The image data includes at least one of a molten pool, a welding wire, and an arc, The welding information includes at least one of information related to the change of the molten pool, information related to the position of the welding wire, and information related to the arc. The learning model is generated using the image data and the welding information corresponding to the image data in the teaching data.

2. The method for generating a learning model of welding information according to claim 1, characterized in that: The image data corresponds to two or more settings related to at least one of the shape of the welding tip, the brightness of the image data, the shape of the object to be welded, the shape of the conductive tip, the welding current, the arc voltage, the welding speed, the swing conditions, and the welding posture.

3. The method for generating a learning model of welding information according to claim 1, characterized in that: The protrusion length may be set to a plurality of lengths selected from at least a range of 10 to 40 mm.

4. The method for generating a learning model of welding information according to any one of claims 1 to 3, characterized in that: The information related to the change of the molten pool includes: Information on one or more positions of the front end of the molten pool appearing in the direction in which welding is advancing; and Information on the positions of the left and right ends of the molten pool relative to the direction in which welding is proceeding.

5. The method for generating a learning model of welding information according to any one of claims 1 to 3, characterized in that: The information related to the position of the welding wire includes information on the position of the front end of the welding wire.

6. The method for generating a learning model of welding information according to any one of claims 1 to 3, characterized in that: The information related to the arc includes information of a center point of the arc.

7. A learning model that, when image data output from a visual sensor that captures a welding situation is input, outputs welding information corresponding to the image data, The learning model is characterized by: The image data used in generating the learning model is captured for a plurality of settings related to at least a protruding length of the welding wire and a positional relationship between the welding wire and the molten pool. The image data includes at least one of a molten pool, a welding wire, and an arc, The welding information includes at least one of information related to the change of the molten pool, information related to the position of the welding wire, and information related to the arc. The learning model is generated using the image data and the welding information corresponding to the image data in the teaching data.

8. A program product that enables a computer to realize a function of generating a learning model that, when image data outputted from a visual sensor that captures a welding situation is inputted, outputs welding information corresponding to the image data, The program product is characterized in that The function uses the image data and the welding information corresponding to the image data in teaching data, The image data used in generating the learning model is captured for a plurality of settings related to at least a protruding length of the welding wire and a positional relationship between the welding wire and the molten pool. The image data includes at least one of a molten pool, a welding wire, and an arc, The welding information includes at least one of information related to a change in a molten pool, information related to a position of a welding wire, and information related to an arc.

9. A welding system, characterized in that: have: A welding device for welding an object to be welded; A visual sensor that takes pictures of the welding process; a learning model that, when image data output from the visual sensor is input, outputs welding information corresponding to the image data; and a control device for controlling the movement of the welding device using the welding information output when the image data obtained by the visual sensor is input into the learning model, The image data used in generating the learning model is captured for a plurality of settings related to at least a protruding length of the welding wire and a positional relationship between the welding wire and the molten pool. The image data includes at least one of a molten pool, a welding wire, and an arc, The welding information includes at least one of information related to the change of the molten pool, information related to the position of the welding wire, and information related to the arc. The control device generates the learning model using the image data and the welding information corresponding to the image data in teaching data.

10. The welding system according to claim 9, characterized in that The welding system is provided with another learning model that inputs the welding information output from the learning model and outputs a correction signal for the welding information, in addition to the learning model that outputs the welding information corresponding to the image data.

Images