Welding equipment, welding method, and program

Abstract

To provide a welding device that can perform automatic welding with a higher welding quality, using a learning model learning a welding skill of a skilled welder.SOLUTION: A welding device, which performs welding by controlling at least one object to be controlled of a welding wire and an electrode, comprises: a camera that photographs an image of welding including tips of the object to be controlled and a welding position; a data generating part that generates data for learning that includes images for learning which are images of welding photographed in the past and information showing a degree of properness of operation of the object to be controlled determined by a welder who looks at the images for learning; a leaning part that constitutes a learning model using the data for learning; an estimating part that estimates a degree of properness of operation of the object to be controlled in the obtained images of welding, using the learning model; and a control part that controls the object to be controlled, on the basis of a result of estimation of the degree of properness of the operation.SELECTED DRAWING: Figure 2

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , , , ,

[0001] The present disclosure relates to a welding apparatus, a welding method, and a program.

Background Art

[0002] For the purpose of improving welding quality (reducing defects), the need for welding automation is increasing. For example, Patent Document 1 describes an automatic welding system that detects feature points such as electrodes and the ends of molten pools from an image of a welded portion and controls the weaving width of the electrode based on the coordinate positions of the feature points.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Skilled welders not only use the positions of electrodes and the ends of molten pools, but also use information that is difficult to quantify, such as the shape of the molten pool and the extent to which the molten pool covers the groove boundary of the base material, as judgment materials to adjust the position of the welding torch. However, in conventional automatic welding systems, information that is difficult to quantify other than coordinate information cannot be used for automatic control. Therefore, it has been difficult for conventional automatic welding systems to improve welding quality to a level close to that of skilled welders.

[0005] The present disclosure has been made in view of such problems, and provides a welding apparatus, a welding method, and a program that can perform automatic welding with higher welding quality using a learning model that has learned the welding skills of skilled welders.

Means for Solving the Problems

[0006] According to one aspect of the present disclosure, a welding apparatus that performs welding by controlling at least one of a welding wire and an electrode is equipped with: a camera that takes a welding image including the tip of the controlled object and the welding location; a data generation unit that generates learning data including a learning image which is a welding image taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by a welder based on the learning image; a learning unit that uses the learning data to construct a learning model that takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; an estimation unit that uses the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; a control unit that controls the controlled object based on the estimation result of the degree of appropriateness of the operation; and a first sensor provided in a wire nozzle that feeds the welding wire and outputs first sensor information that can detect the contact state of the welding wire with the welding location, wherein the learning unit constructs the learning model using the learning data which further includes the operation of the welding wire identified based on the first sensor information.

[0007] According to one aspect of the present disclosure, a welding apparatus that performs welding by controlling at least one of a welding wire and an electrode is equipped with: a camera that takes a welding image including the tip of the controlled object and the welding location; a data generation unit that generates learning data including a learning image which is a welding image taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by a welder based on the learning image; a learning unit that uses the learning data to construct a learning model that takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; an estimation unit that uses the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; a control unit that controls the controlled object based on the estimation result of the degree of appropriateness of the operation; and a second sensor provided in the welding apparatus that outputs second sensor information capable of detecting at least one of the position and orientation of the welding apparatus, wherein the learning unit constructs the learning model using the learning data which further includes the second sensor information.

[0008] According to one aspect of the present disclosure, a welding method for performing welding by controlling at least one of a welding wire and an electrode comprises the steps of: taking a welding image including the tip of the controlled object and the welding location; generating learning data including a learning image which is a welding image taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by a welder based on the learning image; constructing a learning model using the learning data that takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; estimating the degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model; controlling the controlled object based on the estimation result of the degree of appropriateness of the operation; and a first sensor provided on a wire nozzle that feeds the welding wire outputs first sensor information that can detect the contact state of the welding wire with the welding location, wherein the step of constructing the learning model is to construct the learning model using the learning data which further includes the operation of the welding wire identified based on the first sensor information.

[0009] According to one aspect of the present disclosure, the program causes a welding apparatus to perform welding by controlling at least one of a welding wire and an electrode to be a controlled object, to perform welding, to execute the following steps: taking a welding image including the tip of the controlled object and the welding location; generating learning data including a learning image which is a welding image taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by a welder based on the learning image; constructing a learning model using the learning data that takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; estimating the degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model; controlling the controlled object based on the estimation result of the degree of appropriateness of the operation; and outputting first sensor information which enables a first sensor provided on a wire nozzle that feeds the welding wire to detect the contact state of the welding wire with the welding location, wherein the step of constructing the learning model is to construct the learning model using the learning data which further includes the operation of the welding wire identified based on the first sensor information.

[0010] According to one aspect of the present disclosure, a welding method for performing welding by controlling at least one of a welding wire and an electrode is a welding method comprising the steps of: taking a welding image including the tip of the controlled object and the welding location; generating learning data including a learning image which is a welding image taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by a welder based on the learning image; constructing a learning model using the learning data that takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; estimating the degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model; controlling the controlled object based on the estimation result of the degree of appropriateness of the operation; and outputting second sensor information provided in the welding apparatus that can detect at least one of the position and orientation of the welding apparatus, wherein the step of constructing the learning model is to construct the learning model using the learning data which further includes the second sensor information.

[0011] According to one aspect of the present disclosure, the program causes a welding apparatus to perform welding by controlling at least one of a welding wire and an electrode to be a controlled object, to perform welding, to take a welding image including the tip of the controlled object and the welding location; to generate learning data including a learning image which is a welding image taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by a welder based on the learning image; to construct a learning model using the learning data that takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model; to control the controlled object based on the estimation result of the degree of appropriateness of the operation; and to output second sensor information provided in the welding apparatus that can detect at least one of the position and orientation of the welding apparatus, wherein the step of constructing the learning model is to construct the learning model using the learning data which further includes the second sensor information. [Effects of the Invention]

[0012] According to the welding apparatus, welding method, and program described herein, automated welding with higher welding quality can be performed using a learning model that has learned the welding skills of a skilled welder. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram showing the overall configuration of a welding apparatus according to the first embodiment of this disclosure. [Figure 2] This is a block diagram showing the functional configuration of a control device according to the first embodiment of the present disclosure. [Figure 3] This flowchart shows an example of processing in learning mode for a welding apparatus according to the first embodiment of this disclosure. [Figure 4] This figure shows an example of training data according to the first embodiment of this disclosure. [Figure 5] This flowchart shows an example of processing during the control mode of a welding apparatus according to the first embodiment of this disclosure. [Figure 6] This is a block diagram showing the functional configuration of a control device according to a second embodiment of the present disclosure. [Figure 7] This figure shows an example of training data according to the third embodiment of this disclosure. [Figure 8] This is a schematic diagram showing the overall configuration of a welding apparatus according to the fourth embodiment of this disclosure. [Figure 9] This figure shows an example of training data according to the fourth embodiment of this disclosure. [Figure 10] This is a schematic diagram showing the overall configuration of a welding apparatus according to the fifth embodiment of this disclosure. [Figure 11] This figure shows an example of training data according to the fifth embodiment of this disclosure. [Figure 12] This is a diagram illustrating the functions of a control device according to the sixth embodiment of this disclosure. [Figure 13] This is a block diagram showing the functional configuration of a welding apparatus according to another embodiment of the present disclosure.

Mode for Carrying Out the Invention

[0014] <First Embodiment> Hereinafter, the welding apparatus 1 according to the first embodiment of the present disclosure will be described with reference to FIGS. 1 to 5.

[0015] (Overall Configuration of Welding Apparatus) FIG. 1 is a schematic diagram showing the overall configuration of the welding apparatus according to the first embodiment of the present disclosure. As shown in FIG. 1, the welding apparatus 1 includes a main body portion 10 and a control device 20.

[0016] The main body portion 10 executes arc welding on the base material 3 according to the control of the control device 20. The main body portion 10 includes a welding torch 100, a welding wire supply unit 110, and a camera 120.

[0017] An electrode 101 made of tungsten or the like is attached to the welding torch 100.

[0018] The welding wire supply unit 110 sends out the welding wire 40 toward the welding portion of the base material 3 through the wire nozzle 111.

[0019] When the welding torch 100 supplies power to the electrode 101, the base material 3 (groove 31) is melted by arc discharge from the electrode 101, and the tip portion of the welding wire 40 is supplied to the molten portion of the base material 3 and melted, thereby forming a molten pool 41 in which the base material and the welding wire are diluted. When this molten pool cools and solidifies, the base material 3 is welded.

[0020] The camera 120 captures a welding image including the tip of the electrode 101 and the welding wire 40 and the welding portion of the base material 3.

[0021] The control device 20 outputs a control signal to control at least one of the control targets, the electrode 101 and the welding wire 40, based on the welding image captured by the camera 120. In this embodiment, as shown in Figure 1, an example is described in which the welding apparatus 1 is a so-called TIG welding apparatus that uses an electrode 101 made of tungsten or the like. Furthermore, an example is described in which the control device 20 according to this embodiment controls both the electrode 101 and the welding wire 40. In this case, the control of the electrode 101 also includes control of swinging the electrode from side to side (weaving width).

[0022] In other embodiments, the welding apparatus 1 may be a MAG welding apparatus, a MIG welding apparatus, or the like. In this case, the welding torch 100 uses welding wire 40 supplied from the welding wire supply unit 110 into the welding torch 100 instead of the electrode 101. Also, in this case, the control device 20 controls only the welding wire 40.

[0023] The control signals are signals that indicate, for example, the position of the electrode 101, the position of the welding wire 40, and the amount of welding wire 40 to be pulled out (feeded out). The welding torch 100 moves the position of the electrode 101 in the left-right direction (±Y direction) according to the control signals of the control device 20. The welding wire supply unit 110 moves the position of the welding wire 40 in the up-down direction (±Z direction) and the left-right direction (±Y direction) according to the control signals of the control device 20. The welding wire supply unit 110 also feeds the welding wire 40 toward the base material 3 according to the control signals of the control device 20.

[0024] (Functional configuration of the control unit) Figure 2 is a block diagram showing the functional configuration of a control device according to the first embodiment of this disclosure. As shown in Figure 2, the control device 20 includes a processor 21, a memory 22, a storage 23, and a communication interface 24.

[0025] The processor 21 performs the functions of an acquisition unit 210, an estimation unit 211, a control unit 212, a data generation unit 213, and a learning unit 214 by operating according to a predetermined program.

[0026] The acquisition unit 210 acquires welding images captured by the camera 120 of the main unit 10.

[0027] The estimation unit 211 takes a welding image as input and uses a learning model M that outputs the degree of appropriateness of the operation of the welding wire 40 and electrode 101 to estimate the degree of appropriateness of the operation of the welding wire 40 and electrode 101 in the acquired welding image. The estimation unit 211 reads and uses a learning model M that has been previously trained by the learning unit 214 (described later) from the storage 23.

[0028] The estimation unit 211 estimates, for example, the appropriateness of the operation for moving the electrode 101 in the left-right and up-down directions in three stages: "excessive," "appropriate," and "insufficient." The estimation unit 211 also estimates the operation for moving the welding wire 40 in the left-right and up-down directions in three stages: "excessive," "appropriate," and "insufficient." The estimation unit 211 may further express the operation regarding the amount of welding wire 40 pulled out in three stages: "excessive," "appropriate," and "insufficient." In other embodiments, the evaluation of the operation may be subdivided into more than three stages.

[0029] The control unit 212 controls the welding wire 40 and electrode 101 based on the estimation results of the estimation unit 211. For example, if the control unit 212 estimates that the leftward movement of electrode 101 is "excessive," it generates a control signal to move electrode 101 to the right, and if it estimates that the movement is "insufficient," it generates a control signal to move electrode 101 to the left, and outputs this signal to the welding torch 100. As a result, the welding torch 100 moves electrode 101 left or right according to the control signal. Also, if the control unit 212 evaluates that the leftward movement of electrode 101 is "appropriate," it does not control the left or right movement of electrode 101 (it does not output a control signal, or it outputs a control signal that maintains the left or right position). As a result, the welding torch 100 does not move the left or right position of electrode 101 until it receives the next control signal.

[0030] The data generation unit 213 generates training data that includes training images, which are welding images taken in the past, and information indicating the degree of appropriateness of the welding wire 40 and electrode 101 operations performed by the welder when the training images were taken.

[0031] The learning unit 214 constructs a learning model M by performing deep learning using the training data.

[0032] Memory 22 has a memory area necessary for the operation of the processor 21.

[0033] Storage 23 is a so-called auxiliary storage device, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). Storage 23 stores welding images taken by the camera 120 and the trained learning model M.

[0034] The communication interface 24 is an interface for sending and receiving various types of information (signals) with an external device (such as the main unit 10).

[0035] The predetermined program executed by the processor 21 of the control device 20 is stored on a computer-readable recording medium. A computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, this computer program may be distributed to a computer via a communication line, and the computer that receives the distribution may execute the program. Furthermore, this program may be intended to implement only a part of the functions described above. Moreover, it may be a program that can implement the above functions in combination with a program already recorded in the computer system, a so-called differential file (differential program).

[0036] The control device 20 according to this embodiment has two modes: a learning mode and a control mode. In learning mode, the control device 20 constructs a learning model M using the learning unit 214. In control mode, the control device 20 uses the learning model M learned by the learning unit 214 to control the main body 10 of the welding apparatus 1 (automatic execution of welding). The details of the processing in each mode will be described below.

[0037] (Processing flow in learning mode) Figure 3 is a flowchart showing an example of processing in learning mode of a welding apparatus according to the first embodiment of this disclosure. Here, with reference to Figure 3, we will explain the processing flow of the welding apparatus 1 in learning mode.

[0038] First, the learning unit 214 of the control device 20 acquires welding images and operation information to be used for learning (step S100). For example, the learning unit 214 acquires multiple welding images taken of welding performed in the past by a welder (skilled welder), and associates these images with the details of the operations (operation information) that the welder performed on the electrode 101 and welding wire 40 when each welding image was taken.

[0039] Furthermore, the data generation unit 213 creates training data D1 and evaluation data D2 based on the acquired welding images and operation information (step S101).

[0040] Figure 4 shows an example of training data according to the first embodiment of this disclosure. For example, as shown in Figure 4, the data generation unit 213 generates training data D1 by adding labels (ground truth data) to each welding image indicating the degree of appropriateness of the operation of the electrode 101 and the welding wire 40. For example, one of the following is input for each of the electrode 101 and the welding wire 40: "excessive," "appropriate," or "insufficient." Similarly, the data generation unit 213 labels all welding images to indicate which direction the operation is "excessive," "appropriate," or "insufficient." Alternatively, the welder may specify the labels by looking at each welding image. The data generation unit 213 generates training data D1 by adding the labels specified by the welder to each welding image. The data generation unit 213 also separates a portion of the labeled training data D1 as evaluation data D2.

[0041] The data generation unit 213 may also determine the appropriateness of the operation of the electrode 101 and welding wire 40 in each welding image based on the time series of operation information and automatically add labels. For example, suppose the operation information associated with the welding image shown in Figure 4 and the operation information from the previous time are detected to indicate that the welder stopped moving the electrode 101 to the right and moved it to the left, and stopped the welding wire 40 (no operation). In this case, the data generation unit 213 adds a label to this welding image indicating that the operation of the electrode 101 in the left-right direction (to the right) is "excessive" and the operation of the welding wire 40 in the left-right direction is "appropriate". Also, for example, if the time series of operation information detects an operation in which the electrode 101 continues to move to the right, the data generation unit 213 adds a label indicating that the operation of the electrode 101 in the left-right direction (to the right) is "insufficient".

[0042] Next, the learning unit 214 constructs a learning model M by performing deep learning using the learning data D1, which consists of welding images (training images) and labels (ground truth data) (step S102).

[0043] For example, as shown in Figure 4, a welding image taken of a welded area includes various elements such as the base material 3, groove 31, electrode 101, welding wire 40, molten pool 41, bead 42, and arc A. As described above, a skilled welder adjusts the manipulation amount of the electrode 101 and welding wire 40 by taking into account not only positional information such as the position of the edge of the molten pool 41 and the tip positions of the electrode 101 and welding wire 40, but also information that is difficult to quantify, such as how well the molten pool 41 covers the boundary of the groove 31 (groove wettability) and how well the molten pool 41 covers the welded area (on the bead 42). In the welding apparatus 1 according to this embodiment, the learning unit 214 learns the relationship between the feature quantities of the welding image and the operations performed by the welder after viewing this welding image, thereby enabling the incorporation of information that is difficult to quantify into the learning model M. As a result, the learning unit 214 can construct a learning model M that can make estimations closer to the judgment of a skilled welder based on the welding image.

[0044] Next, the learning unit 214 evaluates the learning model M constructed in step S102 using the training data D1 (step S103). The same dataset as in step S102 is used for this training data D1. Specifically, the learning unit 214 inputs only the welding images from the training data D1 into the learning model M and outputs the degree of appropriateness of the manipulation amount for each welding image. Based on the output results of each welding image of the learning model M and the ground truth data, the learning unit 214 evaluates the performance of the training data D1.

[0045] Furthermore, the learning unit 214 evaluates the learning model M constructed in step S102 using evaluation data D2 (step S104). This evaluation data D2 is a dataset not used for training the learning model M. The learning unit 214 inputs only the welding images from the evaluation data D2 into the learning model M and outputs the degree of appropriateness of the manipulation amount for each welding image. The learning unit 214 evaluates the performance of the evaluation data D2 based on the output results of each welding image from the learning model M and the ground truth data. The learning unit 214 also calculates, for example, the accuracy rate and loss function as performance evaluation results for the learning data D1 and evaluation data D2, respectively.

[0046] Furthermore, the learning unit 214 stores the learning model M, the performance evaluation results of the learning data D1, and the performance evaluation results of the evaluation data D2 in the storage 23 (step S105).

[0047] Next, the learning unit 214 determines whether the termination condition has been met (step S106). For example, the learning unit 214 determines that the termination condition has been met if both the performance evaluation result of the training data D1 and the performance evaluation result of the evaluation data D2 converge to a predetermined value or higher (step S106; YES). In this case, the learning unit 214 adopts the learning model M with the highest performance evaluation result among the multiple learning models M constructed as the model to be used in control mode (step S107).

[0048] On the other hand, if the termination condition is not met (step S106; NO), the learning unit 214 returns to step S102 and constructs a new learning model M.

[0049] (Processing flow in control mode) Figure 5 is a flowchart showing an example of processing in the control mode of a welding apparatus according to the first embodiment of this disclosure. Here, with reference to Figure 5, we will explain the processing flow of the welding apparatus 1 in control mode.

[0050] When the welding apparatus 1 automatically performs welding on the base material 3, the camera 120 photographs the welding area of ​​the base material 3. The acquisition unit 210 of the control device 20 acquires the welding image captured by the camera 120 (step S110).

[0051] Next, the estimation unit 211 estimates the degree of appropriateness of the operation of the electrode 101 and welding wire 40 at the present time (step S111). Specifically, the estimation unit 211 inputs the acquired welding image into the learning model M and obtains an output estimation result (information indicating either "over-excessive," "appropriate," or "under-excessive").

[0052] Next, the control unit 212 generates control signals to control the electrode 101 and the welding wire 40 based on the estimation results of the estimation unit 211, and outputs them to the welding torch 100 and the welding wire supply unit 110, respectively (step S112). For example, as shown in the example in Figure 4, if the movement of the electrode 101 to the right is "excessive", the control unit 212 generates a control signal to move the electrode 101 to the left and outputs it to the welding torch 100. Also, as shown in the example in Figure 4, if the movement of the welding wire 40 downwards is "insufficient", the control unit 212 generates a control signal to move the electrode 101 downward and outputs it to the welding torch 100. Furthermore, as shown in the example in Figure 4, if the downward movement of the electrode 101 and the rightward movement of the welding wire 40 are "appropriate," the control unit 212 determines that vertical movement of the electrode 101 and horizontal movement of the welding wire 40 are unnecessary, and does not need to output control signals related to these movements to the welding wire supply unit 110.

[0053] The welding apparatus 1 automatically performs welding by continuously repeating the series of processes shown in Figure 5.

[0054] (Effect, Action) As described above, the welding apparatus 1 according to this embodiment includes a camera 120 that captures a welding image including the tip of the electrode 101 and welding wire 40 and the welding location of the base material 3; a learning unit 214 that constructs a learning model M using learning data including learning images which are welding images taken in the past and the operation of the electrode 101 and welding wire 40 judged by the welder based on the learning images; an estimation unit 211 that estimates the degree of appropriateness of the operation in the acquired welding image using the learning model M; and a control unit 212 that controls the electrode 101 and welding wire 40 based on the estimation result of the estimation unit 211.

[0055] For example, even if the positional relationship of electrodes, welding wire, and molten pool is the same, a welder may make different judgments about whether the operation of the electrodes and welding wire is appropriate, excessive, or insufficient, based on a combination of various conditions of the weld area obtained from the welding image (e.g., groove wettability). Therefore, if the electrodes and welding wire are controlled based only on the positional coordinates of each part, as in conventional technology, the control may differ from the welder's judgment, potentially resulting in lower welding quality than manual welding by the welder. In contrast, as described above, the welding apparatus 1 according to this embodiment learns the relationship between the feature quantities of the training image and the operations performed by the welder while observing the state of the weld area in the training image. This makes it possible to incorporate not only the positional information of each part but also information that the welder uses to make welding judgments, which is difficult to quantify, into the learning model M. As a result, in control mode, the welding apparatus 1 can estimate the degree of appropriateness of the operation (e.g., whether the operation is "excessive," "appropriate," or "insufficient") at a level close to the welder's judgment using the learning model M. Furthermore, the welding apparatus 1 controls the electrode 101 and welding wire 40 based on this estimation result to perform automatic welding, thereby approaching the welding quality of a manual weld performed by a welder. In other words, the welding apparatus 1 according to this embodiment can perform automatic welding with higher welding quality compared to conventional technology that controls based only on the positional information of each part in the welding image.

[0056] <Second Embodiment> Next, a welding apparatus 1 according to the second embodiment of this disclosure will be described with reference to Figure 6. Components common to the first embodiment are denoted by the same reference numerals, and detailed descriptions are omitted.

[0057] Figure 6 shows an example of training data according to the second embodiment of this disclosure. As shown in Figure 6, in step S101 of Figure 3, the data generation unit 213 according to this embodiment generates training data D1, which includes training images in which areas not focused on by the welder in previously taken welding images are masked.

[0058] The welder observes the conformation of the molten pool 41 to the side of the groove 31 (change in the shape of the groove 31) and how the molten pool 41 covers the bead 42 to determine whether the amount of manipulation of the electrode 101 and welding wire 40 is appropriate. For example, as shown in Figure 7, the welder makes the determination of the amount of manipulation by referring to the area of ​​interest R1 in the welding image where these features appear. However, deep learning may learn easily recognizable features contained in the welding image (for example, areas with large changes in brightness such as around arc A), and may not be able to sufficiently learn the features of the area that the welder is actually focusing on (area of ​​interest R1).

[0059] Therefore, as shown in Figure 6, the data generation unit 213 according to this embodiment generates training data D1 in which the non-focused region R2 of the welding image that the welder does not focus on is arbitrarily masked. For example, for each welding image, the data generation unit 213 sets the region specified by the welder as the non-focused region R2. The non-focused region R2 may be one or more. For example, the non-focused region R2 may be a region that includes the electrode 101 or the arc A with large brightness changes, or a region that includes the groove shoulder (a part that is not welded).

[0060] Furthermore, in step S102 of Figure 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 which includes a masked welding image (learning image).

[0061] In this way, the welding apparatus 1 can effectively perform deep learning so that the learning model M can acquire common features in the area of ​​interest R1, even when it can only learn from welding images under specific conditions (the number of samples in the training data D1 is small). Furthermore, this enables the welding apparatus 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate based on the features included in the area of ​​interest R1 in the welding image when in control mode.

[0062] <Third Embodiment> Next, a welding apparatus 1 according to the third embodiment of this disclosure will be described with reference to Figure 7. Components common to each of the embodiments described above are denoted by the same reference numerals, and detailed descriptions are omitted.

[0063] Figure 7 shows an example of training data according to the third embodiment of this disclosure. As shown in Figure 7, in step S101 of Figure 3, the data generation unit 213 according to this embodiment generates training data D1 which includes training images from which contours of welding images taken in the past have been extracted.

[0064] As described in the second embodiment, deep learning may learn easily recognizable features contained in the welding image (for example, areas with large changes in brightness, such as around arc A), and may not adequately learn the features of the areas that a welder actually focuses on. Therefore, in order to suppress the learning of areas that a welder does not focus on, the data generation unit 213 in this embodiment generates training data D1 by omitting information unnecessary for operation decisions from the welding image.

[0065] Specifically, the data generation unit 213 applies known edge detection processing to the original welding image to generate an edge image by detecting and extracting only the contours of each part included in the welding image. The data generation unit 213 may also further process the edge image to remove information unnecessary for determining the operation. In the example in Figure 7, the data generation unit 213 processes the image of the electrode reflected on the side wall of the groove to remove it.

[0066] Furthermore, in step S102 of Figure 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 which includes a welding image (learning image) that is an edge image.

[0067] In this way, the welding apparatus 1 can efficiently learn the learning model M by using an image containing only the minimum necessary information, such as the edges of each part, for the input image, thereby omitting areas that the welder is not paying attention to. As a result, the welding apparatus 1 can accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate when in control mode. Consequently, the welding apparatus 1 can appropriately control the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation.

[0068] <Fourth Embodiment> Next, a welding apparatus 1 according to the fourth embodiment of this disclosure will be described with reference to Figures 8 and 9. Components common to each of the embodiments described above are denoted by the same reference numerals, and detailed descriptions are omitted.

[0069] Figure 8 is a schematic diagram showing the overall configuration of a welding apparatus according to the fourth embodiment of this disclosure. As shown in Figure 8, the welding apparatus 1 according to this embodiment further includes a torque sensor 112 (first sensor) provided on the wire nozzle 111 of the welding wire supply unit 110. The torque sensor 112 detects the force generated in the wire nozzle 111 when the tip of the welding wire 40 comes into contact with the base material 3.

[0070] In this embodiment, the acquisition unit 210 further acquires first sensor information from the torque sensor 112 in step S100 of Figure 3, which is capable of detecting the contact state of the welding wire 40 with the base material 3.

[0071] Figure 9 shows an example of training data according to the fourth embodiment of this disclosure. In step S101 of Figure 3, the data generation unit 213 detects the operation of the welding wire 40 based on the first sensor information and identifies the degree of appropriateness of the operation of the welding wire 40 in the vertical direction (±Z direction). The data generation unit 213 also generates training data D1 (Figure 9) by adding information (labels) indicating the degree of appropriateness of the operation of the welding wire 40 in the vertical direction (±Z direction) to the welding image acquired at the same time as the sensor information of the torque sensor 112. For example, if the time series of the first sensor information of the torque sensor 112 detects that the welder has performed an operation to continuously move the welding wire 40 downwards, the data generation unit 213 adds a label indicating that the operation of the welding wire 40 in the vertical direction (downward direction) is "insufficient" to the welding image corresponding to this sensor information. In other embodiments, the welder may specify a label by looking at a combination of the welding image and the first sensor information. In this case, the data generation unit 213 generates training data D1 with the label specified by the welder attached to the welding image.

[0072] In step S102 of Figure 3, the learning unit 214 trains the learning model M based on the training data D1 generated by the data generation unit 213.

[0073] As shown in the example in Figure 8, when the camera 120 is positioned diagonally above and in front of the welding area (+Z direction and +X direction), the difference in the position of the welding wire 40 in the welding image becomes small, especially when the rigidity of the welding wire 40 is low and it is soft (easily melted). In such cases, it may be difficult to determine the vertical position (±Z direction) of the welding wire 40 from the welding image. In contrast, the welding apparatus 1 according to this embodiment can construct a learning model M that learns the appropriateness of the vertical operation of the welding wire 40 from the contact state of the welding wire 40 detected by the torque sensor 112.

[0074] Furthermore, in control mode, the acquisition unit 210 acquires the welding image along with the first sensor information of the torque sensor 112 in step S110 of Figure 5. In step S111 of Figure 5, the estimation unit 211 inputs the welding image and the first sensor information into the learning model M and outputs the degree of appropriateness of the operation of the electrode 101 and the welding wire 40 at the current time. In step S112 of Figure 5, the control unit 212 generates and outputs control signals to control the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation of the electrode 101 and the welding wire 40. As a result, the welding apparatus 1 can accurately estimate whether the operation of the welding wire 40 in the vertical direction is appropriate when in control mode. Therefore, the welding apparatus 1 can appropriately control the vertical direction of the welding wire 40 and further improve the welding quality.

[0075] In the fourth embodiment, an example was described in which the data generation unit 213 generates learning data D1 with information (labels) indicating the appropriateness of the vertical operation of the welding wire 40 based on the sensor information of the torque sensor 112, but the invention is not limited to this. In other embodiments, the data generation unit 213 may directly include the first sensor information of the torque sensor 112 in the learning data D1. The learning unit 214 learns the appropriateness of the horizontal operation of the electrode 101 and the appropriateness of the horizontal and vertical operation of the welding wire 40 based on the welding image, the labels, and the first sensor information of the torque sensor 112. Even in such an embodiment, the welding apparatus 1 can accurately estimate whether the vertical operation of the welding wire 40 is appropriate or not.

[0076] <Fifth Embodiment> Next, a welding apparatus 1 according to the fifth embodiment of this disclosure will be described with reference to Figures 10 to 11. Components common to each of the embodiments described above are denoted by the same reference numerals, and detailed descriptions are omitted.

[0077] Figure 10 is a schematic diagram showing the overall configuration of a welding apparatus according to a fifth embodiment of the present disclosure. As shown in Figure 10, the welding apparatus 1 according to this embodiment further includes a sensor 102 (second sensor) provided on the welding torch 100 for detecting at least one of the position and orientation of the welding torch 100. Sensor 102 is, for example, an acceleration sensor capable of detecting the orientation (angle) of the welding torch 100. Sensor 102 is also an encoder capable of detecting the position and orientation of the welding torch 100. Furthermore, sensor 102 may be a combination of an acceleration sensor and an encoder. The position and orientation of the welding torch 100 are expressed, for example, as relative position and relative angle with respect to a reference position (for example, initial position and initial angle when the welding apparatus 1 is installed).

[0078] In this embodiment, the acquisition unit 210 further acquires second sensor information from the sensor 102 in step S100 of Figure 3, which is capable of detecting the position or orientation of the welding wire 40.

[0079] Figure 11 shows an example of training data according to the fifth embodiment of this disclosure. In step S101 of Figure 3, the data generation unit 213 generates training data D1 (Figure 11) which includes second sensor information from sensor 102.

[0080] In step S102 of Figure 3, the learning unit 214 trains the learning model M based on the training data D1 generated by the data generation unit 213.

[0081] When the position and orientation of the welding torch 100 change during welding, the condition of the welded area (for example, the flow rate of the molten pool 41) changes. Therefore, in cases where welding is performed while changing the position and orientation of the welding torch 100, if the learning model M is trained based only on welding images, the accuracy of estimating the appropriateness of the operation may decrease. In contrast, the welding apparatus 1 according to this embodiment can learn a learning model M that takes into account the position and orientation of the welding torch 100 by using learning data D1 which includes second sensor information capable of detecting the position and orientation of the welding torch 100.

[0082] Furthermore, in control mode, the acquisition unit 210 acquires the welding image along with the second sensor information of the sensor 102 in step S110 of Figure 5. In step S111 of Figure 5, the estimation unit 211 inputs the welding image and the second sensor information into the learning model M and outputs the degree of appropriateness of the operation of the electrode 101 and the welding wire 40 according to the position and orientation of the welding apparatus 1 at the time the welding image was taken. In addition, in step S112 of Figure 5, the control unit 212 generates and outputs control signals for controlling the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation of the electrode 101 and the welding wire 40.

[0083] With this configuration, the welding apparatus 1 can accurately estimate the degree of appropriateness of the operation of the electrode 101 and welding wire 40, even when welding is performed while changing the position and orientation of the welding torch 100. As a result, the welding apparatus 1 can appropriately control the electrode 101 and welding wire 40 in response to changes in the position and orientation of the welding torch 100, thereby improving welding quality.

[0084] <Sixth Embodiment> Next, a welding apparatus 1 according to the sixth embodiment of this disclosure will be described with reference to Figure 12. Components common to each of the embodiments described above are denoted by the same reference numerals, and detailed descriptions are omitted.

[0085] If the welding image changes due to variations in the setting of the welding apparatus 1 (such as the arrangement of the welding torch 100, welding wire supply unit 110, and camera 120) or variations in the setting of the base material 3, it may not be possible to effectively train the learning model M. For this reason, in step S101 of Figure 3, the data generation unit 213 according to this embodiment generates learning data D1 which includes learning images that have been processed to simulate variations in the relative position between the welding apparatus 1 and the base material 3 by applying image processing to welding images taken in the past.

[0086] Figure 12 is a diagram illustrating the functions of a control device according to the sixth embodiment of this disclosure. Specifically, as shown in Figure 12, the data generation unit 213 generates a plurality of processed images by applying various image processing to the original welding image, and generates training data D1 including each processed image.

[0087] For example, the data generation unit 213 generates processed images by translating or rotating the welding image to generate training data D1 that simulates variations in the setting of the welding apparatus 1 and the base material. The data generation unit 213 generates processed images by enlarging or reducing the welding image to generate training data D1 that simulates variations in the field of view due to focusing. The data generation unit 213 generates processed images by changing the brightness of the welding image to generate training data D1 that simulates variations in brightness. The data generation unit 213 generates processed images by adding Gaussian noise to the welding image to generate training data D1 that simulates the case where the welding image was taken in a blurry state. The data generation unit 213 generates processed images by changing the aspect ratio of the welding image to generate training data D1 that simulates variations in the distance between the electrode 101 and the welding wire 40.

[0088] Furthermore, in step S102 of Figure 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 including the original welding image and the learning data D1 including the processed image (learning image).

[0089] In this way, the welding apparatus 1 can construct a learning model M that takes into account variations in settings and other factors. As a result, the welding apparatus 1 can accurately control the electrode 101 and welding wire 40 in control mode without being affected by variations in settings and other factors.

[0090] <Other Embodiments> Figure 13 is a block diagram showing the functional configuration of a welding apparatus according to another embodiment of the present disclosure. In the embodiments described above, examples were given in which the control device 20 has a data generation unit 213 that generates training data D1 and evaluation data D2, but the invention is not limited to this. For example, as shown in Figure 13, the welding apparatus 1 may also include a data generation server 50 having a data generation unit 213, separate from the control device 20. In this case, the training data D1 and evaluation data D2 are generated in the data generation unit 213 of the data generation server 50. The control device 20 also acquires the training data D1 and evaluation data D2 from the data generation server 50 via the communication interface 24 and stores them in the storage 23. Even with such a configuration, the welding apparatus 1 can obtain the same effects as in the embodiments described above.

[0091] As described above, several embodiments relating to this disclosure have been explained, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0092] <Note> The welding apparatus, welding method, and program described in the above-described embodiment can be understood, for example, as follows:

[0093] (1) According to a first aspect of the present disclosure, a welding apparatus 1 that performs welding by controlling at least one of a welding wire 40 and an electrode 101 is equipped with: a camera 120 that takes a welding image including the tip of the controlled object and the welding location; a data generation unit that generates learning data D1 including learning images which are welding images taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder by looking at the learning images; a learning unit 214 that uses the learning data D1 to construct a learning model M that takes a welding image as input and outputs the degree of appropriateness of the operation of the controlled object; an estimation unit 211 that uses the learning model M to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; and a control unit 212 that controls the controlled object based on the estimation result of the degree of appropriateness of the operation.

[0094] In this way, the welding device 1 learns the relationship between the features of the training images and the operations performed by the welder while observing the state of the welding area in the image when the training image is taken. This makes it possible to incorporate not only the positional information of each part in the welding image, but also information that the welder uses to make welding decisions, which is difficult to quantify, into the learning model M. In this way, the welding device 1 can perform high-quality automatic welding in control mode by estimating the appropriateness of the operation using the learning model M, which has learned the judgment of a skilled welder.

[0095] (2) According to a second aspect of the present disclosure, in the welding apparatus 1 according to the first aspect, the data generation unit 213 generates training data D1 which includes training images in which areas not of interest to the welder in welding images taken in the past are masked.

[0096] In this way, the welding apparatus 1 can effectively perform deep learning so that the learning model M can acquire common features in the area of ​​interest R1, even when it can only learn from welding images under specific conditions (the number of samples in the training data D1 is small). Furthermore, this enables the welding apparatus 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate based on the features included in the area of ​​interest R1 in the welding image when in control mode.

[0097] (3) According to a third aspect of the present disclosure, in the welding apparatus 1 according to the first aspect, the data generation unit 213 generates training data D1 which includes training images from which contours of welding images taken in the past have been extracted.

[0098] In this way, the welding apparatus 1 can efficiently learn the learning model M by using an image containing only the minimum necessary information, such as the edges of each part, for the input image, thereby omitting areas that the welder is not paying attention to. As a result, the welding apparatus 1 can accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate when in control mode. Consequently, the welding apparatus 1 can appropriately control the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation.

[0099] (4) According to a fourth aspect of the present disclosure, in a welding apparatus 1 according to any one of the first to third aspects, the data generation unit 213 generates training data D1 which includes training images that have been processed to simulate variations in the relative positions of the controlled object, the camera 120, and the welding location on welding images taken in the past.

[0100] In this way, the welding apparatus 1 can construct a learning model M that takes into account variations in settings and other factors. As a result, the welding apparatus 1 can accurately control the electrode 101 and welding wire 40 in control mode without being affected by variations in settings and other factors.

[0101] (5) According to a fifth aspect of the present disclosure, the welding apparatus 1 according to any one of the first to fourth aspects further comprises a first sensor 112 provided on a wire nozzle 111 that feeds out a welding wire 40 and outputs first sensor information capable of detecting the contact state of the welding wire 40 with the welding location, and the learning unit 214 constructs a learning model M using learning data D1 which further includes the operation of the welding wire 40 identified based on the first sensor information.

[0102] (6) According to a sixth aspect of the present disclosure, the welding apparatus 1 according to any one of the first to fifth aspects further comprises a second sensor 102 provided in the welding apparatus 1 that outputs second sensor information capable of detecting at least one of the position and orientation of the welding apparatus 1, and the learning unit 214 constructs a learning model M using learning data D1 which further includes the second sensor information.

[0103] In this way, the welding apparatus 1 can accurately estimate the degree of appropriateness of the operation of the electrode 101 and the welding wire 40, even when welding is performed while changing the position and orientation of the welding torch 100. As a result, the welding apparatus 1 can appropriately control the electrode 101 and the welding wire 40 in accordance with changes in the position and orientation of the welding torch 100.

[0104] (7) According to a seventh aspect of the present disclosure, a welding method for performing welding by controlling at least one of a welding wire 40 and an electrode 101 is a welding method comprising the steps of: taking a welding image including the tip of the controlled object and the welding location; generating learning data D1 including learning images which are welding images taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder by looking at the learning images; constructing a learning model M using the learning data D1 which takes a welding image as input and outputs the degree of appropriateness of the operation of the controlled object; estimating the degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model M; and controlling the controlled object based on the estimation result of the degree of appropriateness of the operation.

[0105] (8) According to the eighth aspect of the present disclosure, the program causes a welding apparatus 1 that performs welding by controlling at least one of the welding wire 40 and the electrode 101 to perform welding to take a welding image including the tip of the controlled object and the welding location; to generate learning data D1 including learning images which are welding images taken in the past and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder by looking at the learning images; to construct a learning model M using the learning data D1 that takes a welding image as input and outputs the degree of appropriateness of the operation of the controlled object; to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model M; and to control the controlled object based on the estimation result of the degree of appropriateness of the operation. [Explanation of Symbols]

[0106] 1. Welding equipment 10 Main body 100 welding torches 101 Electrode 102 Sensor (Second Sensor) 110 Welding wire supply unit 111 Wire Nozzle 112 Torque sensor (1st sensor) 120 Cameras 20 Control device 21 processors 210 Acquisition Department 211 Estimation Department 212 Control Unit 213 Data Generation Unit 214 Learning Department 22 memory 23 Storage 24 Communication Interfaces 3 Base material 40 welding wires 50 Data Generation Servers

Claims

1. A welding apparatus that performs welding by controlling at least one of the controllable objects, a welding wire and an electrode, A camera that captures a welding image including the tip of the controlled object and the welding location, A data generation unit that generates learning data including learning images which are welding images taken in the past, and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder based on the learning images, A learning unit constructs a learning model that uses the aforementioned training data as input to the welding image and outputs the degree of appropriateness of the operation of the controlled object. An estimation unit that uses the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image, Based on the estimation result of the appropriateness of the operation, a control unit controls the controlled object, A first sensor is provided in the wire nozzle that feeds the welding wire, and outputs first sensor information capable of detecting the force generated when the welding wire comes into contact with the welding location. Equipped with, The learning unit constructs the learning model using the learning data, which further includes operations that move the welding wire identified based on the first sensor information in the vertical direction, with the direction away from the welding location being the upward direction and the direction towards the welding location being the downward direction. Welding equipment.

2. A welding apparatus that performs welding by controlling at least one of the controllable objects, a welding wire and an electrode, A camera that captures a welding image including the tip of the controlled object and the welding location, A data generation unit that generates learning data including learning images which are welding images taken in the past, and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder based on the learning images, A learning unit constructs a learning model that uses the aforementioned training data as input to the welding image and outputs the degree of appropriateness of the operation of the controlled object. An estimation unit that uses the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image, Based on the estimation result of the appropriateness of the operation, a control unit controls the controlled object, A second sensor provided in the welding apparatus outputs second sensor information capable of detecting at least one of the position and orientation of the welding apparatus, Equipped with, The learning unit constructs a learning model by using the learning data, which further includes the second sensor information, to learn the changes in the state of the welding area in the learning image in response to changes in at least one of the position and orientation of the welding apparatus in addition to the operation of the controlled object, and the degree of appropriateness of the operation of the controlled object as judged by the welder. Welding equipment.

3. The data generation unit generates training data, which includes the training image in which areas not focused on by the welder in the welding image taken in the past are masked. The welding apparatus according to claim 1 or 2.

4. The data generation unit generates training data including the training image from which the contours of the welding images taken in the past have been extracted. The welding apparatus according to claim 1 or 2.

5. The data generation unit generates training data, which includes the training images, to which image processing has been applied to previously captured welding images to simulate variations in the relative positions of the controlled object, the camera, and the welding location. The welding apparatus according to claim 1 or 2.

6. A welding method that performs welding by controlling at least one of the welding wire and the electrode, The steps include taking a welding image that includes the tip of the controlled object and the welding location, A step of generating learning data that includes learning images, which are welding images taken in the past, and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder based on the learning images, The steps include: constructing a learning model using the aforementioned training data, which takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; The steps include: using the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; A step of controlling the controlled object based on the estimation result of the appropriateness of the operation, The first sensor, provided in the wire nozzle that feeds the welding wire, outputs first sensor information capable of detecting the force generated when the welding wire comes into contact with the welding location. It has, The step of constructing the learning model further includes constructing the learning model using the learning data, which includes operations to move the welding wire identified based on the first sensor information in the vertical direction, where the direction away from the welding location is defined as upward and the direction towards the welding location is defined as downward. Welding method.

7. A welding apparatus that performs welding by controlling at least one of the control objects, a welding wire and an electrode, includes the step of taking a welding image including the tip of the control object and the welding location, A step of generating learning data that includes learning images, which are welding images taken in the past, and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder based on the learning images, The steps include: constructing a learning model using the aforementioned training data, which takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; The steps include: using the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; A step of controlling the controlled object based on the estimation result of the appropriateness of the operation, The first sensor, provided in the wire nozzle that feeds the welding wire, outputs first sensor information capable of detecting the force generated when the welding wire comes into contact with the welding location. A program that executes, The step of constructing the learning model further includes constructing the learning model using the learning data, which includes operations to move the welding wire identified based on the first sensor information in the vertical direction, where the direction away from the welding location is defined as upward and the direction towards the welding location is defined as downward. program.

8. A welding method that performs welding by controlling at least one of the welding wire and the electrode, The steps include taking a welding image that includes the tip of the controlled object and the welding location, A step of generating learning data that includes learning images, which are welding images taken in the past, and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder based on the learning images, The steps include: constructing a learning model using the aforementioned training data, which takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; The steps include: using the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; A step of controlling the controlled object based on the estimation result of the appropriateness of the operation, A step of outputting second sensor information provided in the welding apparatus, which is capable of detecting at least one of the position and orientation of the welding apparatus, It has, The step of constructing the learning model involves using the learning data, which further includes the second sensor information, to construct a learning model that learns the changes in the state of the welding area in the learning image in response to changes in at least one of the position and orientation of the welding apparatus in addition to the operation of the controlled object, and the degree of appropriateness of the operation of the controlled object as judged by the welder. Welding method.

9. A welding apparatus that performs welding by controlling at least one of the control objects, a welding wire and an electrode, includes the step of taking a welding image including the tip of the control object and the welding location, A step of generating learning data that includes learning images, which are welding images taken in the past, and information indicating the degree of appropriateness of the operation of the controlled object as judged by the welder based on the learning images, The steps include: constructing a learning model using the aforementioned training data, which takes the welding image as input and outputs the degree of appropriateness of the operation of the controlled object; The steps include: using the learning model to estimate the degree of appropriateness of the operation of the controlled object in the acquired welding image; A step of controlling the controlled object based on the estimation result of the appropriateness of the operation, The welding apparatus is provided with a second sensor that outputs information from a second sensor capable of detecting at least one of the position and orientation of the welding apparatus. A program that executes, The step of constructing the learning model involves using the learning data, which further includes the second sensor information, to construct a learning model that learns the changes in the state of the welding area in the learning image in response to changes in at least one of the position and orientation of the welding apparatus in addition to the operation of the controlled object, and the degree of appropriateness of the operation of the controlled object as judged by the welder. program.

Images