Welding device, welding method, and program

Abstract

To provide a welding device that can perform automatic welding with a higher welding quality, using a learning model learning a welding skill of a skilled welder.SOLUTION: A welding device, which performs welding by controlling at least one object to be controlled of a welding wire and an electrode, comprises: a camera that photographs an image of welding including tips of the object to be controlled and a welding position; a data generating part that generates data for learning that includes images for learning which are images of welding photographed in the past and information showing a degree of properness of operation of the object to be controlled determined by a welder who looks at the images for learning; a leaning part that constitutes a learning model using the data for learning; an estimating part that estimates a degree of properness of operation of the object to be controlled in the obtained images of welding, using the learning model; and a control part that controls the object to be controlled, on the basis of a result of estimation of the degree of properness of the operation.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a welding apparatus, a welding method, and a program. [Background technology]

[0002] There is an increasing need for automating welding in order to improve welding quality (reduce defects). For example, Patent Document 1 describes an automatic welding system that detects characteristic points such as the electrodes and the edge of the molten pool from an image of the welding point, and controls the weaving width of the electrodes based on the coordinate positions of the characteristic points. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-192524 A Summary of the Invention [Problem to be solved by the invention]

[0004] A skilled welder adjusts the position of the welding torch based on information that is difficult to quantify, such as the shape of the molten pool and the extent to which the molten pool covers the groove boundary of the base material, as well as the positions of the electrodes and the end of the molten pool. However, conventional automatic welding systems cannot use information that is difficult to quantify, other than coordinate information, for automatic control. For this reason, it has been difficult to improve the welding quality to a level close to that of a skilled welder with conventional automatic welding systems.

[0005] The present disclosure has been made in consideration of such problems, and provides a welding device, a welding method, and a program that can perform automatic welding with higher welding quality using a learning model that has learned the welding skills of experienced welders. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a welding device that performs welding by controlling at least one of a welding wire and an electrode includes a camera that captures a welding image including a tip of the controlled object and a welding point, a data generation unit that generates learning data including a learning image that is a welding image captured in the past and information indicating a degree of appropriateness of operation of the controlled object determined by a welder by looking at the learning image, a learning unit that uses the learning data to construct a learning model that takes the welding image as an input and outputs a degree of appropriateness of operation of the controlled object, an estimation unit that uses the learning model to estimate a degree of appropriateness of operation of the controlled object in the acquired welding image, a control unit that controls the controlled object based on an estimation result of the degree of appropriateness of the operation, and a first sensor that is provided in a wire nozzle that feeds the welding wire and outputs first sensor information capable of detecting a contact state of the welding wire with the welding point, and the learning unit constructs the learning model using the learning data that further includes an operation of the welding wire identified based on the first sensor information.

[0007] According to one aspect of the present disclosure, a welding apparatus that performs welding by controlling at least one of a welding wire and an electrode includes a camera that captures a welding image including a tip of the controlled object and a welding point, a data generation unit that generates learning data including a learning image that is a welding image captured in the past and information indicating a degree of appropriateness of operation of the controlled object judged by a welder by looking at the learning image, a learning unit that uses the learning data to construct a learning model that takes the welding image as an input and outputs a degree of appropriateness of operation of the controlled object, an estimation unit that uses the learning model to estimate a degree of appropriateness of operation of the controlled object in the acquired welding image, a control unit that controls the controlled object based on an estimation result of the degree of appropriateness of the operation, and a second sensor provided in the welding apparatus that outputs second sensor information capable of detecting at least one of a position and posture of the welding apparatus, and the learning unit constructs the learning model using the learning data that further includes the second sensor information.

[0008] According to one aspect of the present disclosure, a welding method for controlling at least one of a welding wire and an electrode to perform welding includes the steps of: capturing a welding image including a tip of the controlled object and a welding point; generating learning data including a learning image that is a welding image captured in the past and information indicating a degree of appropriateness of an operation of the controlled object determined by a welder by looking at the learning image; constructing a learning model using the learning data, which inputs the welding image and outputs a degree of appropriateness of the operation of the controlled object; estimating a degree of appropriateness of the operation of the controlled object in the acquired welding image using the learning model; controlling the controlled object based on an estimation result of the degree of appropriateness of the operation; and outputting first sensor information by a first sensor provided in a wire nozzle that feeds the welding wire, capable of detecting a contact state of the welding wire with the welding point, wherein the step of constructing the learning model constructs the learning model using the learning data further including an operation of the welding wire identified based on the first sensor information.

[0009] According to one aspect of the present disclosure, a program causes a welding device that performs welding by controlling at least one of a welding wire and an electrode to capture a welding image including the tip of the control object and a welding point; a step of generating learning data including a learning image that is a welding image captured in the past and information indicating a degree of appropriateness of operation of the control object determined by a welder by looking at the learning image; a step of constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of appropriateness of operation of the control object; a step of estimating a degree of appropriateness of operation of the control object in the acquired welding image using the learning model; a step of controlling the control object based on an estimation result of the degree of appropriateness of the operation; and a step of outputting first sensor information by a first sensor provided in a wire nozzle that feeds the welding wire, the first sensor outputting first sensor information capable of detecting a contact state of the welding wire with the welding point, wherein the step of constructing the learning model constructs the learning model using the learning data that further includes an operation of the welding wire identified based on the first sensor information.

[0010] According to one aspect of the present disclosure, a welding method for controlling at least one of a welding wire and an electrode to perform welding includes the steps of: capturing a welding image including a tip of the controlled object and a welding location; generating learning data including a learning image that is a welding image captured in the past and information indicating a degree of appropriateness of operation of the controlled object determined by a welder by looking at the learning image; constructing a learning model using the learning data, which inputs the welding image and outputs a degree of appropriateness of operation of the controlled object; estimating a degree of appropriateness of operation of the controlled object in the acquired welding image using the learning model; controlling the controlled object based on an estimation result of the degree of appropriateness of the operation; and outputting information of a second sensor provided in a welding device and capable of detecting at least one of a position and an attitude of the welding device, wherein the step of constructing the learning model constructs the learning model using the learning data that further includes the second sensor information.

[0011] According to one aspect of the present disclosure, a program causes a welding device that controls at least one of a welding wire and an electrode to perform welding to capture a welding image including the tip of the control object and a welding point; a step of generating learning data including a learning image that is a welding image captured in the past and information indicating the appropriateness of operation of the control object judged by a welder by looking at the learning image; a step of constructing a learning model using the learning data, which takes the welding image as an input and outputs the appropriateness of the operation of the control object; a step of estimating the appropriateness of the operation of the control object in the acquired welding image using the learning model; a step of controlling the control object based on the estimated result of the appropriateness of the operation; and a step of outputting second sensor information provided in the welding device and capable of detecting at least one of a position and posture of the welding device, wherein the step of constructing the learning model constructs the learning model using the learning data that further includes the second sensor information. Effect of the Invention

[0012] According to the welding device, welding method, and program disclosed herein, automatic welding with higher welding quality can be performed by using a learning model that has learned the welding skills of experienced welders. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic diagram showing an overall configuration of a welding device according to a first embodiment of the present disclosure. [Diagram 2] 2 is a block diagram showing a functional configuration of a control device according to a first embodiment of the present disclosure. FIG. [Diagram 3] 5 is a flowchart showing an example of a process in a learning mode of the welding device according to the first embodiment of the present disclosure. [Figure 4] FIG. 2 is a diagram showing an example of learning data according to the first embodiment of the present disclosure. [Diagram 5] 5 is a flowchart showing an example of processing in a control mode of the welding device according to the first embodiment of the present disclosure. [Figure 6] FIG. 11 is a block diagram showing a functional configuration of a control device according to a second embodiment of the present disclosure. [Figure 7] FIG. 13 is a diagram illustrating an example of learning data according to a third embodiment of the present disclosure. [Figure 8] FIG. 11 is a schematic diagram showing an overall configuration of a welding device according to a fourth embodiment of the present disclosure. [Figure 9] FIG. 13 is a diagram showing an example of learning data according to a fourth embodiment of the present disclosure. [Figure 10] 11 is a schematic diagram showing an overall configuration of a welding device according to a fifth embodiment of the present disclosure. FIG. [Figure 11] FIG. 13 is a diagram showing an example of learning data according to the fifth embodiment of the present disclosure. [Figure 12] FIG. 13 is a diagram for explaining functions of a control device according to a sixth embodiment of the present disclosure. [Figure 13] FIG. 11 is a block diagram showing a functional configuration of a welding device according to another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <First embodiment> Hereinafter, a welding device 1 according to a first embodiment of the present disclosure will be described with reference to FIGS.

[0015] (Overall configuration of welding equipment) FIG. 1 is a schematic diagram showing an overall configuration of a welding device according to a first embodiment of the present disclosure. As shown in FIG. 1, the welding device 1 includes a main body 10 and a control device 20.

[0016] The main body 10 performs arc welding on the base material 3 under the control of the control device 20. The main body 10 includes a welding torch 100, a welding wire supply unit 110, and a camera 120.

[0017] Welding torch 100 is equipped with electrode 101 made of tungsten or the like.

[0018] The welding wire supply unit 110 delivers the welding wire 40 via a wire nozzle 111 toward a welding point of the base material 3 .

[0019] When the welding torch 100 supplies power to the electrode 101, the base material 3 (groove 31) melts due to arc discharge from the electrode 101, and the tip of the welding wire 40 melts as it is supplied to the melting portion of the base material 3, forming a molten pool 41 in which the base material and the welding wire are diluted. The molten pool cools and solidifies, thereby welding the base material 3.

[0020] The camera 120 captures a welding image including the electrode 101, the tip of the welding wire 40, and the welded portion of the base material 3.

[0021] Based on the welding image captured by the camera 120, the control device 20 outputs a control signal for controlling at least one of the control objects, the electrode 101 and the welding wire 40. In this embodiment, as shown in FIG. 1, an example will be described in which the welding device 1 is a so-called TIG welding device that uses an electrode 101 made of tungsten or the like. In addition, an example will be described in which the control device 20 according to this embodiment controls both the electrode 101 and the welding wire 40. In this case, the control of the electrode 101 also includes control of swinging the electrode left and right (weaving width).

[0022] In other embodiments, the welding device 1 may be a MAG welding device, a MIG welding device, or the like. In this case, the welding torch 100 uses the welding wire 40 fed into the welding torch 100 from the welding wire supply unit 110 instead of the electrode 101. In this case, the control device 20 controls only the welding wire 40.

[0023] The control signal is a signal that indicates, for example, the position of the electrode 101, the position of the welding wire 40, and the amount of welding wire 40 that is pulled out (fed out). The welding torch 100 moves the position of the electrode 101 in the left-right direction (±Y direction) in accordance with the control signal from the control device 20. The welding wire supply unit 110 moves the position of the welding wire 40 in the up-down direction (±Z direction) and the left-right direction (±Y direction) in accordance with the control signal from the control device 20. In addition, the welding wire supply unit 110 feeds the welding wire 40 toward the base material 3 in accordance with the control signal from the control device 20.

[0024] (Functional configuration of the control device) 2 is a block diagram showing a functional configuration of a control device according to the first embodiment of the present disclosure. As shown in FIG. 2, the control device 20 includes a processor 21, a memory 22, a storage 23, and a communication interface 24.

[0025] The processor 21 operates according to a predetermined program to thereby fulfill the functions of an acquisition unit 210, an estimation unit 211, a control unit 212, a data generation unit 213, and a learning unit 214.

[0026] The acquisition unit 210 acquires a welding image captured by the camera 120 of the main body unit 10 .

[0027] The estimation unit 211 receives a welding image as an input, and estimates the appropriateness of the operation of the welding wire 40 and the electrode 101 in the acquired welding image by using a learning model M that outputs the appropriateness of the operation of the welding wire 40 and the electrode 101. Note that the estimation unit 211 reads out from the storage 23 the learning model M that has been learned in advance by a learning unit 214 described later, and uses it.

[0028] The estimation unit 211 estimates the appropriateness of the operation for the movement of the electrode 101 in each of the left-right and up-down directions, for example, in three stages: "excessive," "appropriate," and "insufficient." The estimation unit 211 also estimates the operation for the movement of the welding wire 40 in each of the left-right and up-down directions, respectively, in three stages: "excessive," "appropriate," and "insufficient." The estimation unit 211 may further represent the operation regarding the withdrawal amount of the welding wire 40 in three stages: "excessive," "appropriate," and "insufficient." In other embodiments, the evaluation of the operation may be divided into more than three stages.

[0029] The control unit 212 controls the welding wire 40 and the electrode 101 based on the estimation result of the estimation unit 211. For example, the control unit 212 generates a control signal to move the electrode 101 to the right when the operation of the electrode 101 to the left is estimated to be "excessive" and outputs the control signal to the welding torch 100 when the operation of the electrode 101 to the left is estimated to be "insufficient". Then, the welding torch 100 moves the electrode 101 to the left or right in accordance with the control signal. Also, when the operation of the electrode 101 to the left is evaluated to be "appropriate", the control unit 212 does not control the movement of the electrode 101 to the left or right (does not output a control signal, or outputs a control signal that maintains the position in the left or right direction). Then, the welding torch 100 does not move the position of the electrode 101 to the left or right direction until the next control signal is received.

[0030] The data generation unit 213 generates learning data including a learning image, which is a welding image captured in the past, and information indicating the degree of appropriateness of the operation of the welding wire 40 and the electrode 101 performed by the welder when the learning image was captured.

[0031] The learning unit 214 constructs a learning model M by performing deep learning using the learning data.

[0032] The memory 22 has a memory area necessary for the operation of the processor 21 .

[0033] The storage 23 is a so-called auxiliary storage device, and is, for example, a hard disk drive (HDD), a solid state drive (SSD), etc. The storage 23 stores welding images captured by the camera 120, a learned learning model M, and the like.

[0034] The communication interface 24 is an interface for transmitting and receiving various types of information (signals) to and from an external device (such as the main body unit 10).

[0035] The predetermined program executed by the processor 21 of the control device 20 is stored in a computer-readable recording medium. The computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. The computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program. The program may be for realizing part of the above-mentioned functions. The program may also be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0036] The control device 20 according to this embodiment has two modes: a learning mode and a control mode. In the learning mode, the control device 20 constructs a learning model M by the learning unit 214. In the control mode, the control device 20 controls the main body 10 of the welding device 1 (automatic execution of welding) using the learning model M that has been learned by the learning unit 214. Details of the processing in each mode will be described below.

[0037] (Learning mode process flow) FIG. 3 is a flowchart showing an example of a process in a learning mode of the welding device according to the first embodiment of the present disclosure. Here, the flow of processing by the welding device 1 in the learning mode will be described with reference to FIG.

[0038] First, the learning unit 214 of the control device 20 acquires welding images and operation information to be used for learning (step S100). For example, the learning unit 214 acquires a plurality of welding images, which are images of welding previously performed by a welder (an experienced welder), and the contents of operations (operation information) performed by the welder on the electrode 101 and the welding wire 40 when each welding image was captured, in association with each other.

[0039] Moreover, the data generating unit 213 generates learning data D1 and evaluation data D2 based on the acquired welding image and operation information (step S101).

[0040] FIG. 4 is a diagram illustrating an example of learning data according to the first embodiment of the present disclosure. For example, as shown in FIG. 4, the data generating unit 213 generates learning data D1 by adding a label (correct answer data) indicating the degree of appropriateness of the operation of the electrode 101 and the welding wire 40 to each welding image. For example, one of "excessive", "appropriate", and "inadequate" is input for each of the electrode 101 and the welding wire 40. The data generating unit 213 similarly labels all welding images to indicate the direction of the operation as "excessive", "appropriate", or "inadequate". Note that the welder may view each welding image and perform an operation to specify the label. The data generating unit 213 generates learning data D1 by adding a label specified by the welder to each welding image. The data generating unit 213 also divides a part of the labeled learning data D1 into evaluation data D2.

[0041] The data generating unit 213 may determine the appropriateness of the operation of the electrode 101 and the welding wire 40 in each welding image based on the time series of the operation information, and automatically add a label. For example, it is assumed that the operation information associated with the welding image shown in FIG. 4 and the operation information at the previous time point detect that the welder has performed an operation to stop the movement of the electrode 101 to the right and move it to the left, and an operation to stop the welding wire 40 (no operation). In this case, the data generating unit 213 adds a label to this welding image indicating that the operation of the electrode 101 in the left-right direction (right direction) is "excessive" and that the operation of the welding wire 40 in the left-right direction is "appropriate." Also, for example, when an operation of continuously moving the electrode 101 to the right is detected from the time series of the operation information, the data generating unit 213 adds a label indicating that the operation of the electrode 101 in the left-right direction (right direction) is "insufficient."

[0042] Next, the learning unit 214 constructs a learning model M by performing deep learning using learning data D1 including a welding image (learning image) and a label (correct answer data) (step S102).

[0043] For example, as shown in FIG. 4, a welding image of a welding point includes various elements such as the base material 3, the groove 31, the electrode 101, the welding wire 40, the molten pool 41, the bead 42, and the arc A. As described above, a skilled welder adjusts the operation amount of the electrode 101 and the welding wire 40 by taking into account not only position information such as the end position of the molten pool 41 and the tip positions of the electrode 101 and the welding wire 40, but also information that is difficult to quantify, such as the degree to which the molten pool 41 touches the boundary of the groove 31 (wettability of the groove) and the degree to which the molten pool 41 touches the welded portion (on the bead 42). In the welding device 1 according to this embodiment, the learning unit 214 learns the relationship between the feature amount of the welding image and the operation performed by the welder by looking at the welding image, and thereby it becomes possible to incorporate information that is difficult to quantify into the learning model M. As a result, the learning unit 214 can construct a learning model M that can make an estimation closer to the judgment of a skilled welder based on the welding image.

[0044] Next, the learning unit 214 evaluates the learning model M constructed in step S102 with the learning data D1 (step S103). The same data set as in step S102 is used for this learning data D1. Specifically, the learning unit 214 inputs only the welding images of the learning data D1 to the learning model M, and causes it to output the appropriateness of the operation amount for each welding image. The learning unit 214 performs a performance evaluation of the learning data D1 based on the output results of each welding image of the learning model M and the correct answer data.

[0045] Furthermore, the learning unit 214 evaluates the learning model M constructed in step S102 with the evaluation data D2 (step S104). This evaluation data D2 is a data set that is not used for learning the learning model M. The learning unit 214 inputs only the welding images of the evaluation data D2 to the learning model M, and causes the learning model M to output the appropriateness of the operation amount for each welding image. The learning unit 214 performs performance evaluation of the evaluation data D2 based on the output results of each welding image of the learning model M and the correct answer data. The learning unit 214 calculates, for example, an accuracy rate, a loss function, etc. as performance evaluation results for each of the learning data D1 and the evaluation data D2.

[0046] Furthermore, the learning unit 214 associates the learning model M with the performance evaluation result of the learning data D1 and the performance evaluation result of the evaluation data D2, and stores them in the storage 23 (step S105).

[0047] Next, the learning unit 214 judges whether the termination condition is satisfied (step S106). For example, the learning unit 214 judges that the termination condition is satisfied when the performance evaluation result of the learning data D1 and the performance evaluation result of the evaluation data D2 both converge to a predetermined value or more (step S106; YES). In this case, the learning unit 214 adopts the learning model M with the highest performance evaluation result among the multiple constructed learning models M as the model to be used in the control mode (step S107).

[0048] On the other hand, if the termination condition is not satisfied (step S106; NO), the learning unit 214 returns to step S102 and constructs a new learning model M.

[0049] (Control mode process flow) FIG. 5 is a flowchart showing an example of processing in the control mode of the welding device according to the first embodiment of the present disclosure. Here, the flow of processing by the welding device 1 in the control mode will be described with reference to FIG.

[0050] When the welding device 1 automatically performs welding of the base material 3, the camera 120 captures an image of the welding point of the base material 3. The acquisition unit 210 of the control device 20 acquires the welding image captured by the camera 120 (step S110).

[0051] Next, the estimation unit 211 estimates the appropriateness of the current operation of the electrode 101 and the welding wire 40 (step S111). Specifically, the estimation unit 211 inputs the acquired welding image to the learning model M, and obtains an output estimation result (information indicating any one of “excessive”, “appropriate”, or “insufficient”).

[0052] Next, control unit 212 generates control signals for controlling electrode 101 and welding wire 40 based on the estimation result of estimation unit 211, and outputs the control signals to welding torch 100 and welding wire supply unit 110 (step S112). For example, as in the example of FIG. 4, when the operation of electrode 101 in the right direction is “excessive”, control unit 212 generates a control signal for moving electrode 101 in the left direction, and outputs the control signal to welding torch 100. Also, as in the example of FIG. 4, when the operation of welding wire 40 in the downward direction is “insufficient”, control unit 212 generates a control signal for moving electrode 101 in the downward direction, and outputs the control signal to welding torch 100. Furthermore, as in the example of FIG. 4, when the downward movement of the electrode 101 and the rightward movement of the welding wire 40 are “proper”, the control unit 212 determines that the up-down movement of the electrode 101 and the left-right movement of the welding wire 40 are unnecessary, and does not need to output control signals regarding these movements to the welding wire supply unit 110.

[0053] The welding device 1 repeats the series of processes shown in FIG. 5 from moment to moment to automatically perform welding.

[0054] (Action, effect) As described above, the welding apparatus 1 according to this embodiment includes a camera 120 that captures a welding image including the electrode 101 and the tip of the welding wire 40 and the welded portion of the base material 3, a learning unit 214 that constructs a learning model M using learning data including learning images that are welding images captured in the past and the operation of the electrode 101 and the welding wire 40 determined by the welder by looking at the learning images, an estimation unit 211 that uses the learning model M to estimate the appropriateness of the operation in the acquired welding image, and a control unit 212 that controls the electrode 101 and the welding wire 40 based on the estimation result of the estimation unit 211.

[0055] For example, even if the positional relationship of the electrode, welding wire, molten pool, etc. is the same, the welder may make different judgments as to whether the operation of the electrode and welding wire is appropriate, excessive, or insufficient, based on a combination of various conditions of the welding point obtained from the welding image (e.g., wettability of the groove, etc.). For this reason, if the electrode and welding wire are controlled based only on the position coordinates of each part as in the conventional technology, control different from the judgment of the welder may be performed, and the welding quality may be lower than if the welder were to weld manually. In contrast, as described above, the welding device 1 according to the present embodiment learns the relationship between the feature amount of the learning image and the operation performed by the welder while looking at the state of the welding point in the learning image, and it becomes possible to incorporate not only the positional information of each part but also information that the welder uses to judge welding and that is difficult to quantify into the learning model M. As a result, the welding device 1 in the control mode can estimate the appropriateness of the operation (e.g., whether the operation is "excessive", "appropriate", or "insufficient") at a level close to the judgment of the welder, using the learning model M. Furthermore, the welding device 1 controls the electrode 101 and the welding wire 40 based on the estimation result to perform automatic welding, thereby achieving welding quality close to that achieved by a manual welder. In other words, the welding device 1 according to this embodiment can perform automatic welding with higher welding quality than the conventional technology in which control is performed based only on position information of each part in a welding image.

[0056] <Second embodiment> Next, a welding device 1 according to a second embodiment of the present disclosure will be described with reference to FIG. Components common to the first embodiment are given the same reference numerals and detailed description thereof will be omitted.

[0057] FIG. 6 is a diagram illustrating an example of learning data according to the second embodiment of the present disclosure. As shown in FIG. 6, the data generation unit 213 according to this embodiment generates learning data D1 including a learning image in which areas that the welder does not focus on in a previously captured welding image are masked in step S101 of FIG. 3.

[0058] The welder judges whether the amount of manipulation of the electrode 101 and the welding wire 40 is appropriate by observing the fit of the molten pool 41 to the side surface of the groove 31 (change in the shape of the groove 31) and the way the molten pool 41 covers the bead 42. For example, as shown in FIG. 7, the welder judges the amount of manipulation by referring to a focus area R1 in which these features appear in the welding image. However, deep learning may learn easy-to-understand features contained in the welding image (for example, areas with large changes in brightness such as around the arc A), and may not be able to fully learn the features of the area that the welder is actually focusing on (focus area R1).

[0059] For this reason, the data generating unit 213 according to the present embodiment generates learning data D1 in which a non-interest region R2 of a welding image that is not focused on by the welder is arbitrarily masked as shown in Fig. 6. For example, the data generating unit 213 sets an area designated by the welder as the non-interest region R2 for each welding image. There may be one or more non-interest regions R2. For example, an area including the electrode 101 or the arc A with a large brightness change, an area including a groove shoulder (a portion not welded), etc. are set as the non-interest region R2.

[0060] Moreover, in step S102 of FIG. 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 including the masked welding image (learning image).

[0061] In this way, even when the welding device 1 can learn only from welding images under specific conditions (the number of samples in the learning data D1 is small), the welding device 1 can effectively perform deep learning so that the learning model M can acquire common features in the region of interest R1. This also enables the welding device 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate or not, based on the features included in the region of interest R1 in the welding image, during the control mode.

[0062] <Third embodiment> Next, a welding device 1 according to a third embodiment of the present disclosure will be described with reference to FIG. Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

[0063] FIG. 7 is a diagram illustrating an example of learning data according to the third embodiment of the present disclosure. As shown in FIG. 7, the data generating unit 213 according to this embodiment generates learning data D1 including a learning image obtained by extracting the contour of a welding image captured in the past in step S101 of FIG.

[0064] As described in the second embodiment, in deep learning, there is a possibility that easily recognizable features contained in the welding image (for example, areas with large changes in luminance such as around the arc A) are learned, and the features of the area that the welder actually focuses on may not be learned sufficiently. For this reason, in order to prevent learning of areas that the welder does not focus on, the data generation unit 213 according to this embodiment generates learning data D1 by omitting information unnecessary for determining the operation from the welding image.

[0065] Specifically, the data generating unit 213 performs a known edge detection process on the original welding image to detect and extract only the contours of each part included in the welding image to generate an edge image. The data generating unit 213 may further perform a process of removing information unnecessary for determining the operation from the edge image. In the example of Fig. 7, the data generating unit 213 performs a process of removing the mirror image of the electrode reflected on the side wall of the groove.

[0066] Moreover, in step S102 of FIG. 3, the learning unit 214 according to this embodiment performs learning of the learning model M based on the learning data D1 including a welding image (learning image) that is an edge image.

[0067] In this way, the welding apparatus 1 can efficiently learn the learning model M by using an image that includes only the edges of each part, which is the minimum necessary information, for the input image, while omitting areas that the welder does not pay attention to. This enables the welding apparatus 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate in the control mode. As a result, the welding apparatus 1 can appropriately control the electrode 101 and the welding wire 40 based on the appropriateness of the operation.

[0068] <Fourth embodiment> Next, a welding device 1 according to a fourth embodiment of the present disclosure will be described with reference to FIGS. Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

[0069] FIG. 8 is a schematic diagram showing an overall configuration of a welding device according to a fourth embodiment of the present disclosure. 8, the welding device 1 according to this embodiment further includes a torque sensor 112 (first sensor) provided on the wire nozzle 111 of the welding wire supply unit 110. The torque sensor 112 detects a force generated in the wire nozzle 111 when the tip of the welding wire 40 comes into contact with the base material 3.

[0070] The acquisition unit 210 according to this embodiment further acquires, from the torque sensor 112 in step S100 of FIG. 3, first sensor information capable of detecting the contact state of the welding wire 40 with the base material 3.

[0071] FIG. 9 is a diagram illustrating an example of learning data according to the fourth embodiment of the present disclosure. In step S101 of FIG. 3, the data generating unit 213 detects the operation of the welding wire 40 based on the first sensor information, and specifies the appropriateness of the operation in the up-down direction (±Z direction) of the welding wire 40. The data generating unit 213 also generates learning data D1 (FIG. 9) by adding information (label) indicating the appropriateness of the operation in the up-down direction (±Z direction) of the welding wire 40 to the welding image acquired at the same time as the sensor information of the torque sensor 112. For example, when it is detected from the time series of the first sensor information of the torque sensor 112 that the welder has performed an operation of continuously moving the welding wire 40 downward, the data generating unit 213 adds a label indicating that the operation in the up-down direction (downward direction) of the welding wire 40 is "too small" to the welding image corresponding to this sensor information. Note that in another embodiment, the welder may specify a label by looking at a combination of the welding image and the first sensor information. In this case, the data generating unit 213 generates learning data D1 by adding the label specified by the welder to the welding image.

[0072] The learning unit 214 learns the learning model M based on the learning data D1 generated by the data generating unit 213 in step S102 of FIG.

[0073] As in the example of Fig. 8, when the camera 120 is placed diagonally above and in front of the welding point (in the +Z direction and the +X direction), the difference in the position of the welding wire 40 in the welding image is slight, especially when the welding wire 40 has low rigidity and is soft (easily melted). In this case, it may be difficult to identify the position of the welding wire 40 in the up-down direction (±Z direction) from the welding image. In response to this, the welding device 1 according to the present embodiment makes it possible to construct a learning model M that learns the appropriateness of the operation of the welding wire 40 in the up-down direction from the contact state of the welding wire 40 detected by the torque sensor 112.

[0074] In addition, in the control mode, the acquisition unit 210 acquires the first sensor information of the torque sensor 112 together with the welding image in step S110 of FIG. 5. In step S111 of FIG. 5, the estimation unit 211 inputs the welding image and the first sensor information to the learning model M and obtains the degree of appropriateness of the operation of the electrode 101 and the welding wire 40 at the current time as an output. In addition, in step S112 of FIG. 5, the control unit 212 generates and outputs a control signal for controlling the electrode 101 and the welding wire 40 based on the degree of appropriateness of the operation of the electrode 101 and the welding wire 40. This allows the welding apparatus 1 to accurately estimate whether the operation of the welding wire 40 in the up-down direction is also appropriate in the control mode. Therefore, the welding apparatus 1 can appropriately control the up-down direction of the welding wire 40 as well, thereby further improving the welding quality.

[0075] In the fourth embodiment, an example has been described in which the data generating unit 213 generates the learning data D1 to which information (label) on the appropriateness of the operation of the welding wire 40 in the up-down direction is added based on the sensor information of the torque sensor 112, but the present invention is not limited to this. In other embodiments, the data generating unit 213 may directly include the first sensor information of the torque sensor 112 in the learning data D1. The learning unit 214 learns the appropriateness of the operation of the electrode 101 in the left-right direction and the appropriateness of the operation of the welding wire 40 in the left-right direction and the up-down direction based on the welding image, the label, and the first sensor information of the torque sensor 112. Even in this embodiment, the welding device 1 can accurately estimate whether the operation of the welding wire 40 in the up-down direction is appropriate.

[0076] <Fifth embodiment> Next, a welding device 1 according to a fifth embodiment of the present disclosure will be described with reference to FIGS. Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

[0077] FIG. 10 is a schematic diagram showing the overall configuration of a welding device according to a fifth embodiment of the present disclosure. As shown in FIG. 10, the welding device 1 according to this embodiment further includes a sensor 102 (second sensor) that is provided on the welding torch 100 and detects at least one of the position and the posture of the welding torch 100. The sensor 102 is, for example, an acceleration sensor that can detect the posture (angle) of the welding torch 100. The sensor 102 is also an encoder that can detect the position and the posture of the welding torch 100. The sensor 102 may also be a combination of an acceleration sensor and an encoder. The position and posture of the welding torch 100 are expressed, for example, as a relative position and a relative angle with respect to a reference position (for example, an initial position and an initial angle when the welding device 1 is installed).

[0078] The acquisition unit 210 according to this embodiment further acquires, from the sensor 102 in step S100 of FIG. 3, second sensor information capable of detecting the position or posture of the welding wire 40.

[0079] FIG. 11 is a diagram illustrating an example of learning data according to the fifth embodiment of the present disclosure. The data generating unit 213 generates learning data D1 (FIG. 11) including the second sensor information of the sensor 102 in step S101 of FIG.

[0080] The learning unit 214 learns the learning model M based on the learning data D1 generated by the data generation unit 213 in step S102 of FIG.

[0081] When the position or posture of welding torch 100 changes during welding, the state of the welded portion (e.g., the flow of molten pool 41) changes. For this reason, in cases where welding is performed while changing the position and posture of welding torch 100, if learning model M is learned based only on welding images, the estimation accuracy of the appropriateness of the operation may decrease. In contrast, the welding device 1 according to this embodiment can learn learning model M that takes into account the position and posture of welding torch 100 by using learning data D1 including second sensor information capable of detecting the position and posture of welding torch 100.

[0082] 5, the estimation unit 211 inputs the welding image and the second sensor information to the learning model M, and obtains, as an output, the degree of suitability of the operation of the electrode 101 and the welding wire 40 according to the position and posture of the welding device 1 at the time when the welding image was captured. Furthermore, the control unit 212 generates and outputs a control signal for controlling the electrode 101 and the welding wire 40 based on the degree of suitability of the operation of the electrode 101 and the welding wire 40, in step S112 of FIG.

[0083] With this configuration, welding device 1 can accurately estimate the appropriateness of the operation of electrode 101 and welding wire 40 even when welding is performed while changing the position and posture of welding torch 100. As a result, welding device 1 can appropriately control electrode 101 and welding wire 40 by following the changes in the position and posture of welding torch 100, thereby improving welding quality.

[0084] Sixth embodiment Next, a welding device 1 according to a sixth embodiment of the present disclosure will be described with reference to FIG. Components common to the above-described embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

[0085] If the welding image changes due to variations in the settings of the welding equipment 1 (such as the arrangement of the welding torch 100, welding wire supply unit 110, and camera 120) or variations in the settings of the base material 3, it may not be possible to effectively learn the learning model M. For this reason, in step S101 of FIG. 3, the data generation unit 213 according to this embodiment generates learning data D1 including learning images obtained by performing image processing on previously captured welding images that simulates variations in the relative positions of the welding equipment 1 and the base material 3.

[0086] Fig. 12 is a diagram for explaining the function of the control device according to the sixth embodiment of the present disclosure. Specifically, as shown in Fig. 12, the data generating unit 213 generates a plurality of processed images by performing various image processing on the original welding image, and generates learning data D1 including each processed image.

[0087] For example, the data generating unit 213 generates a processed image by translating or rotating the welding image to generate learning data D1 simulating the variation in the setting of the welding device 1 and the base material. The data generating unit 213 generates a processed image by enlarging or reducing the welding image to generate learning data D1 simulating the variation in the angle of view due to focusing. The data generating unit 213 generates a processed image by changing the brightness of the welding image to generate learning data D1 simulating the variation in brightness. The data generating unit 213 generates a processed image by adding Gaussian noise to the welding image to generate learning data D1 simulating the case where the welding image is photographed in a defocused state. The data generating unit 213 generates a processed image by changing the aspect ratio of the welding image to generate learning data D1 simulating the variation in the distance between the electrode 101 and the welding wire 40.

[0088] Further, in step S102 of FIG. 3, the learning unit 214 according to this embodiment learns the learning model M based on the learning data D1 including the original welding image and the learning data D1 including the processed image (learning image).

[0089] In this way, the welding device 1 can construct a learning model M that takes into account the variations in settings, etc. This allows the welding device 1 to accurately control the electrode 101 and the welding wire 40 in the control mode without being affected by the variations in settings, etc.

[0090] <Other embodiments> FIG. 13 is a block diagram showing a functional configuration of a welding device according to another embodiment of the present disclosure. In the above-described embodiments, the control device 20 has the data generating unit 213 that generates the learning data D1 and the evaluation data D2, but the present invention is not limited to this. For example, as shown in FIG. 13, the welding device 1 may include a data generating server 50 having the data generating unit 213 in addition to the control device 20. In this case, the learning data D1 and the evaluation data D2 are generated in the data generating unit 213 of the data generating server 50. The control device 20 also acquires the learning data D1 and the evaluation data D2 from the data generating server 50 via the communication interface 24 and stores them in the storage 23. Even with this configuration, the welding device 1 can obtain the same effects as those of the above-described embodiments.

[0091] As described above, several embodiments according to the present disclosure have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

[0092] <Additional Notes> The welding apparatus, the welding method, and the program described in the above-described embodiments can be understood, for example, as follows.

[0093] (1) According to a first aspect of the present disclosure, a welding apparatus 1 that performs welding by controlling at least one of the control objects, the welding wire 40 and the electrode 101, includes a camera 120 that captures a welding image including the tip of the control object and the welding point, a data generation unit that generates learning data D1 including a learning image that is a welding image captured in the past and information indicating the appropriateness of the operation of the control object determined by a welder by looking at the learning image, a learning unit 214 that uses the learning data D1 to construct a learning model M that takes the welding image as input and outputs the appropriateness of the operation of the control object, an estimation unit 211 that uses the learning model M to estimate the appropriateness of the operation of the control object in the acquired welding image, and a control unit 212 that controls the control object based on the estimation result of the appropriateness of the operation.

[0094] In this way, the welding device 1 learns the relationship between the feature amounts of the learning image and the operations that the welder performs while viewing the state of the welding point in the image when the learning image is captured, and is able to incorporate not only the positional information of each part in the welding image, but also information that the welder uses to make welding judgments and that is difficult to quantify, into the learning model M. In this way, the welding device 1 can perform automatic welding with high welding quality in the control mode by estimating the appropriateness of the operation using the learning model M that has learned the judgments of experienced welders.

[0095] (2) According to a second aspect of the present disclosure, in the welding apparatus 1 of the first aspect, the data generation unit 213 generates learning data D1 including a learning image in which areas of a previously captured welding image that are not of interest to the welder are masked.

[0096] In this way, even when the welding device 1 can learn only from welding images under specific conditions (the number of samples in the learning data D1 is small), the welding device 1 can effectively perform deep learning so that the learning model M can acquire common features in the region of interest R1. This also enables the welding device 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate or not, based on the features included in the region of interest R1 in the welding image, during the control mode.

[0097] (3) According to a third aspect of the present disclosure, in the welding device 1 according to the first aspect, the data generator 213 generates learning data D1 including a learning image in which the contour of a welding image captured in the past is extracted.

[0098] In this way, the welding apparatus 1 can efficiently learn the learning model M by using an image that includes only the edges of each part, which is the minimum necessary information, for the input image, while omitting areas that the welder does not pay attention to. This enables the welding apparatus 1 to accurately estimate whether the operation of the electrode 101 and the welding wire 40 is appropriate in the control mode. As a result, the welding apparatus 1 can appropriately control the electrode 101 and the welding wire 40 based on the appropriateness of the operation.

[0099] (4) According to a fourth aspect of the present disclosure, in the welding apparatus 1 relating to any one of the first to third aspects, the data generation unit 213 generates learning data D1 including learning images obtained by applying image processing to previously captured welding images to simulate variations in the relative positions of the control object, the camera 120, and the welding point.

[0100] In this way, the welding device 1 can construct a learning model M that takes into account the variations in settings, etc. This allows the welding device 1 to accurately control the electrode 101 and the welding wire 40 in the control mode without being affected by the variations in settings, etc.

[0101] (5) According to a fifth aspect of the present disclosure, the welding device 1 according to any one of the first to fourth aspects further includes a first sensor 112 provided on the wire nozzle 111 that sends out the welding wire 40 and that outputs first sensor information capable of detecting the contact state of the welding wire 40 with the welding point, and the learning unit 214 constructs a learning model M using learning data D1 that further includes an operation of the welding wire 40 identified based on the first sensor information.

[0102] (6) According to a sixth aspect of the present disclosure, the welding apparatus 1 of any one of the first to fifth aspects further includes a second sensor 102 provided on the welding apparatus 1 and outputting second sensor information capable of detecting at least one of the position and posture of the welding apparatus 1, and the learning unit 214 constructs a learning model M using learning data D1 further including the second sensor information.

[0103] In this way, welding device 1 can accurately estimate the appropriateness of the operation of electrode 101 and welding wire 40 even when welding is performed while changing the position and posture of welding torch 100. As a result, welding device 1 can appropriately control electrode 101 and welding wire 40 by following the changes in the position and posture of welding torch 100.

[0104] (7) According to a seventh aspect of the present disclosure, a welding method for controlling at least one of the control objects, the welding wire 40 and the electrode 101, to perform welding includes the steps of: capturing a welding image including the tip of the control object and the welding point; generating learning data D1 including a learning image that is a welding image captured in the past and information indicating the appropriateness of the operation of the control object determined by a welder by looking at the learning image; using the learning data D1 to construct a learning model M that takes the welding image as input and outputs the appropriateness of the operation of the control object; using the learning model M to estimate the appropriateness of the operation of the control object in the acquired welding image; and controlling the control object based on the estimated result of the appropriateness of the operation.

[0105] (8) According to an eighth aspect of the present disclosure, the program causes the welding device 1, which performs welding by controlling at least one of the control objects, the welding wire 40 and the electrode 101, to execute the following steps: capturing a welding image including the tip of the control object and the welding point; generating learning data D1 including a learning image that is a welding image captured in the past and information indicating the appropriateness of the operation of the control object determined by a welder by looking at the learning image; using the learning data D1, constructing a learning model M that takes the welding image as input and outputs the appropriateness of the operation of the control object; using the learning model M, estimating the appropriateness of the operation of the control object in the acquired welding image; and controlling the control object based on the estimated result of the appropriateness of the operation. [Explanation of symbols]

[0106] 1 Welding equipment 10 Main body 100 Welding Torch 101 Electrode 102 Sensor (Second Sensor) 110 Welding wire supply unit 111 Wire nozzle 112 Torque sensor (first sensor) 120 Camera 20 Control device 21 Processors 210 Acquisition Department 211 Estimation Department 212 Control section 213 Data Generation Department 214 Learning Department 22 Memory 23. Storage 24 Communication Interface 3 Base material 40 Welding Wire 50 Data Generation Server

Claims

1. A welding device that performs welding by controlling at least one of a welding wire and an electrode, A camera that captures a welding image including the tip of the control object and a welding point; A data generating unit that generates learning data including learning images that are welding images captured in the past and information indicating the appropriateness of the operation of the control target judged by a welder by looking at the learning images; a learning unit that uses the learning data to construct a learning model that receives the welding image as an input and outputs a degree of suitability of the operation of the control object; an estimation unit that estimates a degree of appropriateness of an operation of the control target in the acquired welding image by using the learning model; a control unit that controls the control target based on an estimation result of the appropriateness of the operation; a first sensor provided in a wire nozzle that feeds the welding wire and that outputs first sensor information capable of detecting a contact state of the welding wire with the welding point; Equipped with the learning unit constructs the learning model using the learning data further including an operation of the welding wire identified based on the first sensor information. Welding equipment.

2. A welding device that performs welding by controlling at least one of a welding wire and an electrode, A camera that captures a welding image including the tip of the control object and a welding point; A data generating unit that generates learning data including learning images that are welding images captured in the past and information indicating the appropriateness of the operation of the control target judged by a welder by looking at the learning images; a learning unit that uses the learning data to construct a learning model that receives the welding image as an input and outputs a degree of suitability of the operation of the control object; an estimation unit that estimates a degree of appropriateness of an operation of the control target in the acquired welding image by using the learning model; a control unit that controls the control target based on an estimation result of the appropriateness of the operation; a second sensor provided in the welding device and configured to output second sensor information capable of detecting at least one of a position and a posture of the welding device; Equipped with The learning unit constructs the learning model using the learning data further including the second sensor information. Welding equipment.

3. The data generation unit generates learning data including the learning image in which an area that is not of interest to the welder in the previously captured welding image is masked.

3. The welding device according to claim 1 or 2.

4. The data generation unit generates learning data including the learning image obtained by extracting a contour of the welding image captured in the past.

3. The welding device according to claim 1 or 2.

5. the data generation unit generates learning data including the learning image obtained by performing image processing on the previously captured welding image, the image processing simulating a variation in the relative positions of the control object, the camera, and the welding point; 3. The welding device according to claim 1 or 2.

6. A welding method for performing welding by controlling at least one of a welding wire and an electrode, Taking a welding image including the tip of the control object and a welding point; A step of generating learning data including learning images which are welding images captured in the past and information indicating the appropriateness of the operation of the controlled object judged by a welder by looking at the learning images; constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of suitability of the operation of the control object; estimating a degree of appropriateness of the operation of the control target in the acquired welding image by using the learning model; controlling the control target based on a result of the estimation of the appropriateness of the operation; a step of outputting first sensor information capable of detecting a contact state of the welding wire with the welding point by a first sensor provided in a wire nozzle that delivers the welding wire; having and constructing the learning model by using the learning data further including an operation of the welding wire identified based on the first sensor information. Welding method.

7. A welding device for controlling at least one of a welding wire and an electrode to perform welding, Taking a welding image including the tip of the control object and a welding point; A step of generating learning data including learning images which are welding images captured in the past and information indicating the appropriateness of the operation of the controlled object judged by a welder by looking at the learning images; constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of suitability of the operation of the control object; estimating a degree of appropriateness of the operation of the control target in the acquired welding image by using the learning model; controlling the control target based on a result of the estimation of the appropriateness of the operation; a step of outputting first sensor information capable of detecting a contact state of the welding wire with the welding point by a first sensor provided in a wire nozzle that delivers the welding wire; A program for executing and constructing the learning model by using the learning data further including an operation of the welding wire identified based on the first sensor information. program.

8. A welding method for performing welding by controlling at least one of a welding wire and an electrode, Taking a welding image including the tip of the control object and a welding point; A step of generating learning data including learning images which are welding images captured in the past and information indicating the appropriateness of the operation of the controlled object judged by a welder by looking at the learning images; constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of suitability of the operation of the control object; estimating a degree of appropriateness of the operation of the control target in the acquired welding image by using the learning model; controlling the control target based on a result of the estimation of the appropriateness of the operation; outputting second sensor information provided in the welding device and capable of detecting at least one of a position and a posture of the welding device; having The step of constructing the learning model constructs the learning model using the learning data further including the second sensor information. Welding method.

9. A welding device for controlling at least one of a welding wire and an electrode to perform welding, Taking a welding image including the tip of the control object and a welding point; A step of generating learning data including learning images which are welding images captured in the past and information indicating the appropriateness of the operation of the controlled object judged by a welder by looking at the learning images; constructing a learning model using the learning data, the learning model inputting the welding image and outputting a degree of suitability of the operation of the control object; estimating a degree of appropriateness of the operation of the control target in the acquired welding image by using the learning model; controlling the control target based on a result of the estimation of the appropriateness of the operation; outputting information of a second sensor provided in the welding device and capable of detecting at least one of a position and a posture of the welding device; A program for executing The step of constructing the learning model constructs the learning model using the learning data further including the second sensor information. program.

Images