Method for measuring behavior of welding phenomenon, measuring device, welding system, welding method, method for measuring behavior of deposition phenomenon, and storage medium

Abstract

A method of measuring a behavior of a welding phenomenon includes: an image processing step of performing image processing corresponding to a behavior of a welding phenomenon on a welding image captured by a vision sensor; an image segmentation step of generating a plurality of segmented images for each of constituent elements corresponding to the welding phenomenon using a processed image generated in the image processing step; and an extraction step of extracting the behavior of the welding phenomenon using at least two of the plurality of segmented images.

Description

Technical Field

[0001] This invention relates to a method, apparatus, system, and procedure for measuring the behavior of welding phenomena. Background Technology

[0002] Gas shielded arc welding (hereinafter also known as GMAW) produces various welding phenomena (hereinafter referred to as welding behaviors) that occur during welding. Among these welding behaviors, there are those that damage the weld quality, such as spatter and fumes. In the past, methods were sought to measure these welding behaviors for the purpose of real-time assessment and tracking of weld quality.

[0003] As a conventional measurement method, Patent Document 1 discloses a sputtering identification method and device that aims to accurately measure the amount and behavior of sputtering using an image processing mechanism. Specifically, in arc welding, multiple frames of images are captured of the arc generation location and its surroundings. Each frame is binarized or multi-valued to extract one or more high-brightness independent regions, and the positional information of each extracted independent region is detected. Furthermore, based on the detected positional information of each independent region, the continuity of independent regions between consecutive frames is determined, and a series of independent regions deemed to be continuous is identified as a sputtering event accompanying arc welding.

[0004] Furthermore, Patent Document 2 discloses a sputtering counting method performed by a movable terminal device equipped with a camera, aiming to count the number of sputtering particles in a cost-effective and convenient manner. Specifically, the sputtering counting method discloses that during welding, the area where sputtering can be captured is photographed in a dynamic image manner, and for each still image constituting the captured dynamic image, the number of sputtering particles captured in the still image is counted.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2008-126274

[0008] Patent Document 2: Japanese Patent Application Publication No. 2019-188421 Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] However, the behaviors that need to be measured during welding are not limited to sputtering; for example, the shape of the fumes and the weld pool can also be included. While sputtering can be measured as a welding phenomenon in Patent Documents 1 and 2, the simultaneous measurement of multiple other welding behaviors is not considered. In other words, Patent Documents 1 and 2 are methods specifically designed for sputtering measurement.

[0011] The purpose of this application is to provide a measurement method, apparatus, system, program, and welding method and additive manufacturing method that can measure multiple behaviors of welding phenomena based on images captured by a vision sensor.

[0012] Solution for solving the problem

[0013] To address the aforementioned issues, the present invention has the following structure: a method for measuring the behavior of welding phenomena, wherein...

[0014] The methods for measuring the behavior of the welding phenomenon include:

[0015] The image processing step involves performing image processing on the welding images captured by the vision sensor in accordance with the behavior of the welding phenomenon being observed.

[0016] The image segmentation step uses the processed image generated in the image processing step to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0017] The export process uses at least two of the multiple segmented images to export the behavior of the welding phenomenon.

[0018] Additionally, as another aspect of the invention, it has the following structure: a welding system having:

[0019] Welding equipment;

[0020] A visual sensor that captures images of the welding action performed by the welding apparatus; and

[0021] The measuring device uses welding images captured by the vision sensor to measure the behavior of the welding phenomenon.

[0022] The measuring device has:

[0023] An image processing mechanism that performs image processing on the welding image in accordance with the behavior of the welding phenomenon being observed;

[0024] An image segmentation mechanism that uses the processed image generated by the image processing mechanism to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0025] An exporting mechanism that uses at least two of the plurality of segmented images to export the behavior of the welding phenomenon.

[0026] Furthermore, another aspect of the invention has the following structure: a measuring device for the behavior of a welding phenomenon, wherein...

[0027] The measuring device for the behavior of the welding phenomenon has:

[0028] An image processing mechanism that performs image processing on welding images captured by a vision sensor in accordance with the behavior of the welding phenomenon being observed.

[0029] An image segmentation mechanism that uses the processed image generated by the image processing mechanism to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0030] An exporting mechanism that uses at least two of the plurality of segmented images to export the behavior of the welding phenomenon.

[0031] Furthermore, another aspect of the invention has the following structure: a method for measuring the behavior of a welding phenomenon, wherein...

[0032] The measurement methods for the behavior of the aforementioned welding phenomenon include:

[0033] The image processing step involves performing image processing on the welding images captured by the vision sensor, corresponding to the behavior of the welding phenomenon being observed.

[0034] The image segmentation process uses the processed image generated in the image processing process to generate multiple segmented images for each constituent element corresponding to the fusion phenomenon; and

[0035] The export process uses at least two of the multiple segmented images to export the behavior of the fusion phenomenon.

[0036] Furthermore, another aspect of the invention of this application has the following structure: a procedure wherein,

[0037] The program causes the computer to execute:

[0038] The image processing step involves performing image processing on the welding images captured by the vision sensor in accordance with the behavior of the welding phenomenon being observed.

[0039] The image segmentation step uses the processed image generated in the image processing step to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0040] The export process uses at least two of the multiple segmented images to export the behavior of the welding phenomenon.

[0041] Invention Effects

[0042] According to the present invention, multiple behaviors of welding phenomena can be measured based on images captured by a vision sensor. Attached Figure Description

[0043] Figure 1 This is a schematic diagram illustrating a structural example of a welding system according to one embodiment of the invention of this application.

[0044] Figure 2 This is a schematic diagram illustrating a structural example of a robot control device according to an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram illustrating an example of the structural structure of a data processing apparatus according to an embodiment of the present invention.

[0046] Figure 4 This is a flowchart illustrating the processing flow of one embodiment of the invention of this application.

[0047] Figure 5 This is an explanatory diagram illustrating the generation of each color component image in one embodiment of the invention of this application.

[0048] Figure 6 This is a flowchart illustrating the element classification process of one embodiment of the invention of this application.

[0049] Figure 7 This is an explanatory diagram used to illustrate situations where obstacles are included within an image.

[0050] Figure 8 This is an explanatory diagram illustrating the changes in images to show one embodiment of the invention of this application.

[0051] Figure 9 This is an explanatory diagram illustrating the changes in images to show one embodiment of the invention of this application. Detailed Implementation

[0052] Hereinafter, the embodiments for carrying out the present invention will be described with reference to the accompanying drawings. It should be noted that the embodiments described below are intended to illustrate one embodiment of the present invention and are not intended to limit the interpretation of the present invention. Furthermore, the structures described in each embodiment are not necessarily all necessary to solve the problems of the present invention. In addition, in each drawing, the same reference numerals are used to indicate the correspondence of the same constituent elements.

[0053] Furthermore, the method for measuring welding behavior according to this invention is useful not only in welding but also in additive manufacturing technology utilizing GMAW, specifically in wire and arc additive manufacturing (WAAM). It should be noted that the term "additive manufacturing" is sometimes used broadly in the context of stacked modeling or rapid prototyping, but in this invention, the term "additive manufacturing" is used consistently. When applying the method of this invention to additive manufacturing technology, "welding" is replaced with "deposition," "additive manufacturing," or "stacked modeling," etc. For example, when the process is welding, it becomes "welding behavior," but when this invention is used for additive manufacturing, it is replaced with "deposition behavior," or when the process is welding, it becomes "welding system," but when this invention is used for additive manufacturing, it is replaced with "additive manufacturing system."

[0054] <First Implementation>

[0055] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0056] [Structure of the welding system]

[0057] Figure 1 An example of the structure of the welding system 1 of this embodiment is shown. Figure 1 The welding system 1 shown is configured to include a welding robot 10, a robot control device 20, a power supply device 30, a vision sensor 40, and a data processing device 50. It should be noted that when the method of this invention is applied to additive manufacturing, the welding system 1 may be referred to as an additive manufacturing system, and the welding robot 10 may be referred to as an additive manufacturing robot.

[0058] Figure 1 The welding robot 10 shown is a 6-axis articulated robot with a GMAW welding torch 11 mounted on its front end. It should be noted that GMAW includes, for example, MIG (Metal Inert Gas) welding and MAG (Metal Active Gas) welding; this embodiment uses MAG welding as an example. Furthermore, the welding robot 10 is not limited to a 6-axis articulated robot; for example, a small, mobile robot could also be used.

[0059] Welding wire 13 is supplied from the welding wire feed device 12 to the welding torch 11. The welding wire 13 is fed from the tip of the welding torch 11 toward the welding area. The power supply device 30 supplies power to the welding wire 13. With this power, an arc voltage is applied between the welding wire 13 and the workpiece W, generating an arc. The power supply device 30 is equipped with a current sensor (not shown) for detecting the welding current flowing from the welding wire 13 to the workpiece W and a voltage sensor (not shown) for detecting the arc voltage between the welding wire 13 and the workpiece W.

[0060] The power supply unit 30 includes a processing unit and a storage unit (not shown). The processing unit may be, for example, a CPU (Central Processing Unit). The storage unit may be, for example, a volatile or non-volatile memory such as an HDD (Hard Disk Drive), ROM (Read Only Memory), or RAM (Random Access Memory). The processing unit controls the power applied to the welding wire 13 by executing a computer program for power control stored in the storage unit. The power supply unit 30 is also connected to the welding wire feed device 12, and the processing unit controls the feed speed and feed amount of the welding wire 13.

[0061] The composition and type of welding wire 13 vary depending on the welding object. It should be noted that, in addition to the aforementioned sputtering and fumes, the welding behavior in this embodiment may also include arc deflection, arc pressure, oxide coverage on the molten pool, droplet transition morphology, droplet detachment cycle, number of short circuits, and the generation of welding defects such as pits.

[0062] The vision sensor 40 is, for example, composed of a CCD (Charge Coupled Device) camera. The placement of the vision sensor 40 is not particularly limited; it can be directly mounted on the welding robot 10, or it can be fixed in a specific location in the surrounding area as a monitoring camera. When the vision sensor 40 is directly mounted on the welding robot 10, the vision sensor 40 moves in tandem with the welding robot 10, capturing images of the periphery of the front end of the welding torch 11. The number of cameras constituting the vision sensor 40 can also be multiple.

[0063] Furthermore, there are no particular limitations on the direction in which the vision sensor 40 captures images. For example, if the direction of welding progress is set forward, it can be configured to capture images from the front side, or from the side or rear side. Therefore, the shooting range of the vision sensor 40 can be appropriately determined based on the welding behavior of the object being measured. It should be noted that, in the case of sputtering or fumes, in order to suppress interference from the welding torch 11, it is preferable to capture images from the front side.

[0064] In this embodiment, a vision sensor 40 fixed at a specific location is used to capture dynamic images of the welding process, covering a field of view that includes at least the workpiece W, the welding wire 13, and the arc. It should be noted that the various shooting settings related to the welding images can be predetermined or switched according to the operating conditions of the welding system 1. Examples of shooting settings include frame rate, number of pixels, resolution, and shutter speed.

[0065] The various components constituting the welding system 1 are communicatively connected via various wired / wireless communication methods. The communication method is not limited to one; multiple communication methods can be combined and connected.

[0066] [Structure of the robot control device]

[0067] Figure 2 This diagram illustrates a structural example of a robot control device 20 that controls the actions of a welding robot 10. The robot control device 20 comprises a CPU 201 (the overall control unit), a memory 202 for storing data, an operation panel 203 including multiple switches, a teach pendant 204 for use in teaching operations, a robot connection unit 205, and a communication unit 206. The memory 202 is constructed from volatile or non-volatile storage devices such as ROM, RAM, or HDD. The memory 202 stores a control program 202A used in controlling the welding robot 10. The CPU 201 controls various actions performed by the welding robot 10 by executing the control program 202A.

[0068] The teach pendant 204 is primarily used for inputting instructions to the robot control device 20. The teach pendant 204 is connected to the main body of the robot control device 20 via the communication unit 206. The operator can input a teaching program using the teach pendant 204. The robot control device 20 controls the welding robot 10's welding actions according to the teaching program input from the teach pendant 204. It should be noted that the teaching program can also be automatically generated using a computer (not shown) based on CAD (Computer-Aided Design) information. The actions defined by the teaching program are not particularly limited and can vary depending on the specifications of the welding robot 10 and the welding method.

[0069] The drive circuit of the welding robot 10 is connected to the robot connection section 205. The CPU 201 outputs control signals based on the control program 202A to the drive circuit (not shown) of the welding robot 10 via the robot connection section 205.

[0070] The communication unit 206 is a communication module for wired or wireless communication. The communication unit 206 is used for data communication with the power supply unit 30 and the data processing unit 50. The communication method and specifications used by the communication unit 206 are not particularly limited, and multiple methods can be combined. The power supply unit 30 supplies the CPU 201 with, for example, a welding current value detected by a current sensor (not shown) and an arc voltage value detected by a voltage sensor (not shown).

[0071] The robot control device 20 also controls the moving speed and projection direction of the welding torch 11 by controlling each axis of the welding robot 10. Furthermore, the robot control device 20 also controls the lateral movement of the welding robot 10 according to a set cycle, amplitude, and welding speed. The lateral movement refers to the alternating oscillation of the welding torch 11 along a direction intersecting the welding progress direction. The robot control device 20 performs welding line contour control in conjunction with the lateral movement. Welding line contour control is the movement of controlling the left and right position of the welding torch 11 relative to the progress direction of the welding line in a manner that forms a weld bead. In addition, the robot control device 20 also controls the feed speed of the welding wire 13 by controlling the welding wire feed device 12 via the power supply device 30.

[0072] [Structure of the data processing device]

[0073] Figure 3 This diagram illustrates an example of the functional structure of the data processing apparatus 50. The data processing apparatus 50 comprises, for example, a CPU (not shown), ROM, RAM, hard disk drive, input / output interface, communication interface, and image output interface. Through the cooperation of these components, the data processing apparatus 50 realizes the storage unit 501, image processing unit 502, image segmentation unit 503, calculation unit 504, and display unit 505. Furthermore, if the vision sensor 40 is a fixed surveillance camera, the data processing apparatus 50 also includes a vision sensor control unit 506. It should be noted that the series of processes performed by the image processing unit 502, image segmentation unit 503, calculation unit 504, and display unit 505 can also be performed by software installed on the data processing apparatus 50.

[0074] The storage unit 501 records and manages the image data captured by the vision sensor 40, and provides image data according to requests from each processing unit. The image data referred to here can be still image data, or it can be moving image data obtained by continuously capturing still image data at any frame rate. The frame rate referred to here, for example, represents the number of still image data frames captured by the vision sensor 40 within a predetermined time interval such as one second. Preferably, it is determined within the range of 1 to 10 FPS (Frames Per Second). It should be noted that when used for measuring and tracking welding behavior in real time, it is preferable to record it as moving images.

[0075] The image processing unit 502 uses the image data stored in the storage unit 501 to perform preprocessing for the measurements in this embodiment. Examples of preprocessing include contrast correction, brightness correction, color correction, black-and-white image transformation such as binarization, noise removal, edge enhancement, shrinking / dilation, and image feature extraction. Specific examples of image processing in this embodiment will be described later.

[0076] The image segmentation unit 503 generates segmented images based on the processed image data obtained by applying various image processing techniques in the image processing unit 502, dividing each pre-defined constituent element into multiple images. It should be noted that the pre-defined constituent elements can include, for example, sputtering, fumes, and arc light as welding activities; welding wire 13, nozzle, base material, molten pool, obstacles, and other background elements as components of the welding system 1. Here, sputtering, fumes, and arc light will be specifically discussed. It should be noted that the processing performed by the image processing unit 502 is not limited to the pre-processing used by the image segmentation unit 503 to generate the segmented images. The image processing unit 502 may also process the segmented images as needed.

[0077] The calculation unit 504 calculates various index values ​​for quantitatively measuring sputtering, fumes, and arc light as welding behaviors based on image segmentation and image data processing. The calculation here is preferably performed sequentially in a time series manner. Examples of time series include elapsed time, teaching procedure sequence, and sequence of continuous still image data constituting a dynamic image.

[0078] The display unit 505 displays a screen based on the calculation results calculated by the calculation unit 504. Additionally, the display unit 505 displays values ​​detected by the current sensor 41 and voltage sensor 42, as well as various information obtained from the robot control device 20, on the screen. The results calculated by the calculation unit 504 along a time sequence are preferably displayed synchronously with data such as welding current and arc voltage.

[0079] The vision sensor control unit 506 controls the imaging action of the vision sensor 40. For example, if the vision sensor 40 is installed outside the welding robot 10, it is preferable to use a camera with at least PTZ functionality as the vision sensor 40, so that its pan, tilt, and zoom functions can be controlled in conjunction with the movements of the welding robot 10. In addition, the vision sensor control unit 506 can obtain information related to the movements of the welding robot 10 from the robot control device 20 and control the movement of the vision sensor 40 based on this information.

[0080] [Measurement Method]

[0081] Figure 4 This is a diagram illustrating a series of processes used to measure multiple welding actions from image data. Figure 4 The diagram illustrates a series of processes involving sputtering, fumes, and arc light, as an example of welding, but it's not necessary to measure all of them; a structure where only a portion is measured is also acceptable. Furthermore, in... Figure 4 For example, multiple index values ​​can be calculated for a single welding action, but it is not necessary to calculate all the index values. Alternatively, it can be switched to calculate only one index value for a single welding action. It should be noted that this embodiment is an example of processing dynamic image data after welding for tracking purposes, but the processing is not limited to after welding. The processing shown below can also be performed in real time in parallel with the welding action.

[0082] Figure 4 The processing flow can be implemented as follows: using images captured in parallel with or during welding, when measuring welding behavior, the processing unit of the data processing device 50 reads and executes the program stored in the storage unit, thereby... Figure 3 Each part shown performs its function.

[0083] In step S401, the data processing unit 50 acquires dynamic image data that is the object of processing. Here, shooting conditions such as frame rate and shutter speed are set for the vision sensor 40, and dynamic image data is captured by the vision sensor 40 of the area of ​​the welding position that will be the object of shooting. The shooting conditions can be set to arbitrary values ​​by the operator, or pre-defined fixed values ​​can be used. The captured dynamic image data can be directly saved to the storage unit 501 of the data processing unit 50. If the vision sensor 40 itself has a memory, the dynamic image data can be temporarily saved in the memory of the vision sensor 40 before being transferred to the storage unit 501. It should be noted that subsequent processing processes the multiple still image data included in the dynamic image data separately.

[0084] In S402, the data processing device 50 performs color component decomposition processing on the acquired motion image data. The motion image data in this embodiment is, for example, a color image composed of RGB signals representing red, green, and blue color components for each pixel. For example, each color component of the RGB signal is set to 8 bits, and each pixel is represented by a total of 24 bits. In this case, the signal value corresponding to each color component takes a value from 0 to 255. In this color component decomposition processing, focusing on each color component, motion image data is created separately for each RGB color component. In other words, motion image data is generated by dividing a single motion image data into motion image data containing only the R color component, motion image data containing only the G color component, and motion image data containing only the B color component. More specifically, when generating motion image data containing only the R color component, the signal values ​​of the G and B components of the motion image data are transformed to 0, thereby performing color component decomposition processing.

[0085] Figure 5 This is a diagram used to illustrate the generation of dynamic image data from dynamic image data, where only the RGB color components are represented. For example... Figure 5 As shown, the three still image data generated from a single still image data contained in the dynamic image data each have different representations, and different features can be captured even when the same welding behavior is produced. Hereinafter, the still image data with only the R color component, the still image data with only the G color component, and the still image data with only the B color component will be referred to as "red color component image", "green color component image", and "blue color component image", respectively.

[0086] The inventors of this application, through experiments and verifications, discovered that blue component images can clearly identify thermal light. Thermal light is a phenomenon associated with electric arc light or flue gas. That is, by using blue component images, it is possible to extract the fainter shades of light within the thermal light, and based on these fainter shades, it is possible to calculate the previously difficult-to-extract flue gas spreading outwards.

[0087] Furthermore, the inventors of this application, through experiments and tests, discovered that the red component image can be clearly identified in the high-temperature luminescence of metals, molten slag, etc. That is, by using the red component image, sputtering, molten pools, or flue gas with high particle density can be captured based on the high-temperature luminescence it contains. Hereinafter, flue gas with high particle density will be referred to as "dense flue gas" in the image, and flue gas with low particle density will be referred to as "thin flue gas." It should be noted that the terms "dense" and "thin" are relative, and the concentration values ​​are not limited.

[0088] By decomposing the image into its RGB components, feature extraction for various welding behaviors becomes easier. It should be noted that in this embodiment, an example of measuring welding behavior using red and blue color component images is described. However, this is not a limitation; green color component images can also be used to measure welding behavior. For example, green color component images can also be used in areas such as determining the regions of constituent elements, as described later.

[0089] It should be noted that this embodiment uses the RGB color space as an example, but it is not limited to this. For example, other color spaces that can be transformed in accordance with the parameters of R, G, and B can also be used. More specifically, RGBA, YCbCr, YUV, etc., can be used as examples of color spaces.

[0090] First, the calculation of the index value for measuring thin smoke using the blue component image will be explained. In S403, the data processing apparatus 50 applies background subtraction processing to the blue component image using the image processing unit 502. The method of background subtraction processing is not particularly limited, but for example, noise can be removed using a known rolling ball algorithm. Alternatively, background subtraction processing can be performed using a predefined filter. Through this process, spike-like signals can be removed, resulting in smoothly varying pixel values. In this embodiment, the smoothly varying pixel values ​​are derived from the thin smoke.

[0091] In S404, the data processing device 50 uses the calculation unit 504 to calculate the total brightness value of the blue component image, which underwent background subtraction processing in S403, as an index value for the thin smoke. Here, the index value can be calculated based on the weighted total value of each brightness value in the brightness histogram represented by the entire blue component image.

[0092] Next, the index values ​​for measuring arc light and dense smoke using the red color component image will be explained. In S405, the data processing apparatus 50 applies background subtraction processing to the red color component image using the image processing unit 502. The method for background removal is not particularly limited, but for example, it can be performed by removing noise using a known rolling ball algorithm, similar to the processing in S403. Alternatively, background subtraction can be performed by using a predefined filter. Through this process, spike-like signals can be removed, resulting in smoothly varying pixel values.

[0093] In S406, the data processing device 50 calculates the total brightness value as an index value of the arc light from the red color component image, which has undergone background subtraction processing by the calculation unit 504 in S405. Here, the index value can be calculated as a weighted total value based on each brightness value in the brightness histogram represented by the red color component image as a whole.

[0094] In step S407, the data processing device 50 uses the image processing unit 502 to extract the brightness values ​​of each pixel in the red component image generated in step S402, excluding the brightness values ​​of the red component image after background subtraction processing generated in step S405. This process allows for the exclusion of smoothly varying pixel values ​​from the red component image.

[0095] In step S408, the data processing device 50 uses the image processing unit 502 to binarize the red color component image processed in step S407, generating a binarized image. The binarization method is not particularly limited, and known methods can be used. Furthermore, the threshold setting during binarization is not particularly limited; for example, the center value of the possible pixel values ​​can be set as the threshold.

[0096] In S409, the data processing apparatus 50, using the image processing unit 502, performs labeling processing on the regions contained in the binarized image generated in S408. The binarized image includes multiple regions composed of one or more pixels, and each region is extracted. In this embodiment, in the binarized image, each region composed of pixels with a pixel value of "1" is labeled as a region corresponding to any one of the constituent elements generated by the welding action. The labeling method is not particularly limited, and known methods can be used. Furthermore, the lower limit of the region size is not particularly limited; for example, the smallest region can be set to a region composed of one pixel. It should be noted that regions composed of pixels with a pixel value of "0" can also be labeled if they correspond to a constituent element generated by the welding action.

[0097] In step S410, the data processing apparatus 50 uses the image segmentation unit 503 to perform feature classification processing on each region within the image marked in step S409. For details of this process, please refer to... Figure 6 This process is performed each time, based on the results of labeling processing using multiple red color component images that are the objects of the processing.

[0098] In S601, the image segmentation unit 503 focuses on unprocessed regions among one or more labeled regions contained in the binarized image. At this time, the order of focusing is not particularly limited, but for example, it can be sorted in descending order based on the size of the regions, and focusing on the regions in turn starting from the larger regions.

[0099] In S602, the image segmentation unit 503 determines whether the number of pixels constituting the viewing area is below a first threshold. Here, the first threshold is explained as 300 pixels. It should be noted that the first threshold can also be determined based on the overall size of the red color component image, or it can vary depending on the welding condition. For example, if the vision sensor 40 is a fixed-position surveillance camera, the size of the object being photographed changes due to changes in the welding position—that is, changes in the distance between the vision sensor 40 and the welding position, or changes in the shooting direction or shooting angle. Therefore, the relationship between distance, direction, and the size of the object can be preset, and the first threshold can be changed based on this relationship. On the other hand, the magnification on the camera side can be changed in a way that keeps the size of the object constant even if the distance between the vision sensor 40 and the welding position changes, that is, keeps the first threshold constant. If the number of pixels in the viewing area is below the first threshold, that is, if it is in S602, the processing of the image segmentation unit 503 proceeds to S604. On the other hand, if the number of pixels in the focus area is greater than the first threshold, i.e., if not in S602, the processing of the image segmentation unit 503 proceeds to S603.

[0100] In S603, the image segmentation unit 503 determines whether the focus area is located in the center of the image and is the largest area among the marked regions. That is, in an image obtained by capturing a typical welding process, the arc light is located in the center, and the area of ​​the arc light is the largest area in the image. On the other hand, if obstacles or the like are captured during the capture, there may be cases where the arc light is not in the center. In such cases, the focus area is treated as noise. Figure 7 An example is shown where an obstacle is captured in the image. In such cases, the image becomes one where the arc light is not located in the center of the image. It should be noted that the center here can be preset within a certain range, or it can vary depending on the image size, welding behavior, etc. In addition, the largest size used in the determination of this process becomes the relative size between multiple regions within the image, and therefore varies depending on the image. If the observed region meets the above conditions, that is, if it is true in S603, the processing of the image segmentation unit 503 proceeds to S605. On the other hand, if the observed region does not meet the above conditions, that is, if it is not true in S603, the processing of the image segmentation unit 503 proceeds to S606.

[0101] In S604, the image segmentation unit 503 determines whether the size of the eye region is greater than or equal to a second threshold. The second threshold defines the smallest rectangular region containing the eye region and is set as the ratio of the number of pixels in the eye region to the number of pixels in that rectangular region. Therefore, the size of the rectangular region varies depending on the size of each eye region. Here, the second threshold is described as 15%. That is, the determination in this process is based on whether the following conditions are met.

[0102] The second threshold is ≤ (size of the eye region) / (size of the rectangular region containing the eye region).

[0103] If the size of the area being viewed is greater than or equal to the second threshold (i.e., if it is true in S604), the image segmentation unit 503 proceeds to S607. On the other hand, if the size of the area being viewed is smaller than the second threshold (i.e., if it is not true in S604), the image segmentation unit 503 proceeds to S608.

[0104] In S605, the image segmentation unit 503 classifies the area of ​​interest as a region of electric arc light. Then, the processing of the image segmentation unit 503 proceeds to S609.

[0105] In S606, the image segmentation unit 503 classifies the region of interest as a noisy region. Then, the processing of the image segmentation unit 503 proceeds to S609.

[0106] In S607, the image segmentation unit 503 classifies the area of ​​interest as a sputtered region. Then, the processing of the image segmentation unit 503 proceeds to S609.

[0107] In S608, the image segmentation unit 503 classifies the area of ​​focus as a region of dense smoke. Then, the processing of the image segmentation unit 503 proceeds to S609.

[0108] In S609, the image segmentation unit 503 determines whether there are any unprocessed regions. If there are unprocessed regions, i.e., if yes in S609, the image segmentation unit 503 returns to S601 and repeats the process. On the other hand, if there are no unprocessed regions, i.e., if no in S609, the current processing flow ends, and the process is returned to... Figure 4 The S411 advances.

[0109] return Figure 4 The process of calculating the index value of the electric arc light is explained. In S411, the data processing device 50 generates the index value by using... Figure 6The binary image described is composed of regions classified as arc light by the element classification processing. This binary image is generated by extracting the regions classified as arc light from the binary image marked in S409. The binary image at this time contains the constituent elements corresponding to glare. Glare is light generated by reflection in the lens or camera constituting the vision sensor 40. Therefore, the data processing device 50 uses the image segmentation unit 503 to perform shrinking and dilation processing on the generated binary image in order to remove the constituent elements of glare. The shrinking and dilation processing can use known methods. In order to properly remove the constituent elements of glare, multiple shrinking and dilation processing can be performed, and the processing order is not particularly limited.

[0110] In S412, the data processing device 50 uses the calculation unit 504 to calculate the index value of the arc light using the binarized image processed in S411. The calculation unit 504 counts the number of pixels in the region of the arc light contained in the binarized image and uses the value as the index value.

[0111] It should be noted that the S406 process calculates the arc light index value based on the brightness value, while the S412 process calculates the arc light index value based on the number of pixels. These can be processed as individual index values, or a single index value for the entire arc light can be derived from the two index values ​​mentioned above. Furthermore, as an index value for the arc light, the arc width, arc length, and arc deflection direction can also be calculated based on factors such as the area, centroid, and principal axis angle of the arc light region.

[0112] Next, the operation of calculating the sputtering index value will be explained. In S413, the data processing device 50 uses the image processing unit 502 to generate a value based on the red color component image processed in S407. Figure 6 The red color component image is formed by the areas classified as sputtered through element classification processing.

[0113] In S414, the data processing device 50 uses the calculation unit 504 to calculate the sputtering index value using the red color component image generated in S413. First, the calculation unit 504 removes regions from each sputtered region included in the red color component image that have an area of ​​a predetermined threshold or a number of pixels of a predetermined threshold or higher. For this, each sputtering region is assumed to be smaller than a predetermined size, and that region is treated as background and removed. The threshold is not particularly limited here, but is set to a predetermined threshold. Next, the calculation unit 504 determines the remaining sputtered regions and calculates the number of pixels thereon and the number of regions formed by the determined pixels as the sputtering index value. At this time, when counting the number of pixels, only pixels with a value of R of a predetermined threshold or higher may be counted.

[0114] It should be noted that when calculating the evaluation value here, the correspondence between the actual sputtering measurements and the values ​​calculated from the image in this embodiment, which are predefined using formulas, tables, etc., can also be used to derive the index value. In this case, formulas and tables can be used to transform the values ​​calculated from the image into a sputtering amount representing the weight per unit time.

[0115] Next, the operation of calculating the index value of dense smoke will be explained. In S415, the data processing device 50 uses the image processing unit 502 to subtract the value of the sputtered area generated in S413 from the image generated in S407, thereby generating an image with the sputtering removed.

[0116] In step S416, the data processing apparatus 50 performs gamma correction on the image generated in step S415 using the image processing unit 502. By transforming the brightness values ​​using gamma correction, regions with small brightness values ​​within the image are excluded. The threshold for the excluded regions is not particularly limited and is set to a predefined threshold. Furthermore, known methods can be used for gamma correction; for example, the structure of the gamma curve is not particularly limited.

[0117] In step S417, the data processing apparatus 50 uses the image processing unit 502 to perform filtering processing on the image processed in step S416. Through filtering, edges in the image are detected, and areas with steep brightness gradients are emphasized. For example, a Laplacian filter can be used in this filtering process, but other filters can also be used.

[0118] In S418, the data processing device 50, using the image processing unit 502, performs region segmentation based on brightness gradients on the image processed by the filter in S417. This region segmentation process may be performed using, for example, the watershed algorithm. The watershed algorithm can more finely segment and emphasize areas with steep brightness gradients. It should be noted that the region segmentation method used is not particularly limited and other methods may also be used.

[0119] In S419, the data processing device 50 uses the calculation unit 504 to calculate the index value of dense smoke based on the image generated in S418. In the processing of S418, the image is divided into multiple regions. At this time, the more small the segmented regions, the more density exists. In this embodiment, the regions with more density, that is, the segmented regions with a predetermined area or less, are defined as the locations where dense smoke is generated, and the total area is assumed to be the index value representing dense smoke. In this embodiment, the index value of dense smoke is derived using the following formula (1). In the following formula (1), T n This represents the area of ​​the segmented region n, i.e., the number of pixels (n=1,…,i).

[0120] [Mathematical Formula 1]

[0121]

[0122] It should be noted that when calculating the evaluation value here, the correspondence between the actual measurement of the generated flue gas and the value calculated from the image in this embodiment can also be predefined using formulas, tables, etc., and these can be used to derive the index value. In this case, formulas and tables can be used to transform the measured value from the image into a quantity of flue gas representing the weight per unit time.

[0123] Furthermore, the index value for thin smoke is calculated in the S404 process, and the index value for dense smoke is calculated in the S419 process. These can be processed into individual index values, or a single index value for the entire smoke can be derived from the two index values ​​mentioned above using a prescribed transformation formula.

[0124] After exporting the aforementioned index values, the data processing device 50 displays them on a screen (not shown) using the display unit 505. At this time, the multiple welding behaviors such as sputtering and fumes determined by the calculated index values ​​are preferably displayed in a time sequence. Alternatively, the results of the calculations for the index values ​​not only for these welding behaviors can be displayed in a time sequence, synchronizing and arranging welding current, arc voltage, and other parameters, making it easy to visually confirm the comparison objects.

[0125] Figure 8 , Figure 9 This is a diagram used to illustrate the changes in an image based on image processing in the above-described process. Figure 8 This illustrates the transition from the red color component image to the image that generates each component, and is equivalent to... Figure 4 Image processing based on S405 to S415 in the processing sequence.

[0126] Image 801 shows an example of a red color component image, and also shows the image after color component decomposition processing. Image 802 shows the image after applying binarization processing to image 801. Images 803 and 805 show images generated for each constituent element by applying labeling processing and element decomposition processing to image 802. Image 803 is an image composed of regions classified as sputtering, and is equivalent to the image generated in S411. Image 805 is an image composed of regions classified as arc light, and is equivalent to the image generated in S415.

[0127] Image 804 is an image from image 803 with the area considered as background removed, and is equivalent to the image generated in S414. Image 806 represents the case where no obstacle was captured in image 805. Figure 7As shown, when an obstacle is included in the image, there is no region in the center of the image that is classified as arc light. On the other hand, in image 806, there is a region in the center of the image that is classified as arc light.

[0128] Figure 9 This illustrates the transition from the red color component image to a region-segmented image, and is equivalent to... Figure 4 Image processing based on S415 to S418 in the processing sequence.

[0129] Image 901 shows an example of an image with the red component of the sputtered region removed, and is equivalent to the image generated in S415. Image 902 is an image of image 901 with gamma correction and filtering applied, and is equivalent to the processed image in S417. Image 903 is an image of image 902 with region segmentation applied, and is equivalent to the processed image in S418.

[0130] It should be noted that a more preferred option is as follows: Figure 4 As shown in the S402, S405, and S408 flowcharts, the processing proceeds in the order of color processing, background subtraction (a smoothing-related process), and binarization. This allows for the accurate detection of areas containing welding activity within the image.

[0131] In this embodiment, index values ​​corresponding to multiple welding behaviors occurring during welding can be calculated based on images. In particular, previously difficult-to-detect phenomena such as varying concentrations of fumes and sputtering can also be detected simultaneously. Therefore, multiple behaviors of the welding phenomenon can be measured based on images captured by an image sensor. Especially noteworthy is that while conventional methods struggled to simultaneously measure sputtering and fumes, the method of this embodiment enables the measurement of multiple behaviors including both.

[0132] <Other Implementation Methods>

[0133] The structure described above can also include a structure that allows setting the measurement time. For example, it could be a structure that allows specifying a time period within a set length of dynamic image data that will be the object of measurement. Furthermore, during this time period, the pixel and brightness values ​​of fumes, sputtering, and arc light can be counted, and the values ​​of each index can be calculated. Thus, for example, welding behavior can be confirmed based on welding conditions within the specified time period.

[0134] Alternatively, the structure described above could also be one where the actions of the welding robot 10 and the vision sensor 40 are controlled based on measurement results, i.e., index values ​​corresponding to each welding action. For example, the shooting settings of the vision sensor 40 could be switched, and various welding parameters of the welding robot 10 could be controlled. Thus, for example, the welding robot 10 could be made to move more appropriately according to the welding action.

[0135] In addition, in this invention, a program or application that performs the functions of one or more of the above-described embodiments can also be implemented by the following process: the program is supplied to a system or device using a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program.

[0136] Alternatively, it can be achieved using a circuit that performs more than one function. It should be noted that examples of circuits that perform more than one function include ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays).

[0137] As stated above, the following matters are disclosed in this specification.

[0138] (1) A method for measuring the behavior of a welding phenomenon, characterized in that,

[0139] The methods for measuring the behavior of the welding phenomenon include:

[0140] The image processing step involves performing image processing on the welding images captured by the vision sensor in accordance with the behavior of the welding phenomenon being observed.

[0141] The image segmentation step uses the processed image generated in the image processing step to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0142] The export process uses at least two of the multiple segmented images to export the behavior of the welding phenomenon.

[0143] Based on this structure, multiple behaviors of the welding phenomenon can be measured based on images captured by an image sensor.

[0144] (2) The method for measuring the behavior of welding phenomena according to (1) is characterized in that,

[0145] The constituent elements corresponding to the welding phenomenon include at least two of the following: sputtering, fumes, arc light, molten pool, background, and obstacles.

[0146] According to this structure, as constituent elements corresponding to welding phenomena, it is possible to detect at least two of the following: sputtering, fumes, arc light, molten pool, background, and obstacles.

[0147] (3) The method for measuring the behavior of welding phenomena according to (1) or (2), characterized in that,

[0148] The image segmentation process includes:

[0149] The marking process involves marking the pixels contained in the processed image; and

[0150] The classification process categorizes one or more regions, consisting of the pixel groups marked in the marking process, into constituent elements corresponding to the welding phenomenon.

[0151] The classification process includes at least one of the following steps: classifying the one or more regions into constituent elements corresponding to the welding phenomenon based on the number of pixels; classifying the one or more regions into constituent elements corresponding to the welding phenomenon based on both position and size; and classifying the one or more regions into constituent elements corresponding to the welding phenomenon based on the proportion of the pixel group constituting the focus area relative to the rectangular area containing the focus area.

[0152] Based on this structure, it is possible to properly classify the regions within the image that correspond to the welding phenomenon.

[0153] (4) A method for measuring the behavior of welding phenomena described in any one of (1) to (3), characterized in that,

[0154] In the image segmentation process, at least a segmented image consisting of the sputtered region and a segmented image consisting of the flue gas region are generated from the constituent elements corresponding to the welding phenomenon.

[0155] According to this structure, sputtering and fumes, which are part of the welding process, can be detected separately.

[0156] (5) A method for measuring the behavior of welding phenomena described in any one of (1) to (4), characterized in that,

[0157] In the export process, as a behavior of the welding phenomenon, an index value of at least one of sputtering and flue gas is exported.

[0158] Based on this structure, the behavior of sputtering and flue gas can be measured.

[0159] (6) The method for measuring the behavior of welding phenomena according to (5) is characterized in that,

[0160] In the export process, when exporting the index values ​​of the flue gas,

[0161] Edge detection is performed on the segmented image after removing the sputtered regions.

[0162] Based on the detected edges, the system is segmented into multiple regions.

[0163] The index values ​​of the flue gas are calculated based on the area of ​​the multiple regions.

[0164] Based on this structure, the flue gas can be properly measured as a welding phenomenon.

[0165] (7) The method for measuring the behavior of welding phenomena according to (5) is characterized in that,

[0166] The method for measuring the behavior of the welding phenomenon also includes a setting process for setting the period that is the object of measurement.

[0167] Using the welding images included in the period set in the set process, processing is performed based on the image processing process, the image segmentation process, and the export process.

[0168] Based on this structure, welding behavior can be measured within a desired time range.

[0169] (8) A method for measuring the behavior of welding phenomena described in any one of (1) to (7), characterized in that,

[0170] The image processing steps include at least one of the following: decomposing the welding image into images for each color component, binarization, and acquiring or excluding smoothly varying pixel values.

[0171] Based on this structure, as image processing for measuring welding behavior, any one of color decomposition processing, binarization processing, or exception processing for specific pixel values ​​can be applied.

[0172] (9) The method for measuring the behavior of welding phenomena according to (8) is characterized in that,

[0173] In the process of decomposing the image into images for each of the color components, color component images for each RGB color component are generated from the welding image.

[0174] According to this structure, in order to measure the welding behavior under observation, it is possible to generate and utilize images of the respective color components of RGB.

[0175] (10) The method for measuring the behavior of welding phenomena according to (9) is characterized in that,

[0176] Using the color component image of R, derive the index values ​​for at least one of the following: arc light, sputtering, and dense smoke.

[0177] Based on this structure, any one of arc light, sputtering, or dense smoke can be measured by using the color component image of R.

[0178] (11) A method for measuring the behavior of welding phenomena according to (9) or (10), characterized in that,

[0179] Using the color component image of B, derive the index values ​​for the thin smoke.

[0180] Based on this structure, thin smoke can be measured by using the color component image of B.

[0181] (12) A welding system, characterized in that,

[0182] The welding system has the following features:

[0183] Welding equipment;

[0184] A visual sensor that captures images of the welding action performed by the welding apparatus; and

[0185] The measuring device uses welding images captured by the vision sensor to measure the behavior of the welding phenomenon.

[0186] The welding system is characterized in that,

[0187] The measuring device has the following features:

[0188] An image processing mechanism that performs image processing on the welding image in accordance with the behavior of the welding phenomenon being observed;

[0189] An image segmentation mechanism that uses the processed image generated by the image processing mechanism to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0190] An exporting mechanism that uses at least two of the plurality of segmented images to export the behavior of the welding phenomenon.

[0191] Based on this structure, multiple behaviors of welding phenomena generated in the welding system can be measured based on images captured by an image sensor.

[0192] (13) A measuring device for the behavior of welding phenomena, characterized in that,

[0193] The measuring device for the behavior of the welding phenomenon has:

[0194] An image processing mechanism that performs image processing on welding images captured by a vision sensor in accordance with the behavior of the welding phenomenon being observed.

[0195] An image segmentation mechanism that uses the processed image generated by the image processing mechanism to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0196] An exporting mechanism that uses at least two of the plurality of segmented images to export the behavior of the welding phenomenon.

[0197] Based on this structure, multiple behaviors of the welding phenomenon can be measured based on images captured by an image sensor.

[0198] (14) A welding method, characterized in that,

[0199] The welding method controls the welding action based on the behavior of the welding phenomenon derived from the measuring device of claim 13.

[0200] Based on this structure, welding actions can be controlled based on multiple behaviors of the measured welding phenomenon.

[0201] (15) A method for measuring the behavior of a welding phenomenon, characterized in that,

[0202] The measurement methods for the behavior of the aforementioned welding phenomenon include:

[0203] The image processing step involves performing image processing on the welding images captured by the vision sensor, corresponding to the behavior of the welding phenomenon being observed.

[0204] The image segmentation process uses the processed image generated in the image processing process to generate multiple segmented images for each constituent element corresponding to the fusion phenomenon; and

[0205] The export process uses at least two of the multiple segmented images to export the behavior of the fusion phenomenon.

[0206] According to this structure, multiple behaviors of the deposition phenomenon in additive manufacturing can be measured based on images captured by an image sensor.

[0207] (16) A program, wherein,

[0208] The above is used to enable the computer to perform:

[0209] The image processing step involves performing image processing on the welding images captured by the vision sensor in accordance with the behavior of the welding phenomenon being observed.

[0210] The image segmentation step uses the processed image generated in the image processing step to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and

[0211] The export process uses at least two of the multiple segmented images to export the behavior of the welding phenomenon.

[0212] Based on this structure, multiple behaviors of the welding phenomenon can be measured based on images captured by an image sensor.

[0213] Various embodiments have been described above with reference to the accompanying drawings, but the present invention is not limited to these examples. It is self-evident that those skilled in the art will be able to conceive of various modifications or alterations within the scope of the patented technical solutions, and these are also understood to fall within the technical scope of the present invention. Furthermore, the constituent elements of the above embodiments can be arbitrarily combined without departing from the spirit of the invention.

[0214] It should be noted that this application is based on Japanese patent application (Japanese Patent Application No. 2021-118757) filed on July 19, 2021, the contents of which are referenced in this application.

[0215] Explanation of reference numerals in the attached figures

[0216] 1 Welding System

[0217] 10 Welding Robots

[0218] 11 Welding torch

[0219] 12 Welding wire feeding device

[0220] 13 Welding wire

[0221] 20 Robot control device

[0222] 30 Power supply unit

[0223] 40 visual sensors

[0224] 41 Current Sensor

[0225] 42 Voltage Sensor

[0226] 50 Data processing devices

[0227] 201 CPU (Central Processing Unit)

[0228] 202 Memory

[0229] 202A Control Procedure

[0230] 203 Control Panel

[0231] 204 Teach Pendant

[0232] 205 Robot Connection Section

[0233] 206 Ministry of Communications

[0234] 501 Storage Department

[0235] 502 Image Processing Department

[0236] 503 Image Segmentation Unit

[0237] 504 Calculation Department

[0238] 505 Display Section

[0239] 506 Vision Sensor Control Unit.

Claims

1. A method for measuring the behavior of welding phenomena, characterized in that, The methods for measuring the behavior of the welding phenomenon include: The image processing step involves performing image processing on the welding images captured by the vision sensor in accordance with the behavior of the welding phenomenon being observed. The image segmentation step uses the processed image generated in the image processing step to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and The export process uses at least two of the multiple segmented images to export the behavior of the welding phenomenon. In the export process, as a behavior of the welding phenomenon, the index values ​​of at least one of sputtering and fumes are exported. In the export process, when exporting the index values ​​of the flue gas, Edge detection is performed on the segmented image after removing the sputtered regions. Based on the detected edges, the system is segmented into multiple regions. The index values ​​of the flue gas are calculated based on the area of ​​the multiple regions.

2. The method for measuring the behavior of welding phenomena according to claim 1, characterized in that, The constituent elements corresponding to the welding phenomenon include at least two of the following: sputtering, fumes, arc light, molten pool, background, and obstacles.

3. The method for measuring the behavior of welding phenomena according to claim 1 or 2, characterized in that, The image segmentation process includes: The marking process involves marking the pixels contained in the processed image; and The classification process categorizes one or more regions, consisting of the pixel groups marked in the marking process, into constituent elements corresponding to the welding phenomenon. The classification process includes at least one of the following steps: classifying the one or more regions into constituent elements corresponding to the welding phenomenon based on the number of pixels; classifying the one or more regions into constituent elements corresponding to the welding phenomenon based on both position and size; and classifying the one or more regions into constituent elements corresponding to the welding phenomenon based on the proportion of the pixel group constituting the focus area relative to the rectangular area containing the focus area.

4. The method for measuring the behavior of welding phenomena according to claim 1 or 2, characterized in that, In the image segmentation process, at least a segmented image consisting of the sputtered region and a segmented image consisting of the flue gas region are generated from the constituent elements corresponding to the welding phenomenon.

5. The method for measuring the behavior of welding phenomena according to claim 1, characterized in that, The method for measuring the behavior of the welding phenomenon also includes a setting process for setting the period that is the object of measurement. Using the welding images included in the period set in the set process, processing is performed based on the image processing process, the image segmentation process, and the export process.

6. The method for measuring the behavior of welding phenomena according to claim 1 or 2, characterized in that, The image processing steps include at least one of the following: decomposing the welding image into images for each color component, binarization, and acquiring or excluding smoothly varying pixel values.

7. The method for measuring the behavior of welding phenomena according to claim 6, characterized in that, In the process of decomposing the image into images for each color component, color component images for each RGB color component are generated from the welding image.

8. The method for measuring the behavior of welding phenomena according to claim 7, characterized in that, Using the color component image of R, derive the index values ​​for at least one of the following: arc light, sputtering, and dense smoke.

9. The method for measuring the behavior of welding phenomena according to claim 7 or 8, characterized in that, Using the color component image of B, derive the index values ​​for the thin smoke.

10. A welding system comprising: Welding equipment; A visual sensor that captures images of the welding action performed by the welding apparatus; and The measuring device uses welding images captured by the vision sensor to measure the behavior of the welding phenomenon. The welding system is characterized in that, The measuring device has: An image processing mechanism that performs image processing on the welding image in accordance with the behavior of the welding phenomenon being observed; An image segmentation mechanism that uses the processed image generated by the image processing mechanism to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; and An exporting mechanism that uses at least two of the plurality of segmented images to export the behavior of the welding phenomenon. The export mechanism exports an index value for at least one of the sputtering and flue gas, as a behavior of the welding phenomenon. When the exporting mechanism exports the index values ​​of the flue gas, Edge detection is performed on the segmented image after removing the sputtered regions. Based on the detected edges, the system is segmented into multiple regions. The index values ​​of the flue gas are calculated based on the area of ​​the multiple regions.

11. A measuring device for the behavior of welding phenomena, characterized in that, The measuring device for the behavior of the welding phenomenon has: An image processing mechanism that performs image processing on welding images captured by a vision sensor in accordance with the behavior of the welding phenomenon being observed. An image segmentation mechanism that uses the processed image generated by the image processing mechanism to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; as well as An exporting mechanism that uses at least two of the plurality of segmented images to export the behavior of the welding phenomenon. The export mechanism exports an index value for at least one of the sputtering and flue gas, as a behavior of the welding phenomenon. When the exporting mechanism exports the index values ​​of the flue gas, Edge detection is performed on the segmented image after removing the sputtered regions. Based on the detected edges, the system is segmented into multiple regions. The index values ​​of the flue gas are calculated based on the area of ​​the multiple regions.

12. A welding method, characterized in that, The welding method controls the welding action based on the behavior of the welding phenomenon derived from the measuring device of claim 11.

13. A method for measuring the behavior of a welding phenomenon, characterized in that, The measurement methods for the behavior of the aforementioned welding phenomenon include: The image processing step involves performing image processing on the welding images captured by the vision sensor, corresponding to the behavior of the welding phenomenon being observed. The image segmentation process uses the processed image generated in the image processing process to generate multiple segmented images for each constituent element corresponding to the fusion phenomenon; and The export process uses at least two of the multiple segmented images to export the behavior of the fusion phenomenon. In the export process, as a behavior of the deposition phenomenon, the index values ​​of at least one of sputtering and flue gas are exported. In the export process, when exporting the index values ​​of the flue gas, Edge detection is performed on the segmented image after removing the sputtered regions. Based on the detected edges, the system is segmented into multiple regions. The index values ​​of the flue gas are calculated based on the area of ​​the multiple regions.

14. A storage medium storing a program, wherein, The program is used to cause the computer to execute: The image processing step involves performing image processing on the welding images captured by the vision sensor in accordance with the behavior of the welding phenomenon being observed. The image segmentation process uses the processed image generated in the image processing process to generate multiple segmented images for each constituent element corresponding to the welding phenomenon; as well as The export process uses at least two of the multiple segmented images to export the behavior of the welding phenomenon. In the export process, as a behavior of the welding phenomenon, the index values ​​of at least one of sputtering and fumes are exported. In the export process, when exporting the index values ​​of the flue gas, Edge detection is performed on the segmented image after removing the sputtered regions. Based on the detected edges, the system is segmented into multiple regions. The index values ​​of the flue gas are calculated based on the area of ​​the multiple regions.

Images