Spatter detection device

Abstract

To shorten a time required in detecting a spatter.SOLUTION: An image processing part is made to execute: a detection target region identifying process for identifying some detection target region including a variable pixel in which a pixel value indicating brightness has a difference above a predetermined threshold in comparison with a pixel value of a corresponding pixel in a photographed image photographed at a timing different from a timing when a photographed image which is detected for a spatter; and a spatter detecting process for detecting an image in the detection target region identified by the detection target region identifying process, for a spatter, using a machine learning model.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a spatter detection device that detects spatter using a machine learning model on an image captured of a spatter scattering area. [Background technology]

[0002] Patent Document 1 discloses a sputter detection device that detects sputters by using a machine learning model on an image captured of a sputter scattering area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7165924 Summary of the Invention [Problem to be solved by the invention]

[0004] In a spatter detection device such as that disclosed in Patent Document 1, if spatter detection is performed using the entire captured image as input to a machine learning model, the time required for spatter detection increases.

[0005] The present disclosure has been made in consideration of the above-mentioned points, and an object of the present disclosure is to reduce the time required for detecting sputters. [Means for solving the problem]

[0006] In one aspect of the present disclosure, a spatter detection device detects spatters using a machine learning model for an image captured of a spatter scattering area, and is characterized in that it includes an image processing unit that performs a detection target area identification process that identifies a portion of the entire captured image in which the spatter is to be detected, the portion including changed pixels whose pixel values ​​indicating brightness have a difference of a predetermined threshold or more from the pixel values ​​of corresponding pixels in an captured image captured at a different time from the captured image, and a spatter detection process that detects spatters using a machine learning model for the image of the detection target area identified by the detection target area identification process.

[0007] According to this aspect, spatter detection is performed using a machine learning model on an image of a partial region of the captured image, thereby reducing the time required to detect spatters compared to when the entire captured image is used as input to the machine learning model to detect spatters. Effect of the Invention

[0008] According to the present disclosure, the time required for detecting sputters can be reduced. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a schematic configuration of a welding system including a computer as a spatter detection device according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is an explanatory diagram showing spatters generated during arc welding. [Diagram 3] 5 is a flowchart showing a procedure for setting welding conditions using a computer as a spatter detection device according to an embodiment of the present disclosure. [Figure 4] FIG. 11 is an explanatory diagram illustrating a procedure for calculating a difference between pixel values. [Diagram 5] FIG. 2 is an explanatory diagram illustrating a binarized image. [Figure 6] 11 is an explanatory diagram illustrating an example of a captured image in which a detection target area is surrounded by a frame. FIG. [Figure 7] 10 is an explanatory diagram illustrating an example of an image of a cut-out detection target region. FIG. [Figure 8] FIG. 11 is an explanatory diagram illustrating an example of a processed image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its application, or its uses.

[0011] 1 shows a welding system 100. This welding system 100 includes a welding robot 110, a video camera 120, a memory card 130 accommodated in the video camera 120, a computer 140 as a spatter detection device according to an embodiment of the present disclosure, and a card reader 150 connected to the computer 140.

[0012] 2, welding robot 110 includes welding torch 111 capable of holding welding wire 160, and performs arc welding by applying a voltage between workpiece 170 held by a welding jig (clamp) and welding wire 160 as an electrode held by welding torch 111 to generate arc A between workpiece 170 and welding wire 160. During arc welding, the portion to be welded of workpiece 170 melts to form molten pool 171, and spatter SP scatters from molten pool 171. Note that a nozzle hole (not shown) for spraying shielding gas is provided at the tip of welding torch 111.

[0013] The video camera 120 is installed in a position where it can capture the entire scattering area of ​​the sputter SP, including the entire workpiece 170, through an ND (Neutral Density) filter (not shown), and the captured video is stored in the memory card 130. The frame rate (shooting speed) of the video camera 120 is set to 60 fps. The focus, aperture, and shutter speed of the electronic shutter of the video camera 120 are fixed.

[0014] The computer 140 includes a computer main body 141 and a display 142 as a display unit. The computer main body 141 includes a storage unit 141a and an image processing unit 141b.

[0015] The storage unit 141a of the computer main body 141 stores parameters that specify a machine learning model. This machine learning model is generated by supervised learning using multiple images that include spatter SP and multiple images that do not include spatter SP as training data. For example, deep learning is used as a method of supervised learning for generating a machine learning model. In this embodiment, in order to generate the machine learning model, about 2000 images that include spatter SP and about 1000 images that do not include spatter SP are used as training data. Note that the number of images that make up the training data is not limited to this number. The storage unit 141a further stores a video captured by the video camera 120 and still images (frames) obtained by dividing the video.

[0016] The image processing unit 141b of the computer main body 141 reads out the video stored in the memory card 130 inserted in the card reader 150, and stores it in the storage unit 141a. The image processing unit 141b also divides the video stored in the storage unit 141a into frames, that is, photographed images that are still images, and stores them in the storage unit 141a. Furthermore, the image processing unit 141b specifies a detection target area from the entire photographed image that is the detection target of spatter SP based on the multiple photographed images stored in the storage unit 141a, detects spatter SP in the image of the detection target area, and specifies the number of detected spatter SP (detection target area specifying process, spatter detection process, spatter number specifying process). The detection of spatter SP is performed using a machine learning model specified by parameters stored in the storage unit 141a. The image processing unit 141b also generates a processed image I (see FIG. 8) by performing a predetermined process on the photographed image that is the detection target of spatter SP. The method of specifying the detection target region, the method of detecting spatters SP, the method of specifying the number of spatters SP, and the method of generating processed image I will be described in detail later.

[0017] The display 142 displays the processed image I generated by the image processing unit 141 b of the computer main body 141 .

[0018] A procedure for setting welding conditions for welding system 100 will be described below with reference to FIG.

[0019] First, in (S101), while the user is causing welding robot 110 to perform arc welding, the user causes video camera 120 to take a picture and to store the taken video in memory card 130. As a result, a video of the entire scattering area of ​​spatter SP including entire workpiece 170 during arc welding is stored in memory card 130. The length of the video is set to, for example, 8 seconds, the image size to, for example, 1080*1920 pixels, and the frame rate to, for example, 30 fps / s. Note that the length of the video, the image size, and the frame rate may be set to other values.

[0020] Next, in (S102), the user removes memory card 130 from video camera 120 and inserts it into card reader 150, and transfers the video stored in memory card 130 from card reader 150 to computer main body 141. Then, image processing unit 141b of computer main body 141 receives the video transferred from card reader 150 and stores it in storage unit 141a.

[0021] Next, the image processing unit 141b of the computer main body 141 executes a detection target area specification process in the following steps (S103) to (S107) to specify a part of an area of ​​the entire captured image that is a detection target for spatter SP as a detection target area OR. The detection target area OR is an area including changed pixels whose pixel values ​​indicating brightness have a difference of a predetermined threshold or more with respect to the pixel values ​​of corresponding pixels in a captured image captured at a timing different from that of the captured image that is a detection target for spatter SP. In this embodiment, the first frame of the video stored in the memory card 130 is used as the captured image captured at a timing different from that of the captured image that is a detection target for spatter SP.

[0022] In (S103), the image processing unit 141b of the computer main body 141 converts the first frame (photographed image) of the video during arc welding stored in the storage unit 141a into a grayscale image. Here, the pixel value of the grayscale image indicates brightness in 8 bits, with the pixel value indicating black being 0 and the pixel value indicating white being 255. Hereinafter, the grayscale image of the first frame is referred to as a first image FFL (see FIG. 4).

[0023] Next, in (S104), the image processor 141b converts the frame (captured image) in which the spatter SP is detected into a grayscale image. The grayscale image of the frame in which the spatter SP is detected is set as a target image OFL (see FIG. 4).

[0024] Then, in (S105), as shown in Fig. 4, the image processor 141b performs a process on the target image OFL obtained in (S104) by subtracting the pixel value of the corresponding pixel in the first image FFL from the pixel value of each pixel, thereby obtaining a difference image DFL. The pixel value of each pixel in the difference image DFL is the difference between the pixel values ​​indicating the brightness of each pixel between the frame in which spatter SP is detected and the first frame. In more detail, the pixel value of each pixel in the difference image DFL is the absolute value of the value obtained by subtracting the pixel value of the corresponding pixel in the first image FFL from the pixel value of each pixel in the target image OFL.

[0025] 4, the reference character BR indicates a background object region, the reference character NS indicates a region corresponding to a noise component, and the reference character MV indicates a moving object region. In the difference image DFL, the pixel value of the background object region is set to 0.

[0026] Next, in (S106), the image processing unit 141b converts the difference image DFL obtained in (S105) into a binarized image BFL as shown in Fig. 5, for example. In detail, the pixel values ​​of the pixels in the difference image DFL whose pixel values ​​are less than a predetermined threshold are converted to a value (0) indicating black, and the pixel values ​​of the pixels in the difference image DFL whose pixel values ​​are equal to or greater than the predetermined threshold are converted to a value (255) indicating white. In the example of Fig. 4, the pixel values ​​of the area NS corresponding to the noise components can be set to 0 by setting the reference value to a value exceeding 20.

[0027] Next, in (S107), the image processing unit 141b specifies a continuous rectangular extraction region ER (region surrounded by a thick line in FIG. 5) that includes a continuous changed region CR (white region in FIG. 5) of a predetermined size or more composed of changed pixels with a pixel value of 255 in the binarized image BFL obtained in (S106). Here, "above a predetermined size" means, for example, that the number of pixels PN1 in the vertical direction is a first predetermined number or more and the number of pixels PN2 in the horizontal direction is a second predetermined number or more. When at least one extraction region ER is specified, all of the specified extraction regions ER constitute the detection target region OR. The predetermined size may be set to another size.

[0028] FIG. 6 illustrates an example of a captured image of a detection target in a state in which the detection target area OR is surrounded by a frame.

[0029] Next, in (S108), the image processing unit 141b cuts out images of all the extraction regions ER identified in (S107), that is, the detection target regions OR, as shown in Fig. 7. In Fig. 7, the images of the extraction regions ER cut out in (S108) are indicated by symbols ER1 to ER3.

[0030] Next, in (S109), the image processing unit 141b executes a spatter detection process for detecting spatters SP using a machine learning model for all images of the extraction regions ER identified in (S107), i.e., the images of the detection target regions OR. In more detail, the image processing unit 141b determines the presence or absence of spatters SP for the images of each extraction region ER cut out in (S108). Then, a spatter list List is created that identifies all detected spatters SP by their coordinates.

[0031] Next, in (S110), the image processing unit 141b executes a sputter number determination process for determining the number of sputters SP specified in the sputter list List created in (S109) as the number of detected sputters SP.

[0032] Next, in (S111), the image processing unit 141b judges whether or not there are any images (frames) remaining in the storage unit 141a that have not yet been subjected to the processes in (S104) to (S110). If there are, the process returns to (S104), whereas if there are no images remaining, the process proceeds to (S112).

[0033] In (S112), the image processing unit 141b processes each frame (photographed image) of the video during arc welding stored in the storage unit 141a by adding a frame number and the number of spatters SP identified in (S110) to the upper left corner of the image, and by surrounding the spatters SP identified in the spatter list List with a rectangular frame F, thereby generating a processed image I as shown in FIG. 8. Then, a video is generated by combining the processed images I for all the generated frames. In FIG. 8, A' indicates an area in which the light of the arc A itself, the workpiece 170 illuminated by the light of the arc A, the welding jig, and the fumes are captured.

[0034] In (S113), the display 142 displays any of the processed images I or videos generated in (S112). By referring to the number of spatter displayed on the display 142, the user determines the suitability of welding conditions such as the voltage value applied between the workpiece 170 and the welding wire 160. When the user determines that the welding conditions are inappropriate, the user changes the welding conditions so as to reduce the number of spatter SP, and executes the processes of (S101) to (S113) again.

[0035] Therefore, according to the present embodiment, in (S109), since the image processing unit 141b detects the spatter SP using the machine learning model for the image of the detection target region OR which is a part of the captured image, the time required for detecting the spatter SP can be shortened as compared with the case where the spatter SP is detected using the entire captured image as the input of the machine learning model.

[0036] Incidentally, as a method for improving the detection accuracy of the spatter SP, it is conceivable to increase the number of image sheets of the teacher data used for generating the machine learning model. However, increasing the number of image sheets of the teacher data increases the man-hours and costs for collecting the teacher data, and also increases the learning time. Further, the pixel value of the region corresponding to the noise component becomes equal to or higher than the reference value, and even when the binarization process in (S106) is performed, noise components may remain in the binarized image BFL. In the present embodiment, even if there is a continuous region composed of changing pixels in the image of the detection target of the spatter SP, if the region is smaller than a predetermined size, it is not regarded as a detection target of the spatter SP by the machine learning model. Therefore, it is possible to reduce the false detection of erroneously detecting a noise component that is not a spatter SP as a spatter SP. Therefore, it is possible to improve the detection accuracy of the spatter SP without increasing the number of image sheets of the teacher data used for generating the machine learning model.

[0037] In the present embodiment, in steps (S103) and (S104), the image processing unit 141b temporarily converts the first frame and the frame to be the detection target of the spatter SP into grayscale images. However, pixels whose difference in pixel values indicating brightness exceeds a predetermined threshold may be specified as changing pixels directly from the color image before conversion.

[0038] Also, in the present embodiment, the pixel values of the difference image DFL are the differences in pixel values indicating brightness between the frame to be the detection target of the spatter SP and the first frame. However, in order to obtain the difference image DFL, the frame for calculating the difference in pixel values from the frame to be the detection target of the spatter SP may be a frame (captured image) captured at a timing different from that of the frame to be the detection target of the spatter SP, and may be a frame other than the first frame.

[0039] Also, in the present embodiment, the detection target region OR is configured by one extraction region ER. However, only the change region CR may be used as the detection target region OR.

[0040] Also, in the present embodiment, the input image is processed by adding a frame F to the detected spatter SP to obtain the processed image I. However, the processed image I may be obtained by performing a process of adding a mark other than the frame F.

[0041] Also, in the present embodiment, the image processing unit 141b of the computer main body 141 receives a moving image including the input image from the card reader 150. However, it may be received from another information transmission device.

[0042] Also, in the present embodiment, the present invention is applied to arc welding using the welding robot 110. However, the present invention can also be applied when the operation of the welding torch is performed manually.

[0043] Also, in the present embodiment, the present invention is applied to a moving image of the spatter SP scattering region captured during arc welding. However, the present invention can also be applied to moving images and still images of the spatter SP scattering region captured during laser processing such as laser welding and laser cutting. [Industrial Applicability]

[0044] The spatter detection device disclosed herein can reduce the time required to detect spatter, and is useful as a spatter detection device that detects spatter using a machine learning model on images taken of the spatter scattering area during welding. [Explanation of symbols]

[0045] 140 Computer (spatter detection device) 141b Image processing section CR change area OR detection target area ER extraction area SP Sputter

Claims

1. A spatter detection device that detects spatter using a machine learning model on an image captured of a spatter scattering area, The sputter detection device is characterized by comprising an image processing unit that executes a detection target area identification process that identifies a portion of a detection target area including changed pixels whose pixel values ​​indicating brightness have a difference of a predetermined threshold or more from the pixel values ​​of corresponding pixels in a captured image captured at a different time from the captured image, and a sputter detection process that detects sputters using a machine learning model on the image of the detection target area identified by the detection target area identification process.

2. 2. The sputter detection device according to claim 1, The sputter detection device is characterized in that the detection target area includes a continuous changed area of ​​a predetermined size or more made up of the changed pixels, and is composed of at least one continuous extracted area.

3. 2. The sputter detection device according to claim 1, The sputter detection device according to claim 1, wherein the image processing unit further executes a sputter number determination process for determining the number of sputters detected by the sputter detection process.

Images