System for automatic welding line tracking using machine learning and method for performing automatic welding line tracking using the same

Abstract

One embodiment of the present invention provides a technology for rapidly and precisely tracking and utilizing the position of a weld line by irradiating a line laser onto a weld groove to obtain an image, processing the image obtained through image processing, and performing machine learning using the image obtained through image processing. An automatic weld line tracking system using machine learning according to an embodiment of the present invention comprises: a laser image acquisition device that obtains a plurality of laser images displaying laser sprites by irradiating a line laser onto a weld groove between a first base material and a second base material while moving; and an information processing device that generates a thinning image displaying a thinning line obtained by performing image processing on the laser images, performs machine learning on a training image displaying a training line obtained by performing image processing including filtering on the thinning image, and calculates a weld center line, which is the center line of the weld groove, using the result data of the machine learning.

Description

Technology Field

[0001] The present invention relates to an automatic weld line tracking system using machine learning and an automatic weld line tracking method using the same. More specifically, it relates to a technology for rapidly and precisely tracking and utilizing the position of a weld line by irradiating a line laser onto a weld groove to obtain an image, processing the image obtained through image processing, and performing machine learning using the image obtained through image processing. Background Technology

[0003] In order to perform precise automatic welding along the welding grooves of various welded parts, such as welded parts of hull blocks, conventional technology is configured such that a rail is installed along the side of the welding line, and a welding carriage is seated on the rail and travels along the rail to perform automatic welding.

[0004] However, when performing automatic welding by installing rails along the welding line in this manner, there is a difficulty in that the operator must precisely install the rails along the welding line. Since it is desirable for the welding torch to move while aligned with the center of the welding line and the allowable displacement tolerance of the welding torch from the center of the welding line is typically ±1mm, there is a problem in that the operator must precisely install the rails along the welding line within the aforementioned tolerance range, namely ±1mm.

[0005] Meanwhile, in addition to a rail-type weld line tracking device that moves along installed rails, a rail-less weld line tracking device that travels along the weld line has also been developed.

[0006] Rail-free welding line tracking devices are generally not equipped with a steering mechanism due to equipment size limitations, or if steering is required, the operator must directly control the direction.

[0007] Therefore, a rail-less weld line tracking device without a steering mechanism can only move in a straight line using parallel wheels. In this case, if the direction of extension of the weld line changes, such as with a curved weld line, the welding torch mounted in the center of the welding carriage cannot move along the weld line, causing it to deviate from the weld line and resulting in a lack of precise control.

[0008] In order to solve the aforementioned problems, research and development have recently been conducted on methods to acquire images of the weld area using vision sensors, lidar, etc., and to derive weld lines by performing data analysis thereon. However, there are limitations in implementing precise weld line tracking through detailed processing steps by using methods such as simply detecting and analyzing objects through 3D image analysis to track weld lines. Prior art literature

[0010] Republic of Korea Registered Patent No. 10-2424501 The problem to be solved

[0011] The objective of the present invention, which is to solve the above-mentioned problems, is to provide a technology that rapidly and precisely tracks and utilizes the position of a weld line by irradiating a weld groove with a line laser to obtain an image, performing image processing on the image obtained through image processing, and performing machine learning using the image obtained through image processing.

[0012] The technical problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which the present invention belongs from the description below. means of solving the problem

[0014] The configuration of the present invention for achieving the above-mentioned purpose comprises: a laser image acquisition device that acquires a plurality of laser images displaying laser sprites by irradiating a line laser onto a welding groove between a first base material and a second base material while moving; and an information processing device that generates a thinning image displaying a thinning line, which is a line obtained by performing image processing on the laser images, performs machine learning on a training image displaying a training line, which is a line obtained by performing image processing including filtering on the thinning image, and calculates a welding centerline, which is a centerline of the welding groove, using the result data of the machine learning.

[0015] The configuration of the present invention for achieving the above-mentioned purpose comprises: a laser image acquisition step in which the laser image acquisition device irradiates a line laser onto the welding groove and acquires a plurality of laser images; a binarization processing step in which the binarization processing unit receives the laser images from the laser image acquisition device and performs binarization on the laser images to generate a binarized image; a thinning processing step in which the thinning processing unit receives the binarized image from the binarization processing unit and generates the thinning image by deriving the thinning lines using a sub-window for searching in the binarized image; and a filtering correction step in which the filtering correction unit receives the binarized image from the binarization processing unit and receives the thinning image from the thinning processing unit, derives a noise region from the binarized image, performs filtering on the thinning image, and corrects the filtered thinning lines to generate the training image. and the machine learning unit includes a machine learning step in which it receives the training image from the filtering correction unit and performs machine learning to extract at least one feature point from the training line. Effects of the invention

[0017] The effect of the present invention according to the above configuration is that, since the weld center island can be tracked with a simple configuration using a camera and a line laser, the position of the weld line can be tracked quickly while simultaneously improving space and cost efficiency.

[0018] Furthermore, the effect of the present invention is that by processing an image and performing machine learning using the processed image, the position of the weld centerline can be measured, thereby enabling precise tracking of the weld line and improving the efficiency of automatic welding.

[0019] The effects of the present invention are not limited to the effects described above, and should be understood to include all effects that can be inferred from the configuration of the invention described in the detailed description of the invention or the claims. Brief explanation of the drawing

[0021] FIG. 1 is a schematic diagram of the configuration of a system according to one embodiment of the present invention. FIG. 2 is a block diagram related to an information processing device and a robot control device according to an embodiment of the present invention. FIG. 3 is an image obtained using a system according to one embodiment of the present invention. Figure 4 is an image of the movement of a sub-window in a thinning processing unit according to one embodiment of the present invention. FIG. 5 is a thinning image according to one embodiment of the present invention. FIG. 6 is an image related to the performance of filtering in a filtering correction unit according to one embodiment of the present invention. Figure 7 is an image related to feature point extraction in a machine learning unit according to one embodiment of the present invention. FIGS. 8 and FIGS. 9 are images related to the implementation of feature point extraction in a machine learning unit according to an embodiment of the present invention. FIG. 10 is a flowchart of a method according to one embodiment of the present invention. Specific details for implementing the invention

[0022] The present invention will be described below with reference to the attached drawings. However, the present invention may be implemented in various different forms and is therefore not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals.

[0023] Throughout the specification, when it is stated that a part is "connected (connected, in contact, combined)" with another part, this includes not only cases where they are "directly connected," but also cases where they are "indirectly connected" with other members interposed between them. Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but rather allows for the inclusion of additional components.

[0024] The terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0025] Hereinafter, the present invention will be described in detail with reference to the attached drawings.

[0027] FIG. 1 is a schematic diagram of the configuration of a system according to one embodiment of the present invention, and FIG. 2 is a block diagram related to an information processing device (200) and a robot control device (310) according to one embodiment of the present invention.

[0028] FIG. 3 is an image obtained using a system according to an embodiment of the present invention. Here, FIG. 3 (a) is a laser image (410), FIG. 3 (b) is a binarized image (420), FIG. 3 (c) is a thinned image (430) before filtering, and FIG. 3 (d) is a thinned image (430) after filtering.

[0029] As shown in FIGS. 1 to 3, the system of the present invention includes a laser image acquisition device; and an information processing device (200).

[0030] The laser image acquisition device can acquire multiple laser images (410) with laser sprites displayed by irradiating a line laser onto the welding groove (20) between the first base material (11) and the second base material (12) while moving.

[0031] To this end, the laser image collecting device may be equipped with a laser unit (120); and an imaging unit (110).

[0032] The laser unit (120) can irradiate a line laser toward the welding groove (20). Specifically, the laser unit (120) can irradiate a line laser toward a first base material (11) and a second base material (12) in contact with the first base material (11), and at this time, the length direction of the laser stripe (411) generated can be formed perpendicular to the length direction of the welding groove (20).

[0033] And, the irradiation angle of the line laser is tilted with respect to the surfaces of the first base material (11) and the second base material (12), so that the line laser can be emitted obliquely with respect to the welding groove (20). At this time, the irradiation angle of the line laser may be the angle between the line connecting the center point of the laser light source and the center point of the line stripe and the surfaces of the base materials.

[0034] Accordingly, when a line laser is simultaneously irradiated onto the first base material (11), the second base material (12), and the welding groove (20), as seen in FIG. 3 (a), the line laser is captured in a straight line on the surface of each base material, and in the welding groove (20), it can be captured in a V-shape according to the V-shaped cross-sectional shape of the welding groove (20).

[0035] In addition, by adjusting the irradiation angle of the line laser as described above and adjusting the emission tilt of the line laser, the thickness of the laser stripe (411) can also be adjusted. By adjusting the thickness in this way, an angle at which the laser stripe (411) is clearly displayed in the laser image (410) can be selected.

[0036] The imaging unit (110) can perform imaging of the area on the surface of the first base material (11) and the second base material (12) where a line laser is irradiated. In order to immediately capture the shape of the laser stripe (411) as the laser unit (120) moves as described above, the imaging unit (110) may be formed by combining with the laser unit (120). At this time, the imaging unit (110) may include a camera, a vision sensor, etc.

[0037] In addition, a mobile robot that moves such a laser image acquisition device may be formed. However, the mobile robot may be formed separately in this manner, or the laser image acquisition device may be installed on the welding robot described below.

[0039] The information processing device (200) generates a thinning image (430) in which a thinning line (431), which is a line obtained by performing image processing on a laser image (410), is displayed, and performs machine learning on a learning image (450) in which a learning line (451), which is a line obtained by performing image processing including filtering on the thinning image (430), is displayed, and can calculate a welding center line, which is a center line of a welding groove (20), using the result data of machine learning.

[0040] To this end, as shown in FIGS. 2 and 3, the information processing device (200) may include a binarization processing unit (210); a thinning processing unit (220); a filtering correction unit (230); a machine learning unit (240); and a centerline calculation unit (250). Here, each of the binarization processing unit (210), the thinning processing unit (220), the filtering correction unit (230), the machine learning unit (240), and the centerline calculation unit (250) may be a device or apparatus that performs calculations, such as a computer, a processor, etc.

[0041] The binarization processing unit (210) receives a laser image (410) from a laser image acquisition device and performs binarization on the laser image (410) to generate a binarized image (420). Through binarization processing in the binarization processing unit (210), the laser image (410) can be binarized into a black and white image, with bright parts having a value of 255 and dark parts having a value of 0. Since such an image binarization algorithm is a prior art, a detailed description will be omitted.

[0042] FIG. 4 is an image of the movement of a sub-window (510) in a thinning processing unit (220) according to one embodiment of the present invention, and FIG. 5 is an image of a thinning image (430) according to one embodiment of the present invention.

[0043] As seen in FIG. 3(b) and FIG. 4 and FIG. 5, the thinning processing unit (220) receives a binarized image (420) from the binarized processing unit (210) and can generate a thinning image (430) by deriving a thinning line (431) using a sub-window (510) for searching in the binarized image (420).

[0044] At this time, the thinning processing unit (220) can derive a thinning line (431) by using the brightness of the sub-window (510) area that changes according to the movement of the sub-window (510) in the binarized image (420).

[0045] Specifically, the thinning processing unit (220) can set pixels arranged in one column direction in the binarized image (420) as one pixel set (520), and accordingly, a plurality of pixel sets (520) can be formed in the binarized image (420) according to a plurality of columns.

[0046] And, an area corresponding to some of the pixels arranged in a single column direction can be set as a sub-window (510). That is, the sub-window (510) may be a movable window having an area corresponding to an area of ​​3 to 5 pixels arranged continuously in a column direction.

[0047] However, the area of ​​the sub-window (510) is not limited to this, and the number of pixels located within the area corresponding to the area of ​​the sub-window (510) can be changed according to the settings.

[0048] As described above, the sub-window (510) may be an auxiliary array of pixels used for searching for a binarized laser stripe (411) in each column, and the height (t) of such sub-window (510) may be set to be equal to the thickness of the laser stripe (411).

[0049] By the operation of the thinning processing unit (220), the sub-window (510) moves from the top to the bottom of a pixel set (520), and the thinning processing unit (220) can search for pixels located in the area of ​​the sub-window (510) according to the movement of the sub-window (510). Pixels located in the area of ​​the sub-window (510) can form a pixel combination.

[0050] Among the discovered pixel combinations, the position of each pixel of the pixel combination with the largest sum of the pixel brightness, i.e., the row position (row number), can be derived. At this time, multiple row positions may be derived depending on the area size of the sub-window (510), and among the multiple row positions (numbers), the middle position, which is the middle position in the corresponding pixel combination, may be the position of the stripe center pixel, which is a pixel included in the center line of the laser stripe (411) in the corresponding pixel set (520).

[0051] And, in the thinning processing unit (220), the following [Equation 1] and [Equation 2] can be used to derive the position of the stripe center pixel included in the center line of the laser stripe (411) in a pixel set (520), which is a set of pixels arranged in one column direction.

[0052] [Mathematical Formula 1]

[0053]

[0054] Here, S(j, k) represents the sum of brightness values ​​of pixels (pixel combinations) within a subwindow (510) in which the topmost pixel is located in row k of the j-th column, I(i, j) represents the brightness value of the pixel in row j of the i-th column, and t represents the thickness (number of pixels) of the laser stripe (411).

[0055] [Mathematical Formula 2]

[0056]

[0057] Here, atgmax k S(j, k) represents the position of the top row (first row k) among the pixels within the corresponding subwindow (510) when the sum of the brightnesses of the pixel combinations, which are pixels within the subwindow (510) in each column, is maximized, and t represents the thickness (number of pixels) of the laser stripe (411), and k max (j) represents the centerline position of the laser stripe (411) in the j-th column of the corresponding column (j), that is, the position of the stripe center pixel.

[0058] In [Mathematical Equation 2], as above, atgmax k Since S(j, k) refers to the first row of the subwindow (510), t / 2 is added for the accurate thin line position (center line position of the laser stripe (411)).

[0059] And, as shown in [Mathematical Formula 3] below, a thinning line (431) can be obtained by combining the stripe center pixels obtained in each column.

[0060] [Mathematical Formula 3]

[0061]

[0062] Here, the total size of matrix C is equal to the total size of the columns.

[0063] By the thinning processing unit (220) performing the above processing, a thinning image (430) with thinning lines (431) displayed as in Fig. 3 (c) and Fig. 5 can be obtained. However, as seen in the thinning image (430), distortion by noise can be observed.

[0065] FIG. 6 is an image related to the performance of filtering in a filtering correction unit (230) according to one embodiment of the present invention.

[0066] As shown in FIG. 6, the filtering correction unit (230) receives a binarized image (420) from the binarization processing unit (210) and a thinning image (430) from the thinning processing unit (220), derives a noise area (421) from the binarized image (420), and performs filtering by removing a portion of the thinning line (431) located in the noise area (421) from the thinning image (430) to generate a filtered image (440).

[0067] To this end, the filtering correction unit (230) sets pixels arranged in one column direction in the binarized image (420) into one pixel set (520), calculates the pixel brightness sum value for each of the multiple pixel sets (520), and can derive a noise area (421) using the pixel brightness sum value.

[0068] As shown in FIG. 6, the filtering correction unit (230) can set a pixel set (520) for each column as performed in the thinning processing unit (220), and after such setting, the filtering correction unit (230) can sum the brightness values ​​of each pixel within one pixel set (520).

[0069] And, the filtering correction unit (230) can determine whether the area of ​​the pixel set (520) is included in the noise area (421) by calculating whether the sum of the brightness values ​​in the pixel set (520) exceeds a threshold value based on the standard brightness value.

[0070] Specifically, the sum of the brightness values ​​of a set of pixels (520), that is, pixels located in one column, can be calculated by the following [Equation 4].

[0071] [Mathematical Formula 4]

[0072]

[0073] Here, S(j) represents the sum of the brightness values ​​of the pixels included in the j-th column of the binarized image (420), I(i, j) represents the brightness value of the pixel in the j-th row of the i-th column, t represents the thickness (number of pixels) of the laser stripe (411), and h represents the height (number of pixels in the j-th column) of the binarized image (420).

[0074] In addition, the standard brightness value can be calculated by the following [Equation 5]. Here, the standard brightness value can be used as a reference value for determining the laser stripe (411).

[0075] [Mathematical Formula 5]

[0076]

[0077] Here, S l represents the standard brightness value, and I max represents a preset maximum pixel brightness value, and t represents the thickness (number of pixels) of the laser stripe (411). That is, I by the thickness (t) of the laser stripe (411) appearing in the binarized image (420). max You can perform operations by multiplying by.

[0078] In addition, using the standard brightness values ​​as described above, whether one pixel set (520) is included in the noise area (421) can be determined using the following [Equation 6].

[0079] [Mathematical Formula 6]

[0080]

[0081] Here, R(j) represents a matrix of 0 or 1 values ​​assigned to each pixel in the j-th column, S(j) represents the sum of the brightness values ​​of the pixels included in the j-th column of the binarized image (420), and S l represents the standard brightness value, and T low Wow T high Each refers to a coefficient value arbitrarily set to calculate the respective thresholds of the minimum and maximum values.

[0082] If the number of columns in the binarized image (420), that is, the number of pixel sets (520), is 1000, then R(j) can also be 1000 (j=0~999).

[0083] [Mathematical Equation 6] is a formula for determining whether a column exists within a certain threshold range based on standard brightness values, and the threshold (S l *T low , S l *T high Each of the ) is a coefficient value (T low Wow T high ) standard brightness value (Sl It can be set by multiplying by ).

[0084] As described above, in each column of the pixel set (520) by the operation of [Mathematical Formula 6], the column determined to be a laser stripe (411) may have a value of 1, and the column determined to be a noise area (421) may have a value of 0.

[0085] And, a logical multiplication expression for removing the image of the pixel set (520) area in the thinning image (430) corresponding to the pixel set (520) in which the column of the pixel set (520) to be calculated is included in the noise area (421) can be formed as [Equation 7] below.

[0086] That is, the filtering correction unit (230) can determine whether each column of each pixel set (520) is a noise area (421) as described above, and any one pixel set (520) in the filtering correction unit (230) can be matched with any one pixel set (520) of the thinning processing unit (220), and using a logical multiplication expression based on this, filtering in the thinning image (430), that is, a process of removing the part corresponding to the noise area (421) among the thinning lines (431) can be performed.

[0087] [Mathematical Formula 7]

[0088]

[0089] Here, C(j) represents a matrix of pixel arrays in the j-th column (j-th pixel set (520)) of the thinned image (430), R(j) represents a matrix of 0 or 1 values ​​assigned to each pixel in the j-th column, and L(j) represents a matrix of pixel arrays in the j-th column (j-th pixel set (520)) of the thinned image (430) after the noise area (421) has been removed.

[0090] The change in the value assigned to each pixel by operating on each pixel by the logical multiplication formula according to [Mathematical Formula 7] is as follows in a specific embodiment.

[0091]

[0092] As described above, the filtering correction unit (230) can determine the pixel sets (520) corresponding to the noise area (421) using [Equation 4] to [Equation 6]. Then, by using the positions of the pixel sets (520) corresponding to the noise area (421), the noise portion of the thinning line (431) corresponding to the noise area (421) in the thinning image (430) can be removed.

[0093] The filtering correction unit (230) can generate a training image (450) by performing a correction that increases the thickness of the filtered thinning line (441) in the filtered image (440) to derive a training line (451).

[0094] When performing machine learning using thinning lines (431), since the thickness of the thinning lines (431) is relatively thin, an error may occur in the thinning recognition in the machine learning unit (240), so the filtering correction unit (230) can perform a correction to increase the thickness of the thinning lines (431) after filtering displayed on the filtered image (440).

[0095] At this time, in order to make the thickness of the thinning line (431) similar to the thickness (t) of the laser stripe (411), the thickness of the thinning line (431) can be increased by increasing the thickness of the thinning line (431) in the column direction (up and down) by t / 2 respectively, and thereby a learning line (451) can be generated. Then, a learning image (450) with such a learning line (451) displayed can be generated and transmitted to the machine learning unit (240).

[0097] FIG. 7 is an image related to feature point extraction in a machine learning unit (240) according to an embodiment of the present invention. FIG. 8 and FIG. 9 are images related to the implementation of feature point extraction in a machine learning unit (240) according to an embodiment of the present invention.

[0098] Here, FIG. 8(a) is for feature point extraction in a training image (450) of the first embodiment, and FIG. 8(b) is for feature point extraction in a training image (450) of the second embodiment.

[0099] And, FIG. 9(a) is for feature point extraction in a training image (450) of the third embodiment, and FIG. 9(b) is for feature point extraction in a training image (450) of the fourth embodiment.

[0100] The machine learning unit (240) receives a training image (450) from the filtering correction unit (230) and performs machine learning to extract at least one feature point from the training line.

[0101] In the machine learning unit (240), a machine learning model is stored, which is a model that has been trained using multiple training images (450). To train such a machine learning model, as shown in FIG. 7, five keypoints are assigned to each training image (450), and multiple training images (450) with such keypoints assigned can be used as training data.

[0102] Five feature points can be set, including a left end point (610), a left corner point (620), a bottom corner point (630), a right corner point (640), and a right end point (650), and a skeleton connecting each feature point can be set.

[0103] In the learning line (451), the left end point (610) is a feature point located at the left end of the left straight line connected to the V-shape, the left corner point (620) is a feature point formed at the bent portion where the left straight line and the V-shape are connected, and the bottom corner point (630) may be a point formed at the lower bent portion of the V-shape. Also, the right corner point (640) is a feature point formed at the bent portion where the right straight line and the V-shape are connected, and the right end point (650) may be a feature point located at the right end of the right straight line connected to the V-shape.

[0104] Each feature point is sequentially assigned a number or character from left to right so that each feature point can be distinguished in the machine learning model. Then, training in the machine learning model can be performed using the training image (450) in which the feature points and skeleton are set as described above.

[0105] In order to form a machine learning model as described above, the YOLOv8-poses model algorithm may be used in the present invention. However, the model algorithm is not limited to this, and other types of kinematic algorithms capable of recognizing the bent parts of each corner may also be used.

[0106] And, when the training image (450) generated by the filtering correction unit (230) is transmitted to the machine learning unit (240), the machine learning unit (240) can perform analysis on the training image (450).

[0107] Specifically, as seen in FIGS. 8 and 9, the training line (451) in the training image (450) may include one or more lines, and a bounding box is set for each line portion so that each line portion can be detected as an object.

[0108] At this time, the left corner point (620), the bottom corner point (630), and the right corner point (640) are essential for tracking the weld groove (20), and if the left corner point (620), the bottom corner point (630), and the right corner point (640) are not present, only the left end point (610) and the right end point (650) on both sides may exist.

[0109] Therefore, the machine learning unit (240) can recognize only objects (objects) that have three or more feature points in each line portion of the training image (450) as target objects used for welding centerline tracking.

[0110] And, when the machine learning unit (240) detects the above target object in the training image (450), it can generate pixel coordinates, i.e., pixel positions corresponding to each feature point in the target object, and transmit them to the centerline calculation unit (250).

[0111] The centerline calculation unit (250) receives pixel coordinates of feature points on a training image (450) from the machine learning unit (240) and receives a filtered thinning image (430) from the filtering correction unit (230), and can derive the position of the welding centerline using the pixel coordinates of the feature points and the thinning image (430).

[0112] Specifically, the centerline calculation unit (250) receives pixel coordinates located midway between the left end point (610) and the right end point (650) in the row direction, and can derive the pixel at the corresponding pixel coordinates as a weld center pixel, which is a pixel located on the weld centerline.

[0113] Then, the centerline calculation unit (250) can match the weld center pixel to the filtered thinning line (441) in the filtered thinning image (430), and next, the centerline calculation unit (250) can derive the 3D (x, yz) coordinates of the weld center point on the weld center line corresponding to the weld center pixel using the filtered thinning image (430) and the weld center pixel.

[0114] That is, when a point on the work surface is used as a 3D coordinate reference point, the imaging unit (110) can generate the x and y coordinates of the imaging center point, i.e., the center point of the laser image (410), and transmit them to the centerline calculation unit (250).

[0115] In the centerline calculation unit (250), the center point x, y coordinates of the laser image (410) received as described above are set as the center point x, y coordinates of the filtered thinning image (430), and the x, y coordinates of the weld center pixel are calculated based on the center point x, y coordinates of the filtered thinning image (430), thereby deriving the x, y coordinates of the weld center point. Additionally, the z coordinate of the weld center point can be set to the depth of the previously stored weld groove (20).

[0116] Through this process, the 3D coordinates of each of the multiple welding center points can be derived, and the welding center line can be tracked by deriving the position of the welding center line by connecting the 3D coordinates of each of the multiple welding center points.

[0117] The system of the present invention may further include: a welding robot that operates such that a welding torch (320) mounted on an end performs welding along a welding centerline; and a robot control device (310) that transmits a control signal to the welding robot so that the welding torch (320) moves along the welding centerline.

[0118] The operation of the mobile robot described above can also be controlled by the control of the robot control device (310) as described above. In addition, the robot control device (310) can receive the position of the welding centerline as described above from the centerline calculation unit (250).

[0119] Also, the robot control device can generate a control signal to enable the welding torch (320) to perform welding along the welding centerline and transmit it to the welding robot.

[0121] FIG. 10 is a flowchart of a method according to one embodiment of the present invention.

[0122] As shown in FIG. 10, the method of the present invention comprises a laser image acquisition step (S10); a binarization processing step (S20); a thinning processing step (S30); a filtering correction step (S40); a machine learning step (S50); and a centerline calculation step (S60).

[0123] First, in the laser image acquisition step (S10), a laser image acquisition device can irradiate a line laser onto a welding groove (20) and acquire multiple laser images (410).

[0124] In the binarization processing step (S20), the binarization processing unit (210) receives a laser image (410) from a laser image acquisition device and performs binarization on the laser image (410) to generate a binarized image (420).

[0125] In the thinning processing step (S30), the thinning processing unit (220) receives the binarized image (420) from the binarized processing unit (210) and can generate a thinning image (430) by deriving a thinning line (431) using a sub-window (510) for searching in the binarized image (420).

[0126] In the filtering correction step (S40), the filtering correction unit (230) receives a binarized image (420) from the binarization processing unit (210) and a thinning image (430) from the thinning processing unit (220), derives a noise region (421) from the binarized image (420), filters the thinning image (430), and corrects the filtered thinning line (441) to generate a training image (450).

[0127] In the machine learning step (S50), the machine learning unit (240) receives a training image (450) from the filtering correction unit (230) and performs machine learning to extract at least one feature point from the training line.

[0128] In the centerline calculation step (S60), the centerline calculation unit (250) receives pixel coordinates of feature points on a training image (450) from the machine learning unit (240) and receives a filtered thinning image (430) from the filtering correction unit (230), and can derive the position of the welding centerline using the pixel coordinates of the feature points and the filtered thinning image (430).

[0129] The remaining details regarding the method of the present invention are the same as those described in the system of the present invention above.

[0130] When using the present invention as described above, the weld center island can be tracked with a simple configuration using a camera and a line laser, thereby allowing for rapid tracking of the weld line position while simultaneously improving space and cost efficiency.

[0131] In addition, by using the present invention, the position of the weld centerline can be measured by processing an image and performing machine learning using the processed image, thereby allowing for precise tracking of the weld line and improving the efficiency of automatic welding.

[0133] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0134] The scope of the present invention is defined by the claims set forth below, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0136] 11: 1st base material 12: 2nd base material 20: Welding groove 110: Imaging unit 120: Laser unit 200: Information processing device 210: Binarization Processing Unit 220: Thinning Processing Unit 230: Filtering Correction Unit 240: Machine Learning Unit 250: Centerline Calculation Unit 310: Robot Control Unit 320: Welding torch 410: Laser image 411: Laserstripe 420: Binarized image 421: Noise area 430: Thinned image 431: Thinned line 440: Filtered image 441: Filtered thinned line 450: Training image 451: Learning line 510: Sub-window 520: Pixel set 610: Left boundary point 620: Left corner point 630: Floor corner point 640: Right Corner Branch 650: Right Side Branch

Claims

Claim 1 A laser image acquisition device that acquires multiple laser images displaying laser sprites by irradiating a line laser onto a welding groove between a first base material and a second base material while moving; and an information processing device that generates a thinning image displaying a thinning line, which is a line obtained by performing image processing on the laser images, performs machine learning on a training image displaying a training line, which is a line obtained by performing image processing including filtering on the thinning image, and calculates a welding centerline, which is a centerline of the welding groove, using the result data of the machine learning; characterized by comprising: a welding line automatic tracking system using machine learning. Claim 2 A welding line automatic tracking system using machine learning according to claim 1, wherein the laser image acquisition device comprises: a laser unit that irradiates a line laser toward the welding groove; and an imaging unit that performs imaging of an area on the surface of the first base material and the second base material where the line laser is irradiated. Claim 3 An automatic welding line tracking system using machine learning according to claim 1, wherein the information processing device comprises a binarization processing unit that receives the laser image from the laser image collection device and performs binarization on the laser image to generate a binarized image. Claim 4 The automatic welding line tracking system using machine learning according to claim 3, wherein the information processing device further comprises a thinning processing unit that receives the binarized image from the binarization processing unit and generates the thinning image by deriving the thinning line using a sub-window for searching in the binarized image. Claim 5 A welding line automatic tracking system using machine learning according to claim 4, wherein the thinning processing unit derives the thinning line using the brightness of the sub-window area that changes according to the movement of the sub-window in the binarized image. Claim 6 The automatic welding line tracking system using machine learning according to claim 4, wherein the information processing device further comprises a filtering correction unit that receives the binarized image from the binarization processing unit and receives the thinning image from the thinning processing unit, derives a noise region from the binarized image, and generates a filtered image by performing filtering by removing a portion of the thinning line located in the noise region from the thinning image. Claim 7 A machine learning automatic welding line tracking system according to claim 6, wherein the filtering correction unit sets pixels arranged in one column direction in the binarized image as one pixel set, calculates the sum of pixel brightness values ​​for each of the plurality of pixel sets, and derives the noise region using the sum of pixel brightness values. Claim 8 The automatic welding line tracking system using machine learning according to claim 6, wherein the filtering correction unit performs a correction that increases the thickness of the filtered thinning line in the filtered image to derive the training line and generate a training image. Claim 9 The automatic welding line tracking system using machine learning according to claim 8, wherein the information processing device further comprises a machine learning unit that receives the training image from the filtering correction unit and performs machine learning to extract at least one feature point from the training line. Claim 10 The automatic weld line tracking system using machine learning according to claim 9, wherein the information processing device further comprises: a centerline calculation unit that receives pixel coordinates of the feature point on the training image from the machine learning unit, receives a filtered thinning image from the filtering correction unit, and derives the position of the weld centerline using the pixel coordinates of the feature point and the filtered thinning image. Claim 11 A welding line automatic tracking system using machine learning according to claim 1, further comprising: a welding robot that operates such that a welding torch mounted on an end performs welding along the welding centerline; and a robot control device that transmits a control signal to the welding robot to move the welding torch along the welding centerline. Claim 12 A method for automatically tracking a weld line using a machine learning-based weld line automatic tracking system according to claim 10, comprising: a laser image acquisition step in which a laser image acquisition device irradiates a line laser onto a weld groove and acquires a plurality of laser images; a binarization processing step in which a binarization processing unit receives the laser images from the laser image acquisition device and performs binarization on the laser images to generate a binarized image; a thinning processing step in which a thinning processing unit receives the binarized image from the binarization processing unit and generates the thinning image by deriving thinning lines using a sub-window for searching in the binarized image; and a filtering correction step in which a filtering correction unit receives the binarized image from the binarization processing unit and receives the thinning image from the thinning processing unit, derives a noise region from the binarized image to perform filtering on the thinning image, and corrects the filtered thinning lines to generate the training image. A method for automatically tracking a weld line, characterized by including: a machine learning step in which the machine learning unit receives the training image from the filtering correction unit and performs machine learning to extract at least one feature point from the training line. Claim 13 The automatic weld line tracking method according to claim 12, further comprising: a centerline calculation step in which the centerline calculation unit receives pixel coordinates of the feature point on the training image from the machine learning unit, receives the filtered thinning image from the filtering correction unit, and derives the position of the weld centerline using the pixel coordinates of the feature point and the filtered thinning image.

Images