A welding seam real-time identification algorithm based on a welding robot

Abstract

The application discloses a kind of welding seam real-time identification algorithm based on welding robot, including obtaining the local image of welding seam area of welding piece, image pre-processing, image gray scale and binarization processing;Mark the connected region of binarization image, according to the size of region, cut off, on this basis, extract contour by bwperim function, according to the size of contour, screening, after screening contour, contour fitting is carried out, after fitting, obtain welding gap oblique straight line, according to oblique straight line, region filling is carried out, and the final welding gap position and size are obtained.The application can lay the foundation for welding robot autonomous welding, improve welding efficiency;And can identify butt weld, lap weld, V-shaped weld and other various weld types;And the identification algorithm of the present application based on welding drawing comparison optimization welding seam image missing fusion sensor and image processing technology in welding process can realize high-precision welding seam real-time identification.

Description

Technical Field

[0001] This invention relates to a weld seam recognition algorithm, specifically a real-time weld seam recognition algorithm based on a welding robot, belonging to the field of weld seam recognition technology. Background Technology

[0002] With the development of my country's manufacturing industry, intelligent welding technology for welding robots has become a key research focus. Weld seam recognition and weld seam error compensation are crucial intelligent welding technologies for improving the autonomous welding capabilities of welding robots.

[0003] In the prior art, such as the weld identification method and welding system disclosed in CN117206758A, the method includes: acquiring at least one image of a weld area projected by a line light source to obtain at least one weld image; analyzing at least one weld image using at least one target model to obtain at least one analysis image, the analysis image including at least one of a first target frame and a second target frame, wherein the diagonal points of the first target frame are the left top inflection point and the left bottom inflection point of the weld bevel, and the diagonal points of the second target frame are the right top inflection point and the right bottom inflection point of the weld bevel, and the target model is trained by machine learning using multiple sets of data; determining the inflection point position of the weld bevel based on the at least one analysis image; and determining the position of the weld center point and the weld width of the weld area based on the inflection point position. Existing technologies have solved the problem of accurately identifying weld seams, but existing weld seam identification methods lack flexibility and diversity, and can only identify one type of weld seam. This results in problems such as limited weld seam identification, complex algorithms, and slow speed. On the other hand, existing weld seam identification mainly relies on visual detection, but it is easily affected by noise such as strong light and smoke during the welding process. Metal spatter, sparks, and bubbles during the welding process can also cause image occlusion and loss, resulting in low identification accuracy.

[0004] Therefore, weld seam recognition by welding robots is a challenging task that requires the use of effective sensor fusion, image processing, deep learning and other technologies to improve the accuracy of weld seam recognition during the welding process. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time weld seam recognition algorithm based on a welding robot in order to solve at least one of the above-mentioned technical problems.

[0006] This invention achieves the above objective through the following technical solution: a real-time weld seam recognition algorithm based on a welding robot, which includes the following steps:

[0007] Step 1: Obtain a local image of the weld seam area of ​​the welded part using image recognition technology, pick out the red, green and blue values ​​of each pixel in the image, and create a generalized matrix that contains only the color values ​​of all pixels;

[0008] Step 2: The image is preprocessed by first adding Gaussian noise and salt-and-pepper noise, and then reducing noise through mean filtering and median filtering. Then, local images are grayscaled and binarized.

[0009] Step 3: After filtering out image noise, re-extract the third dimension information from the 3D matrix, update the three variable values ​​of each element in the generalized matrix, and perform grayscale processing on the image.

[0010] Step 4: Binarize the grayscale image to maximize the discriminative power of local features, thereby accurately extracting the size and distribution characteristics of the weld seam area of ​​the welded component.

[0011] Step 5: Mark the connected regions of the binarized image. Use a traversal scanning algorithm to check the connectivity between the currently scanned pixel and the neighboring pixels of the previous scan. Use the eight-neighborhood method to mark the connected regions to identify weld gaps.

[0012] Step 6: Cut out the area according to its size. Then, extract the contour using the bwperim function, filter the contours according to their size, fit the filtered contours, obtain the oblique line of the weld gap after fitting, fill the area according to the oblique line, and obtain the final position and size of the weld gap.

[0013] As a further aspect of the present invention: In step one, the creation of the generalized matrix specifically includes:

[0014] An image is composed of pixels, which are the smallest image units. The identified local image has n pixels in the length direction and m pixels in the width direction. Therefore, the local image is composed of a three-dimensional pixel matrix n×m×3.

[0015] The generalized matrix created, containing only the color values ​​of all pixels, is as follows:

[0016]

[0017] As a further aspect of the present invention: In step three, the image grayscale processing uses a weighted average method to equalize the three variable values ​​in the updated matrix elements, thereby achieving image grayscale processing. Since the human eye is most sensitive to green and least sensitive to blue, the RGB three components are weighted and averaged according to the following formula to obtain a reasonable grayscale image:

[0018] Gray(m,n)=0.3*R(m,n)+0.59*G(m,n)+0.11*B(m,n).

[0019] As a further step in this invention: a weighted average method is used to equalize the three variable values ​​in the updated matrix elements, thereby reducing the corresponding grayscale matrix to a two-dimensional matrix. The specific formula is as follows:

[0020]

[0021] As a further aspect of the present invention: In step four, the grayscale image is binarized using a thresholding method, specifically including:

[0022] The image is set to two different levels based on the difference between the target and the background in the image;

[0023] Find the optimal pixel value to determine whether the pixel is the target or the background;

[0024] The pixel values ​​of the grayscale image are set to 0 or 1 according to the optimal threshold to obtain a binarized image;

[0025] The core of image binarization algorithms is the selection of an ideal threshold. Using the OTSU algorithm, the pixel value (Gray) at any point (m, n) in the grayscale image is... mn for:

[0026] Gray mn =0.3*R′ mn +0.59*G′ mn +0.11*B′ mn

[0027] Based on the threshold T, the inter-class difference between the image background and the target is calculated as follows:

[0028]

[0029] u = w0u0 + w1u1

[0030] g(T)=w0(u0-u) 2 +w1(u1-u) 2

[0031] Where U0 and U1 are the sum of background pixels and target pixels, respectively; w0 and w1 are the number of background pixels and target pixels, respectively; w0 and w1 are the ratios of background pixels and target pixels to the entire image, respectively; u0 and u1 are the average gray values ​​of the background and target, respectively; u is the average gray value of the entire image; g(T) is the inter-class difference when the segmentation threshold is T; when g(T) is the maximum value, the corresponding T is the optimal threshold; by setting the pixel values ​​of the grayscale image to 0 or 1 according to the threshold T, a binarized image can be obtained.

[0032] As a further aspect of the present invention: in step five, marking the connected regions of the binarized image specifically includes:

[0033] The pixel matrix of a binary image is a two-dimensional matrix composed of 0s and 1s, as shown below:

[0034]

[0035] Record the start and end positions of each sequence of white pixels in the first row of P;

[0036] Apart from the first row (which is directly marked), determine whether P(x,y) overlaps with several adjacent pixels in the previous row:

[0037] If there is no overlap, assign a new label to P(x,y);

[0038] If there is overlap, assign P(x,y) the same label as the previous row order; if there are multiple overlaps, label P(x,y) with the smallest label in the previous overlap sequence, and then label the next few labels as equivalent pairs; repeat step 2 until all pixels have been scanned; finally, multiple equivalent pairs are obtained.

[0039] In each equivalent pair <a0,b0>, the labeled region a0 is associated with the labeled region b0; by traversing the graph, the labels in each equivalent pair are updated to the same value, and the maximum value of the label is the number of connected regions.

[0040] The beneficial effects of this invention are as follows: This invention can acquire local images of the weld seam area of ​​a welded component through image recognition technology. It preprocesses the image by first adding Gaussian and salt-and-pepper noise, then using mean and median filtering for noise reduction. The local image before welding is then grayscaled and binarized to maximize the discriminative power of local features, accurately extracting the size and distribution characteristics of the weld seam area. Connected regions of the binarized image are marked and removed according to region size. Based on this, contours are extracted using the bwperim function, filtered according to contour size, and fitted to obtain the oblique line of the weld seam. The region is then filled based on the oblique line to obtain the final weld seam position and size. The proposed real-time weld seam recognition algorithm lays the foundation for autonomous welding by welding robots, improving welding efficiency. It can also identify various weld seam types such as butt welds, lap welds, and V-shaped welds. Furthermore, this invention's recognition algorithm, based on weld seam drawing comparison and optimization of weld seam image missing during the welding process, integrates sensor and image processing technologies to achieve high-precision real-time weld seam recognition. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the process for extracting the weld gap contour in this invention;

[0042] Figure 2This is a picture of the present invention before welding and a partially enlarged schematic diagram of the welding part;

[0043] Figure 3 This is a schematic diagram of image noise removal according to the present invention;

[0044] Figure 4 This is a schematic diagram of a grayscale image of the present invention;

[0045] Figure 5 This is a schematic diagram of the binarized image of the present invention;

[0046] Figure 6 This is a schematic diagram of the equivalent pair of the present invention;

[0047] Figure 7 This is a schematic diagram of the original contour extraction in this invention;

[0048] Figure 8 This is a schematic diagram illustrating the process of filtering and eliminating the original outlines based on size in this invention.

[0049] Figure 9 This is a schematic diagram of contour fitting for the present invention;

[0050] Figure 10 This is a schematic diagram of the area filling in this invention;

[0051] Figure 11 This is a schematic diagram illustrating the location and dimensions of the weld gap in this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1, as Figure 1 As shown, a real-time weld seam recognition algorithm based on a welding robot includes the following steps:

[0054] Step 1: Obtain a local image of the weld seam area of ​​the welded part using image recognition technology, pick out the red, green and blue values ​​of each pixel in the image, and create a generalized matrix that contains only the color values ​​of all pixels;

[0055] Step 2: The image is preprocessed by first adding Gaussian noise and salt-and-pepper noise, and then reducing noise through mean filtering and median filtering. Then, local images are grayscaled and binarized.

[0056] Step 3: After filtering out image noise, re-extract the third dimension information from the 3D matrix, update the three variable values ​​of each element in the generalized matrix, and perform grayscale processing on the image.

[0057] Step 4: Binarize the grayscale image to maximize the discriminative power of local features, thereby accurately extracting the size and distribution characteristics of the weld seam area of ​​the welded component.

[0058] Step 5: Mark the connected regions of the binarized image. Use a traversal scanning algorithm to check the connectivity between the currently scanned pixel and the neighboring pixels of the previous scan. Use the eight-neighborhood method to mark the connected regions to identify weld gaps.

[0059] Step 6: Cut out the area according to its size. Then, extract the contour using the bwperim function, filter the contours according to their size, fit the filtered contours, obtain the oblique line of the weld gap after fitting, fill the area according to the oblique line, and obtain the final position and size of the weld gap.

[0060] Example 2: This example provides a real-time weld seam recognition algorithm based on a welding robot. It obtains a local image of the weld seam area of ​​the welded part through image recognition technology. The image is composed of pixels, which are the smallest image units. Figure 2 The image shown is a welded component and a magnified view of a portion thereof, with 161 pixels in the length direction and 280 pixels in the width direction. In other words, the image consists of a three-dimensional pixel matrix of 161×280×3. To facilitate subsequent image processing, the red, green, and blue values ​​of each pixel are extracted, and a generalized matrix containing only the color values ​​of all pixels is created as follows:

[0061]

[0062] To identify the size of the weld, the local image before welding is first processed by grayscale and binarization. Since image signals are frequently affected by various noises during generation, transmission, and recording, mainly Gaussian noise and salt-and-pepper noise, these noises affect image input, acquisition, processing, and final output. Therefore, noise reduction is necessary before image processing, and the effectiveness of noise removal directly impacts subsequent image processing operations, such as image segmentation and edge detection. This paper employs a method of first adding Gaussian and salt-and-pepper noise, followed by mean filtering and median filtering for image preprocessing. The results are shown below. Figure 3 .

[0063] After image noise filtering, the third dimension information in the 3D matrix is ​​extracted again, and the three variable values ​​of each element in the generalized matrix are updated to perform grayscale processing on the image. Image grayscale processing mainly includes component method, maximum value method, average value method, and weighted average method. Among them, the weighted average method calculates the average of the three components with different weights based on their importance and other indicators. Since the human eye is most sensitive to green and least sensitive to blue, a more reasonable grayscale image can be obtained by weighting the RGB components according to the following formula.

[0064] Gray(i,j)=0.3*R(i,j)+0.59*G(i,j)+0.11*B(i,j)

[0065] The weighted average method is used to equalize the three variable values ​​in the updated matrix elements, thus achieving grayscale processing of the image. For example... Figure 4 As shown, grayscale images have the best sharpness. The corresponding grayscale matrix is ​​reduced to a two-dimensional matrix, as shown in the following formula:

[0066]

[0067] Due to the small gradient values ​​of grayscale images, feature recognition remains relatively complex. To more accurately extract the size and distribution features of the weld seam area of ​​the welded component, the grayscale image is binarized to maximize the discriminative power of local features.

[0068] The most common method for binarizing grayscale images is the thresholding method. First, the image is set to two different levels based on the difference between the target and the background. Then, the optimal pixel value is found to determine whether the pixel is the target or the background. Finally, the pixel value of the grayscale image is set to 0 or 1 according to the optimal threshold, resulting in a binarized image. The core of the binarization algorithm is the selection of the ideal threshold. Thresholding methods mainly include the bimodal method, the p-parameter method, the maximum inter-class variance method (OTSU method), the maximum entropy thresholding method, and iterative methods. Considering its advantages of simple computation and insensitivity to image brightness and contrast, this paper chooses the widely applicable OTSU algorithm, whose central idea is to maximize the inter-class difference between the target and the background through the threshold. Based on the corresponding grayscale matrix reduced to a two-dimensional matrix, the pixel value of any point (m,n) in the grayscale image is... mn for:

[0069] Gray mn =0.3R' mn +0.59G' mn +0.11B' mn

[0070] Based on the threshold T, the inter-class difference between the image background and the target is calculated as follows:

[0071]

[0072]

[0073] u = w0u0 + w1u1

[0074] g(T)=w0(u0-u) 2 +w1(u1-u) 2

[0075] Where U0 and U1 are the sum of background pixels and target pixels, respectively; , w0 and w1 are the number of background pixels and target pixels, respectively; w0 and w1 are the ratios of background pixels and target pixels to the entire image, respectively. u0 and u1 are the average gray values ​​of the background and target, respectively; u is the average gray value of the entire image; g(T) is the inter-class difference when the segmentation threshold is T; when g(T) is the maximum value, the corresponding T is the optimal threshold.

[0076] By setting the pixel values ​​of a grayscale image to 0 or 1 based on a threshold T, a binarized image can be obtained.

[0077]

[0078] The obtained binarized image is as follows Figure 5 Since the image with a pixel value of 1 overlaps with the background of the Word document, the pixel value of 1 is set to 0.85 here to highlight the image. The size and distribution characteristics of the weld seam in the part to be welded can be clearly identified in the binarized image.

[0079] Labeling Connected Components: To identify weld seams, it is first necessary to label the connected components of the binarized image. A traversal scanning algorithm is used to check the connectivity between the currently scanned pixel and its previously scanned neighboring pixels. Here, the eight-neighborhood method is used to label connected components. The pixel matrix P of the binarized image is a two-dimensional matrix composed of 0s and 1s, as shown below:

[0080]

[0081] First, record the start and end positions of each sequence of white pixels in the first row of P. Second, except for the first row (which is directly labeled), determine if P(x,y) overlaps with several adjacent pixels in the previous row. If there is no overlap, assign a new label to P(x,y). If there is overlap, give P(x,y) the same label as the previous row. If there are multiple overlaps, label P(x,y) with the smallest label in the preceding overlapping sequence, and then label the next few labels as equivalent pairs as shown in Table 1. Repeat step two until all pixels have been scanned. Finally, multiple equivalent pairs are obtained, some of which are as follows: Figure 6 As shown.

[0082] Table 1. Core Algorithm for Connected Component Labeling

[0083]

[0084] In each equivalent pair <a0, b0>, the labeled region a0 is associated with the labeled region b0. By traversing the graph, the labels in each equivalent pair are updated to the same value, with the maximum value of the label being the number of connected components. It is calculated that... Figure 5 The number of connected regions is 73.

[0085] The region is cut off based on its size, and then the contour is extracted using the bwperim function, as follows: Figure 7 Filter by outline size, such as Figure 8 The selected contours are fitted to obtain a straight line with an oblique weld seam. Figure 9 Fill the area according to the diagonal line, such as... Figure 10 Obtain the final location and size of the weld gap, such as... Figure 11 .

[0086] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0087] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A real-time weld seam recognition algorithm based on a welding robot, characterized in that, The real-time weld seam identification algorithm includes the following steps: Step 1: Obtain a local image of the weld seam area of ​​the welded part using image recognition technology, pick out the red, green and blue values ​​of each pixel in the image, and create a generalized matrix that contains only the color values ​​of all pixels; Step 2: The image is preprocessed by first adding Gaussian noise and salt-and-pepper noise, and then reducing noise through mean filtering and median filtering. Then, local images are grayscaled and binarized. Step 3: After filtering out image noise, re-extract the third dimension information from the 3D matrix, update the three variable values ​​of each element in the generalized matrix, and perform grayscale processing on the image. Step 4: Binarize the grayscale image to maximize the discriminative power of local features, thereby accurately extracting the size and distribution characteristics of the weld seam area of ​​the welded component. Step 5: Mark the connected regions of the binarized image. Use a traversal scanning algorithm to check the connectivity between the currently scanned pixel and the neighboring pixels of the previous scan. Use the eight-neighborhood method to mark the connected regions to identify weld gaps. Step 6: Cut out the area according to its size. Then, extract the contour using the bwperim function, filter the contours according to their size, fit the filtered contours, obtain the oblique line of the weld gap after fitting, fill the area according to the oblique line, and obtain the final position and size of the weld gap.

2. The real-time weld identification algorithm according to claim 1, characterized in that: Step one, the creation of the generalized matrix, specifically includes: An image is composed of pixels, which are the smallest image units. The identified local image has n pixels in the length direction and m pixels in the width direction. Therefore, the local image is composed of a three-dimensional pixel matrix n×m×3. The generalized matrix created, containing only the color values ​​of all pixels, is as follows:

3. The real-time weld identification algorithm according to claim 1, characterized in that: In step three, the image grayscale processing uses a weighted average method to equalize the three variable values ​​in the updated matrix elements, thus achieving image grayscale processing. Since the human eye is most sensitive to green and least sensitive to blue, the RGB components are weighted and averaged according to the following formula to obtain a reasonable grayscale image: Gray(m,n)=0.3*R(m,n)+0.59*G(m,n)+0.11*B(m,n).

4. The real-time weld identification algorithm according to claim 3, characterized in that: The weighted average method is used to equalize the three variable values ​​in the updated matrix elements, reducing the corresponding grayscale matrix to a two-dimensional matrix. The specific formula is as follows:

5. The real-time weld identification algorithm according to claim 1, characterized in that: In step four, the grayscale image is binarized using a thresholding method, specifically including: The image is set to two different levels based on the difference between the target and the background in the image; Find the optimal pixel value to determine whether the pixel is the target or the background; The pixel values ​​of the grayscale image are set to 0 or 1 according to the optimal threshold to obtain a binarized image; The core of image binarization algorithms is the selection of an ideal threshold. Using the OTSU algorithm, the pixel value at any point (m, n) in the grayscale image is... Graymn for: Gray mn =0.3*R′ mn +0.59*G′ mn +0.11*B′ mn Based on the threshold T, the inter-class difference between the image background and the target is calculated as follows: u = w0u0 + w1u1 g(T)=w0(u0-u) 2 +w1(u1-u) 2 Where U0 and U1 are the sum of background pixels and target pixels, respectively; , w0 and w1 are the number of background pixels and target pixels, respectively; w0 and w1 are the ratios of background pixels and target pixels to the entire image, respectively; u0 and u1 are the average gray values ​​of the background and target, respectively; u is the average gray value of the entire image; g(T) is the inter-class difference when the segmentation threshold is T; when g(T) When the value is the maximum, the corresponding T is the optimal threshold; by setting the pixel values ​​of the grayscale image to 0 or 1 according to the threshold T, a binarized image can be obtained.

6. The real-time weld identification algorithm according to claim 1, characterized in that: In step five, marking the connected regions of the binarized image specifically includes: The pixel matrix of a binary image is a two-dimensional matrix composed of 0s and 1s, as shown below: Record the start and end positions of each sequence of white pixels in the first row of P; Except for the first row, determine whether P(x,y) overlaps with several adjacent pixels in the previous row: If there is no overlap, assign a new label to P(x,y); If there is overlap, assign P(x,y) the same label as the previous row order; if there are multiple overlaps, label P(x,y) with the smallest label in the previous overlap sequence, and then label the next few labels as equivalent pairs; repeat step 2 until all pixels have been scanned; finally, multiple equivalent pairs are obtained. In each equivalent pair <a0,b0>, the labeled region a0 is associated with the labeled region b0; by traversing the graph, the labels in each equivalent pair are updated to the same value, and the maximum value of the label is the number of connected regions.

Images