Weld joint real-time identification algorithm based on welding robot

Abstract

The invention discloses a real-time welding seam recognition algorithm based on a welding robot. The algorithm comprises the steps of obtaining a local image of a welding seam area of a welding part, preprocessing the image, graying the image and binarizing the image. Marking a connected region of the binarized image, performing cutting according to the size of the region, extracting a contour through a bwperim function, performing screening according to the size of the contour, performing contour fitting on the screened contour, obtaining a welding seam skew line after fitting, performing region filling according to the skew line, and obtaining the final welding seam position and size. A foundation can be laid for autonomous welding of the welding robot, and the welding efficiency is improved; various welding seam types such as butt welding seams, lap welding seams and V-shaped welding seams can be identified; and high-precision real-time welding seam identification can be realized based on a fusion sensor for comparing and optimizing welding seam image missing in a welding process by a welding seam drawing and an identification algorithm of an image processing technology.

Description

Technical Field

[0001] The invention relates to a weld recognition algorithm, in particular to a real-time weld recognition algorithm based on a welding robot, belonging to the technical field of weld recognition. Background Art

[0002] With the development of my country's manufacturing industry, intelligent welding technology of welding robots has become a research focus. Weld seam recognition and weld seam error compensation are key to improving the autonomous welding capability of welding robots.

[0003] In the prior art, there is a weld recognition method and a welding system disclosed in publication number CN117206758A, the method comprising: acquiring at least one image of a weld area projected by a linear light source to obtain at least one weld image; analyzing at least one weld image using at least a target model to obtain at least one analysis image, the analysis image comprising at least one of a first target frame and a second target frame, the diagonal points of the first target frame are the left top inflection point and the left bottom inflection point of the weld groove, the diagonal points of the second target frame are the right top inflection point and the right bottom inflection point of the weld groove, the target model is trained by machine learning using multiple sets of data; determining the inflection point position of the weld groove according to the at least one analysis image; determining the position of the weld center point of the weld area and the weld width according to the inflection point position. The existing technology has solved the problem of difficulty in accurately identifying welds, but on the one hand, the existing weld recognition method lacks flexibility and diversity and can only recognize one type of weld. There are problems such as single weld type recognition, complex algorithm and slow speed. On the other hand, the existing weld recognition mainly relies on visual detection, but it is easily affected by strong light, smoke and other noise during the welding process. Metal splash, sparks, bubbles, etc. during the welding process will also cause image occlusion and loss, resulting in low recognition accuracy.

[0004] Therefore, weld seam recognition by welding robots is a challenging task, which requires the use of effective sensor fusion, image processing, deep learning and other technologies to improve the accuracy of weld seam recognition during the welding process. Summary of the invention

[0005] The purpose of the present invention is to provide a real-time weld recognition algorithm based on a welding robot in order to solve at least one of the above technical problems.

[0006] The present invention achieves the above-mentioned purpose through the following technical scheme: a real-time weld recognition algorithm based on a welding robot, the real-time weld recognition algorithm comprising the following steps:

[0007] Step 1: Obtain a local image of the weld area of ​​the welded part by using image recognition technology, pick up the red, green and blue values ​​of each pixel in the image, and create a generalized matrix containing only the color values ​​of all pixels;

[0008] Step 2: Preprocess the image by adding Gaussian noise and salt and pepper noise, and then perform denoising by using mean filtering and median filtering, and then perform grayscale and binarization processing on the local image;

[0009] Step 3: After the image noise is filtered out, the third dimension information in the 3D matrix is ​​re-extracted, and the three variable values ​​of each element in the generalized matrix are updated to grayscale the image;

[0010] Step 4: Binarize the grayscale image so that the local features of the image have the highest discrimination degree, so as to accurately extract the size and distribution characteristics of the weld area of ​​the welded part;

[0011] Step 5: Mark the connected areas of the binary image, use the traversal scanning algorithm to check the connectivity between the current scanned pixel and the adjacent pixels previously scanned, and use the eight-neighborhood method to mark the connected areas to identify the welding gap;

[0012] Step 6. Cut out according to the area size. On this basis, extract the contour through the bwperim function, filter according to the contour size, fit the filtered contour, obtain the oblique straight line of the welding gap after fitting, fill the area according to the oblique straight line, and obtain the final welding gap position and size.

[0013] As a further solution of the present invention: in step 1, the creation of the generalized matrix specifically includes:

[0014] The image is composed of pixels as the smallest image unit. The identified partial image has n pixels in the length direction and m pixels in the width direction. Therefore, the partial image is composed of a three-dimensional pixel matrix of n×m×3.

[0015] The generalized matrix created containing only the color values ​​of all pixels is as follows:

[0016]

[0017] As a further solution of the present invention: in step 3, the image grayscale processing uses a weighted average method to equalize the three variable values ​​in the updated matrix elements to achieve image grayscale processing. Since the human eye is most sensitive to green and least sensitive to blue, the RGB three components are weighted averaged according to the following formula to obtain a reasonable grayscale image:

[0018] Gray(m,n)=0.3*R(m,n)+0.59*G(m,n)+0.11*B(m,n).

[0019] As a further solution of the present invention: the weighted average method is used to equalize the three variable values ​​in the updated matrix elements, and the corresponding gray matrix is ​​reduced to a two-dimensional matrix. The specific formula is:

[0020]

[0021] As a further solution of the present invention: in step 4, the grayscale image is binarized using a threshold method, specifically including:

[0022] The image is set to two different levels by the difference between the target and the background in the image;

[0023] Find the best pixel value to determine whether the pixel is the target or the background;

[0024] The pixel value of the grayscale image is set to 0 or 1 according to the optimal threshold value to obtain a binary image;

[0025] The core of the binary image algorithm is the selection of the ideal threshold. Using the OTSU algorithm, the pixel value of any point (m, n) in the grayscale image is Gray. mn for:

[0026] Gray mn =0.3*R′ mn +0.59*G′ mn +0.11*B′ mn

[0027] According to the threshold T, the inter-class difference between the image background and the target is calculated as follows:

[0028]

[0029] u=w0u0+w1u1

[0030] g(T)=w0(u0-u) 2 +w1(u1-u) 2

[0031] Among them, U0 and U1 are the sum of background pixels and target pixels respectively; are the number of background pixels and target pixels respectively; w0 and w1 are the ratios of background pixels and target pixels to the entire image respectively; u0 and u1 are the average grayscale values ​​of the background and target respectively; u is the average grayscale value of the entire image; g(T) is the inter-class difference when the segmentation threshold is T; when g(T) is the maximum value, the corresponding T is the optimal threshold; the pixel value of the grayscale image is set to 0 or 1 according to the threshold T to obtain a binary image.

[0032] As a further solution of the present invention: in step 5, marking the connected areas of the binary image specifically includes:

[0033] The pixel matrix of a binary image is a two-dimensional matrix composed of 0s and 1s, as shown below:

[0034]

[0035] Record the start and end positions of each sequence of white pixels in the first row of P;

[0036] Except for the first row (which is marked directly), determine whether P(x,y) overlaps with several adjacent pixels in the previous row:

[0037] If there is no overlap, assign P(x,y) a new label;

[0038] If there is overlap, give P(x,y) the same label as the previous row sequence; if there are multiple overlaps, label P(x,y) with the smallest label in the previous overlap sequence, and then label the next few labels as equivalent pairs; repeat the second step until all pixels are scanned; finally, multiple equivalent pairs are obtained;

[0039] In each equivalent pair <a0,b0>, the labeled area a0 is associated with the labeled area b0; by traversing the graph, the labels in each equivalent pair are updated to the same value, and the maximum value of the label is the number of connected areas.

[0040] The beneficial effects of the present invention are as follows: the present invention can obtain a local image of a weld area of ​​a welded part through an image recognition technology, pre-process the image by first adding Gaussian noise and salt and pepper noise, and then reducing noise through mean filtering and median filtering, and then grayscale and binarize the local image before welding, so that the local features of the image have the highest degree of discrimination, so as to accurately extract the size and distribution characteristics of the weld area of ​​the welded part; mark the connected area of ​​the binary image, cut it out according to the size of the area, and on this basis, extract the contour through the bwperim function, screen it according to the size of the contour, perform contour fitting on the screened contour, obtain the oblique straight line of the welding gap after fitting, fill the area according to the oblique straight line, and obtain the final welding gap position and size. The proposed real-time weld recognition algorithm can lay a foundation for autonomous welding of welding robots and improve welding efficiency; and can recognize a variety of weld types such as butt welds, lap welds, and V-shaped welds; and the recognition algorithm of the present invention based on the fusion sensor and image processing technology of the weld image missing in the welding process based on the comparison and optimization of the weld drawings can realize high-precision real-time weld recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of the process of extracting the welding gap contour of the present invention;

[0042] Figure 2This is a picture before welding and a partial enlarged schematic diagram of the welding part of the present invention;

[0043] Figure 3 This is a schematic diagram of image noise removal according to the present invention;

[0044] Figure 4 It is a schematic diagram of a grayscale image of the present invention;

[0045] Figure 5 It is a schematic diagram of a binarized image of the present invention;

[0046] Figure 6 It is a schematic diagram of an equivalent pair of the present invention;

[0047] Figure 7 This is a schematic diagram of the original contour extraction of the present invention;

[0048] Figure 8 This is a schematic diagram of the original contour being screened and eliminated according to size in the present invention;

[0049] Fig. 9 It is a schematic diagram of contour fitting of the present invention;

[0050] Fig.10 It is a schematic diagram of the area filling of the present invention;

[0051] Fig.11 This is a schematic diagram of obtaining the welding gap position and size of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Embodiment 1, as Figure 1 As shown, a real-time weld recognition algorithm based on a welding robot includes the following steps:

[0054] Step 1: Obtain a local image of the weld area of ​​the welded part by using image recognition technology, pick up the red, green and blue values ​​of each pixel in the image, and create a generalized matrix containing only the color values ​​of all pixels;

[0055] Step 2: Preprocess the image by adding Gaussian noise and salt and pepper noise, and then perform denoising by using mean filtering and median filtering, and then perform grayscale and binarization processing on the local image;

[0056] Step 3: After the image noise is filtered out, the third dimension information in the 3D matrix is ​​re-extracted, and the three variable values ​​of each element in the generalized matrix are updated to grayscale the image;

[0057] Step 4: Binarize the grayscale image so that the local features of the image have the highest discrimination degree, so as to accurately extract the size and distribution characteristics of the weld area of ​​the welded part;

[0058] Step 5: Mark the connected areas of the binary image, use the traversal scanning algorithm to check the connectivity between the current scanned pixel and the adjacent pixels previously scanned, and use the eight-neighborhood method to mark the connected areas to identify the welding gap;

[0059] Step 6. Cut out according to the area size. On this basis, extract the contour through the bwperim function, filter according to the contour size, fit the filtered contour, obtain the oblique straight line of the welding gap after fitting, fill the area according to the oblique straight line, and obtain the final welding gap position and size.

[0060] Embodiment 2: This embodiment provides a real-time weld recognition algorithm based on a welding robot, which acquires a local image of a weld area of ​​a welded part through image recognition technology, and the image is composed of pixel points as the smallest image unit. Figure 2 The weldment and the enlarged partial image of the weldment are shown, which have 161 pixels in the length direction and 280 pixels in the width direction. In other words, the image consists of a three-dimensional pixel matrix of 161×280×3. To facilitate the subsequent processing of the image, the red, green and blue values ​​of each pixel are picked up, and a generalized matrix containing only the color values ​​of all pixels is created as follows:

[0061]

[0062] In order to identify the size of the weld, the local image before welding is first grayed and binarized. Since the image signal is often interfered by various noises during the generation, transmission and recording process, mainly including Gaussian noise and salt and pepper noise, it affects the input, acquisition, processing and final output of the image. Therefore, before processing the image, it is necessary to perform noise reduction on it, and the effect of noise removal directly affects subsequent image processing operations, such as image segmentation and edge detection. This paper adopts the method of adding Gaussian noise and salt and pepper noise first, and then using mean filtering and median filtering to reduce noise to preprocess the image. The results are shown in Figure 2. Figure 3 .

[0063] After the image noise is filtered out, the third dimension information in the 3D matrix is ​​re-extracted, and the three variable values ​​of each element in the generalized matrix are updated to grayscale the image. Image grayscale processing mainly includes component method, maximum method, average method and weighted average method. Among them, the weighted average method weights the three components with different weights according to importance and other indicators. Since the human eye is most sensitive to green and least sensitive to blue, a more reasonable grayscale image can be obtained by weighted averaging the three RGB components according to the following formula.

[0064] Gray(i,j)=0.3*R(i,j)+0.59*G(i,j)+0.11*B(i,j)

[0065] The weighted average method is used to equalize the three variable values ​​in the updated matrix elements to realize the grayscale processing of the image. Figure 4 As shown, the grayscale image has the best clarity. The corresponding grayscale matrix is ​​reduced to a two-dimensional matrix, as shown in the following formula:

[0066]

[0067] Since the gradient value of grayscale images is small, its feature recognition is still relatively complex. In order to more accurately extract the size and distribution characteristics of the weld area of ​​the welded part, the grayscale image is binarized so that the local features of the image have the highest discrimination.

[0068] The most commonly used method for grayscale image binarization is the threshold method. First, the image is set to two different levels by the difference between the target and the background in the image. Then, the optimal pixel value is found to determine whether the pixel is the target or the background. Finally, the pixel value of the grayscale image is set to 0 or 1 according to the optimal threshold to obtain a binary image. The core of the binary image algorithm is the selection of an ideal threshold. The threshold selection methods mainly include the bimodal method, the p-parameter method, the big law method (OTSU method or maximum inter-class variance method), the maximum entropy threshold method and the iterative method. Considering the advantages of simple calculation and not being affected by image brightness and contrast, this paper selects the OTSU algorithm with strong applicability. The central idea is that the threshold maximizes the inter-class difference between the target and the background. According to the corresponding gray matrix reduced to a two-dimensional matrix, the pixel value Gray at any point (m,n) in the grayscale image mn for:

[0069] Gray mn =0.3R' mn +0.59G' mn +0.11B' mn

[0070] According to the threshold T, the inter-class difference between the image background and the target is calculated as follows:

[0071]

[0072]

[0073] u=w0u0+w1u1

[0074] g(T)=w0(u0-u) 2 +w1(u1-u) 2

[0075] Among them, U0 and U1 are the sum of background pixels and target pixels respectively; are the number of background pixels and target pixels respectively; w0, w1 are the ratios of background pixels and target pixels to the entire image respectively; u0, u1 are the average grayscale values ​​of the background and target respectively; u is the average grayscale value of the entire image; g(T) is the inter-class difference when the segmentation threshold is T; when g(T) is the maximum value, the corresponding T is the optimal threshold.

[0076] The pixel value of the grayscale image is set to 0 or 1 according to the threshold T to obtain a binary image:

[0077]

[0078] The obtained binary image is Figure 5 ,Since the image with pixel 1 overlaps with the background of the word drawing, in order to highlight the image, the pixel value 1 is set to 0.85. In the binary image, the size and distribution characteristics of the welding gap of the part to be welded can be clearly identified.

[0079] Marking connected areas: In order to identify welding gaps, it is first necessary to mark the connected areas of the binary image. The traversal scanning algorithm is used to check the connectivity between the current scanned pixel and the adjacent pixels previously scanned. Here, the eight-neighborhood method is used to mark the connected areas. The pixel matrix P of the binary image is a two-dimensional matrix composed of 0 and 1, as shown below:

[0080]

[0081] First, record the start and end positions of each sequence of white pixels in the first row of P. Secondly, except for the first row (which is directly marked), determine whether P(x, y) overlaps with several adjacent pixels in the previous row. If there is no overlap, assign P(x, y) a new label. If there is an overlap, give P(x, y) the same label as the previous row sequence. If there are multiple overlaps, label P(x, y) with the smallest label in the previous overlapping sequence, and then label the next few labels as equivalent pairs as shown in Table 1. Repeat the second step until all pixels are scanned. Finally, multiple equivalent pairs are obtained, some of which are as follows: Figure 6 shown.

[0082] Table 1 Core algorithm for connected region labeling

[0083]

[0084] In each equivalent pair <a0,b0>, the labeled area a0 is related to the labeled area b0. By traversing the graph, the labels in each equivalent pair are updated to the same value, and the maximum value of the label is the number of connected areas. After calculation, Figure 5 The number of connected regions in is 73.

[0085] The area is cut out according to its size, and on this basis, the contour is extracted by the bwperim function, such as Figure 7 . Filter by contour size, such as Figure 8 The contour after screening is fitted, and the welding gap oblique straight line is obtained after fitting, such as Fig. 9 . Fill the area according to the oblique straight line, such as Fig.10 . Get the final welding gap position and size, such as Fig.11 .

[0086] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0087] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A real-time weld recognition algorithm based on a welding robot, characterized in that: The real-time weld identification algorithm comprises the following steps: Step 1: Obtain a local image of the weld area of ​​the welded part by using image recognition technology, pick up the red, green and blue values ​​of each pixel in the image, and create a generalized matrix containing only the color values ​​of all pixels; Step 2: Preprocess the image by adding Gaussian noise and salt and pepper noise, and then perform denoising by using mean filtering and median filtering, and then perform grayscale and binarization processing on the local image; Step 3: After the image noise is filtered out, the third dimension information in the 3D matrix is ​​re-extracted, and the three variable values ​​of each element in the generalized matrix are updated to grayscale the image; Step 4: Binarize the grayscale image so that the local features of the image have the highest discrimination degree, so as to accurately extract the size and distribution characteristics of the weld area of ​​the welded part; Step 5: Mark the connected areas of the binary image, use the traversal scanning algorithm to check the connectivity between the current scanned pixel and the adjacent pixels previously scanned, and use the eight-neighborhood method to mark the connected areas to identify the welding gap; Step 6. Cut out according to the area size. On this basis, extract the contour through the bwperim function, filter according to the contour size, fit the filtered contour, obtain the oblique straight line of the welding gap after fitting, fill the area according to the oblique straight line, and obtain the final welding gap position and size.

2. The real-time weld identification algorithm according to claim 1 is characterized in that: In the step 1, the creation of a generalized matrix specifically includes: The image is composed of pixels as the smallest image unit. The identified partial image has n pixels in the length direction and m pixels in the width direction. Therefore, the partial image is composed of a three-dimensional pixel matrix of n×m×3. The generalized matrix created containing only the color values ​​of all pixels is as follows:

3. The real-time weld identification algorithm according to claim 1 is characterized in that: In step 3, the image grayscale processing uses a weighted average method to equalize the three variable values ​​in the updated matrix elements to achieve image grayscale processing. Since the human eye is most sensitive to green and least sensitive to blue, the RGB three components are weighted averaged according to the following formula to obtain a reasonable grayscale image: Gray(m,n)=0.3*R(m,n)+0.59*G(m,n)+0.11*B(m,n).

4. The real-time weld identification algorithm according to claim 3 is characterized in that: The weighted average method is used to equalize the three variable values ​​in the updated matrix elements, and the corresponding gray matrix is ​​reduced to a two-dimensional matrix. The specific formula is:

5. The real-time weld identification algorithm according to claim 1 is characterized in that: In the step 4, the grayscale image is binarized using a threshold method, which specifically includes: The image is set to two different levels by the difference between the target and the background in the image; Find the best pixel value to determine whether the pixel is the target or the background; The pixel value of the grayscale image is set to 0 or 1 according to the optimal threshold value to obtain a binary image; The core of the binary image algorithm is the selection of the ideal threshold. Using the OTSU algorithm, the pixel value of any point (m, n) in the grayscale image is Graymn for: Gray mn =0.3*R′ mn +0.59*G′ mn +0.11*B′ mn According to the threshold T, the inter-class difference between the image background and the target is calculated as follows: u=w0u0+w1u1 g(T)=w0(u0-u) 2 +w1(u1-u) 2 Among them, U0 and U1 are the sum of background pixels and target pixels respectively; are the number of background pixels and target pixels respectively; w0, w1 are the ratios of background pixels and target pixels to the entire image respectively; u0, u1 are the average grayscale values ​​of the background and target respectively; u is the average grayscale value of the entire image; g(T) is the inter-class difference when the segmentation threshold is T; when g(T) When it is the maximum value, the corresponding T is the optimal threshold; the pixel value of the grayscale image is set to 0 or 1 according to the threshold T to obtain a binary image.

6. The real-time weld identification algorithm according to claim 1 is characterized in that: In the step 5, marking the connected areas of the binary image specifically includes: The pixel matrix of a binary image is a two-dimensional matrix composed of 0s and 1s, as shown below: Record the start and end positions of each sequence of white pixels in the first row of P; Except for the first row, determine whether P(x,y) overlaps with several adjacent pixels in the previous row: If there is no overlap, assign P(x,y) a new label; If there is overlap, give P(x,y) the same label as the previous row sequence; if there are multiple overlaps, label P(x,y) with the smallest label in the previous overlap sequence, and then label the next few labels as equivalent pairs; repeat the second step until all pixels are scanned; finally, multiple equivalent pairs are obtained; In each equivalent pair <a0,b0>, the labeled area a0 is associated with the labeled area b0; by traversing the graph, the labels in each equivalent pair are updated to the same value, and the maximum value of the label is the number of connected areas.

Images