Intelligent T-shaped welding penetration depth measuring method based on machine vision

Abstract

The invention provides a T-type welding penetration intelligent measuring method based on machine vision, which combines image processing technologies such as image enhancement, bilateral filtering, image corrosion and expansion to pre-process an image, uses a U2Net semantic segmentation neural network to obtain a sensitive area, and cooperates with a straight line recognition algorithm and a watershed algorithm to obtain a penetration area. The weld boundary tangent line is obtained by using the characteristics of image pixelation and color numeralization, and finally the fusion depth pixel distance is calculated by using the point-to-line distance, and is converted into the actual fusion depth by combining the image resolution, the magnification factor and the like. Rapid and batch measurement of the fusion depth is realized, and the problem of low manual measurement efficiency is successfully solved.

Description

Technical Field

[0003] This method relates to the field of intelligent detection of weld quality, and more specifically to an intelligent measurement method for weld penetration depth. Background Art

[0004] As one of the important bases for judging weld quality, the weld penetration depth is currently mainly measured manually by each company, supplemented by drawing tools. Although the drawing tools can omit manual calculations, the marking of the penetration depth still needs to be manually drawn, so only one picture can be processed at a time, resulting in low efficiency. In the face of large-scale and large-batch production, the existing methods are difficult to handle and still require a huge amount of manpower and material resources. To improve the efficiency of weld quality evaluation, achieve fast and accurate calculation of weld penetration depth in a short time and in large quantities, and promote the intelligentization of the welding field, an intelligent measurement method is urgently needed. Summary of the Invention

[0005] To solve the above problems, this paper provides a new intelligent measurement method for the penetration depth of T-shaped welds based on the ITA algorithm, aiming to use image processing technology combined with pixel color values, etc. to realize the intelligent calculation of the penetration depth of T-shaped welds. The adaptive line detection algorithm is used to obtain the weld boundary, replacing manual drawing, and then the pixel distance of the penetration depth is calculated, and it is converted into the actual size through proportional conversion.

[0006] To achieve the above object, the technical solutions adopted by the present invention mainly include the following processes:

[0007] Step 1: Use the U2-Net deep learning algorithm to obtain the weld area, i.e., the sensitive area.

[0008] Step 2: Since the weld image has the characteristic of unclear features, image preprocessing is required.

[0009] (1) Use gamma transformation for image enhancement:

[0010]

[0011] I in represents the brightness of the input image, while I out represents the brightness of the output image. The two parameters γ and c are both used to adjust the shape of the gamma function. In particular, when c is equal to 1, the brightness values of the input and output images are both normalized in the range of 0 to 1. Different values of γ will cause different degrees of brightness stretching effects on the image. For welded parts, their surfaces are metallic in color and appear grayish-white. When γ is less than 1, the output image I out will be brighter than the input image I in i.e., more white, and the internal texture contrast will be reduced instead; on the contrary, when γ is greater than 1, the output image Iout will be darker than the input image Iin, and the internal texture

[0012] The theoretical contrast is greater.

[0013] (2) Use bilateral filtering for denoising, and the formula is as follows:

[0014]

[0015] The weight coefficient w(i,j,k,l) depends on the product of the domain kernel d(i,j,k,l) and the range kernel r(i,j,k,l):

[0016]

[0017]

[0018] Most of the fringe interference will be removed after filtering.

[0019] Step 3: Background separation. In the weld image, the background is generally in the far view, and its gray value has a large difference from that of the welded part. Convert the image into a grayscale image and perform binary image processing according to a certain threshold. Obtain the convolution kernel size through machine vision and perform convolution processing on the image to achieve image erosion and dilation, and then optimize the edges to remove interfering pixels to obtain the area containing the weld image.

[0020] Step 4: Straight line extraction. Use the Hough line detection algorithm to detect the straight lines in the image and extend the detected straight lines. The Hough line detection detects straight lines through the mapping relationship between points and straight lines. First, the coordinates of each contour point on the image will be obtained. There is a unique overlapping mapping straight line between two points. Therefore, record the straight line equation of the coordinates of these two points, and read the width and height of the picture as the abscissa and ordinate, and use the equation to obtain the two boundary points, and extend the straight line to ensure that the target straight line will not be ignored when calculating the angle.

[0021] Secondly, adopt inner and outer double-layer iteration to obtain the target straight line. That is, calculate the angles between all intersecting straight lines, subtract them from the reference angle of 90° and take the absolute value, and record the corresponding straight lines at the same time. Sort them according to the absolute value size, for example, [5,20,15,7] corresponds to [1,4,3,2]. According to the rule of taking the abscissa in the vertical direction and the ordinate in the horizontal direction, perform inner iteration calculation in ascending order until the intersection point is successfully obtained, and the included angle between the intersecting straight lines meets the threshold requirements and the intersection point of the two straight lines is located in the sensitive area. If the number of inner iteration times exceeds one-third of the number of straight lines, perform outer iteration, that is, gradually reduce the reference angle until the target straight line is successfully obtained.

[0022] Step 5: Image region segmentation. Use the straight lines identified in Step 3 for segmentation, and then use the watershed algorithm to segment the connected regions. After segmenting the picture, calculate the area of each partition, set a threshold to exclude irrelevant regions that are much smaller or much larger than the weld region. Determine whether the regions that meet the threshold range are within the sensitive region. If they are within the sensitive region, these regions can be considered approximate weld regions.

[0023] Step 6: Find the target tangent line. First, use the target straight line to obtain the vertex coordinates. The T-shaped weld can be approximately processed as a right triangle. Add a mask, create a new image background of the same size, set it as a binary image, read the coordinates of the weld region and fill white in the corresponding coordinates of the background. Secondly, find the tangent line of the curved edge. It is planned to use a straight line with an inclination angle of 45° or 135° (the straight line angle is related to the specific weld) to be tangent to the curved edge. The difficulty lies in that it is difficult to obtain the pixel coordinates of the curved edge boundary, and the tangent straight line cannot be directly found. To solve this problem, this paper proposes an iterative tangent algorithm (ITA), which iteratively moves outward step by step starting from the origin. The specific steps are as follows. 1. For the points in the y-axis direction, the abscissa is the same as the intersection point, and the absolute value of the difference between the ordinate and the ordinate of the intersection point can be used to obtain the distance between each point in the y-axis direction and the intersection point. Similarly, the distance between the points in the x-axis direction and the intersection point can be obtained; 2. Connect the two points with the same distance to obtain a straight line with an acute angle of 45°, and perform iterative operations from near to far; 3. Traverse the number of black pixels on the straight line. When the number of black pixels exceeds a certain value, it can be considered that the straight line is tangent to the curved edge of the weld. Finally, apply the point-to-line distance formula to calculate the penetration depth. At this time, the calculated is the pixel distance, and size conversion needs to be performed according to the picture resolution.

[0024] The beneficial effects of this method are as follows: Automatically and intelligently calculate the penetration depth to replace manual drawing of the edge line for calculating the penetration depth. Batch measurement replaces manual single-picture measurement, greatly improving the measurement efficiency. Brief Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 is the general flowchart

[0027] Figure 2 Tangent acquisition result diagram Detailed Embodiments

[0028] The following will describe the embodiments of the present application in detail with reference to the drawings.

[0029] The following uses specific specific examples to illustrate the implementation manners of the present disclosure. Those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope protected by the present disclosure.

[0030] In order to describe the technical solutions in the embodiments of the present invention more clearly and completely, the specific implementation manners of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The general flowchart is as Figure 1 shown:

[0031] Step 1: Input the image to be measured, as well as the corresponding image resolution and magnification ratio. It should be noted that for the images in the same batch, the internal parameters of the shooting device need to be consistent, and the parameters set by the device during shooting (such as the magnification ratio) need to be consistent.

[0032] Step 2: Use the image taken in Step 1 to make a dataset as the input of U2-Net for training to obtain weights, and then input the subsequent images into the model, and use the trained weights for segmentation to obtain the sensitive area.

[0033] Step 3: Perform image enhancement processing on the image in Step 1 through gamma transformation. The formula is as follows:

[0034]

[0035] I in represents the brightness of the input image, and I out represents the brightness of the output image. The two parameters γ and c are both used to adjust the shape of the gamma function. In particular, when c is equal to 1, the brightness values of the input and output images are both normalized in the range of 0 to 1. Different values of γ will cause different degrees of brightness stretching effects on the image. For welded parts, their surfaces are metallic in color and appear grayish white. When γ is less than 1, the output image I out will be brighter than the input image I in , that is, the white color increases, and the internal texture contrast decreases instead; on the contrary, when γ is greater than 1, the output image I out will be darker than the input image I in , and the internal texture contrast is greater.

[0036] Step 4: To reduce the interference of fringe, bilateral filtering is used to filter the image, and its formula is:

[0037]

[0038] The weight coefficient w(i, j, k, l) depends on the product of the domain kernel d(i, j, k, l) and the range kernel r(i, j, k, l):

[0039]

[0040] Step 3: The foreground and background are separated by using the difference in image grayscale values. In the weld image, the background is generally in the far view, and its grayscale value has a large difference from that of the welded part. The image is converted into a grayscale image, and then the image is binarized according to a certain threshold. The foreground is white and the background is black. A 4×4 convolution kernel is used to perform erosion processing on the binarized image first, and then dilation processing, and the final result.

[0041] Step 4: The foreground area includes the weld area and other areas. To obtain the weld area, it is planned to draw two intersecting lines based on the edge of the base material. The area divided by the two lines and the weld edge can approximately replace the weld area. The Hough line detection is used to obtain the lines in the image. The pixel distance is set to 1 and the angle is 1°, ensuring that the existing lines are obtained to the maximum extent. Secondly, all the lines are extended to the image boundary to ensure that no intersecting lines are missed.

[0042] Step 6: The target line is obtained by adopting an inner and outer double-layer iteration. That is, calculate the included angle between all intersecting lines, subtract it from the reference angle of 90° and take the absolute value, and record the corresponding lines at the same time. Sort them according to the absolute value size, for example, [5, 20, 15, 7] corresponds to [1, 4, 3, 2]. According to the rule of taking the abscissa in the vertical direction and the ordinate in the horizontal direction, perform inner iteration calculation in ascending order until the intersection point is successfully obtained, and judge whether the intersection point of the two lines is located in the sensitive area. If the number of inner iteration times exceeds one-third of the number of lines, perform outer iteration, that is, gradually reduce the reference angle until the target line is successfully obtained. Then use the line group to divide the connected area, and use the watershed algorithm to divide each area, and calculate the area of each connected area. Set the threshold to exclude the too large and too small areas to obtain the approximate weld area.

[0043] Step 7: Using the relationship between the straight line and the coordinates, and with the help of the coordinates of the vertices of the two straight edges of the weld (the coordinates can be calculated from the vertex coordinates obtained in Step 6), a straight line at a 45° angle to the coordinate axes is used to be tangent to the curved edge of the weld. Set the forward gradient of the straight line to 1, traverse the pixel information on the straight line. When the number of black pixels on the straight line is higher than a certain value, it can be considered that the straight line is tangent to the curved edge of the weld. Calculate the points in the same vertical and horizontal directions as the starting point through the starting point coordinates, and match the points in the vertical direction and the horizontal direction with the same distance from the starting point in pairs. The calculation method is as follows:

[0044] The starting point coordinates are (x 0 , y 0 ). To improve the recognition accuracy, adjust the starting point position to (x 0 - 6, y 0 - 6), and calculate the distance from each point in the image to the starting point:

[0045]

[0046] Note: During the research process, the horizontal axis direction of the recognized coordinates is from left to right, and the vertical axis direction is from top to bottom, which is slightly different from the coordinate axis directions usually used, and there will be differences in the calculation methods.

[0047] Calculate the abscissa of the corresponding vertical point and the ordinate of the corresponding horizontal point:

[0048] If:

[0049]

[0050] The coordinates of the corresponding points are: P 1 (x 0 - 6, y 0 ), P 2 (x 1 , y 0 - 6). Connecting the two points can obtain a straight line that meets the conditions.

[0051] Within the weld area, the pixels between the straight lines are all white or there are a small number of black pixels. When the straight line is tangent to the curved edge, a large number of black pixels will appear between the straight lines. Based on this feature, the tangent position can be located as Figure 2 , and the straight line equation can be obtained through the two end points. Applying the point-to-line distance formula, the penetration depth can be calculated. At this time, the pixel distance is calculated, and the size conversion needs to be carried out according to the image resolution. The conversion formula is:

[0052] L = p / r

[0053] L - actual size (inch)

[0054] p - pixel

[0055] r — Resolution

[0056] This method can automatically calculate the penetration depth of T-shaped welds efficiently and accurately, and can input images in batches for calculation, eliminating the steps of manual drawing and manual calculation, thus liberating human labor. This method can be integrated into an intelligent welding system to achieve timely measurement and timely adjustment, promoting the development of welding intelligence.

[0057] Obviously, the above embodiments are merely examples given to clearly illustrate the technical solutions of the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims

1. A method for intelligently measuring the penetration depth of T-type welding based on machine vision, characterized in that: The method comprises the following steps: S1 uses the U2-Net deep learning algorithm to obtain the weld area, i.e., the sensitive area. S2 preprocesses the measured image, including image enhancement, bilateral filtering, image binarization, and image erosion and dilation. S3 improves Hough line detection by extending the detected line to make it intersect with the image boundary. S4 obtains the coordinates of the target straight line and the corresponding intersection point through inner and outer double-layer iteration. S5 uses the target straight line obtained in S4 to segment the connected area, uses the watershed algorithm to perform image segmentation, and obtains the target area approximating the weld. S6 intends to use a straight line with an acute angle of 45° to be tangent to the curved edge of the weld, and the distance from the vertex to the tangent is the weld penetration.

2. According to the novel T-weld penetration intelligent measurement method based on ITA algorithm of claim 1, it is characterized in that: The image preprocessing in step S2 combines image enhancement, bilateral filtering, image binarization, i.e., image erosion and dilation operations. The processing sequence is image enhancement → bilateral filtering → image binarization → image erosion → image dilation.

3. According to the novel T-weld penetration intelligent measurement method based on ITA algorithm of claim 1, it is characterized in that: The method of obtaining the target straight line in steps S3 and S4: S3 improves the Hough straight line detection so that the detected straight line is extended to the image boundary, and then the target straight line is obtained through the inner and outer double-layer iteration of S4. The specific iterative process is as follows: Calculate the angle between all intersecting lines, subtract it from the reference angle of 90°, take the absolute value, and record the corresponding lines. Sort them by absolute value, for example [5,20,15,7] corresponds to [1,4,3,2]. According to the rule of taking the horizontal coordinate in the vertical direction and the vertical coordinate in the horizontal direction, sort from small to large and perform internal iteration calculation until the intersection point is successfully obtained, and the angle of the intersecting lines meets the threshold requirements and the intersection point of the two lines is located in the sensitive area. If the number of internal iterations exceeds one-third of the number of lines, perform external iteration, that is, the reference angle is gradually reduced until the target line and the corresponding intersection coordinates are successfully obtained.

4. According to a novel T-weld penetration intelligent measurement method based on ITA algorithm as claimed in claim 1, it is characterized in that: In step S5, the target straight line obtained in S4 is used to segment the connected areas of the image, and the watershed algorithm is used to segment the image, the area of ​​each area is calculated and a threshold is set to exclude areas that are too large or too small, and it is determined whether the remaining area is located in a sensitive area. If so, it is considered to be an approximate weld area.

5. The novel T-weld penetration intelligent measurement method based on ITA algorithm according to claim 1 is characterized in that: In step S1, deep learning is used to obtain the weld area, i.e., the sensitive area, and used as the judgment condition for subsequent steps S4 and S5.

6. The novel T-weld penetration intelligent measurement method based on ITA algorithm according to claim 1 is characterized in that: In step S6, a straight line with an acute angle of 45° is used to be tangent to the curved edge of the weld, and the distance from the vertex to the tangent line is the weld penetration. The specific process is: By using the relationship between the straight line, the point and the coordinates, the coordinates of the intersection obtained by S4 can be used. For the points in the y-axis direction, the horizontal coordinate is the same as the intersection, and the vertical coordinate is subtracted from the vertical coordinate of the intersection to take the absolute value to obtain the distance between each point in the y-axis direction and the intersection. Similarly, the distance between the point in the x-axis direction and the intersection can be obtained. Connecting two points with the same distance can obtain a straight line with an acute angle of 45°. Iterate from the front to the back to traverse the number of black pixels on the straight line. However, when the black pixels exceed a certain value, it can be considered that the straight line is tangent to the curved edge of the weld.

Images