Weld seam image processing method based on line structured light

Abstract

The present application relates to a welding seam image processing method based on line structured light, which performs top-hat transformation on the welding seam image to eliminate the phenomenon of uneven background illumination; performs image enhancement on the welding seam image after top-hat transformation; performs smoothing processing on the welding seam image after image enhancement to blur the edges of sharp gray scale changes, suppress noise, and obtain a clear welding seam image; performs binarization on the welding seam image to clearly separate the target from the background; dynamically extracts the ROI region of the binary image to remove interference and improve speed; performs isolated point filtering to remove small isolated blocks in the binary image to obtain a target laser stripe; performs thinning processing on the laser stripe and removes burrs to obtain the center point coordinates of the laser stripe, thereby laying a solid foundation for subsequent extraction of welding seam features. The present application can realize online, real-time and automatic welding seam image processing based on line structured light, has high speed and high precision, is not constrained by the type of welding seam, and is robust to noise interference such as arc light, spatter, smoke and dust, and workpiece reflection during the welding process.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a welding seam image processing method based on line structured light. BACKGROUND

[0002] With the rapid development of China's national economy, automation has become an inevitable trend and direction of future development. Using robots to replace manual labor for automatic welding can not only significantly improve production efficiency, but also effectively improve production quality, save production cost, improve production efficiency and safety factor, and reduce the labor intensity of workers, becoming an ideal choice for more and more enterprises.

[0003] In order to realize automatic welding of robots, the position of the welding seam feature point needs to be accurately extracted. The structured light measurement method has the characteristics of strong real-time performance and simple equipment, and can not only detect the center position of the welding seam, but also capture the shape information, size and other parameters of the welding seam section, which is suitable for various shapes of welding seams and different welding methods, so that it receives more and more attention; for the volume, weight, power consumption and other requirements of the measurement device, the advantages of structured light measurement are more obvious.

[0004] As an active optical measurement technology, the main principle of structured light measurement method is to project controllable light points, light strips or light surfaces to the surface of the measured object by a structured light projector, and then acquire images through an image sensor (such as a camera), and calculate the three-dimensional coordinates of the object according to the geometric relationship of the system and the principle of triangle. According to the structure of the structured light projector projected to the surface of the measured object, the structured light can be divided into point structured light, line structured light and plane structured light. The point structured light measurement method needs to scan the object point by point for measurement. The larger the measured object, the longer the image acquisition and processing time will be, and it is difficult to realize real-time measurement; the amount of three-dimensional coordinate point data provided by plane structured light is very large, and the calculation time will also increase. Therefore, line structured light is more suitable for engineering applications.

[0005] The welding environment is harsh, and there are a lot of arc light, spatter, smoke and other interference. Although a series of measures have been taken in hardware, such as installing a baffle on the vision sensor to block the spatter and smoke, and installing a filter in front of the vision sensor to filter out the arc light, the quality of the welding seam image is still very poor. Therefore, denoising to obtain high-quality welding seam image is the key to realizing welding seam tracking. SUMMARY

[0006] In order to solve the problem of noise interference such as spatter, arc light, smoke and other noise interference in the welding process, and the influence of the reflectivity of the workpiece itself on the quality of the welding seam image, the present application provides a method for extracting fast, robust and real-time processing of welding seam images based on line structured light.

[0007] The technical scheme adopted by the present application to achieve the above-mentioned purpose is: a welding seam image processing method based on line structured light, comprising the following steps:

[0008] Top-hat transformation is performed on the welding seam image to correct the influence of uneven illumination;

[0009] Image enhancement is performed on the welding seam image after top-hat transformation to enhance the contrast of bright gray scale;

[0010] The welding seam image after image enhancement is subjected to smoothing processing;

[0011] The welding seam image after smoothing processing is subjected to binarization processing to obtain a binary image to separate the target from the background;

[0012] The ROI region of the binary image containing the target is dynamically extracted;

[0013] Isolated point filtering is performed, that is, isolated blocks existing in the ROI region of the binary image are removed to obtain the target laser stripe;

[0014] The target laser stripe is subjected to thinning and burr removal to obtain the coordinates of the center points of the laser stripe.

[0015] The top-hat transformation is used to correct the influence of uneven illumination, and its expression is,

[0016]

[0017] In the formula, f is a gray scale image, that is, a welding seam image, b is a structural unit,

[0018]

[0019] is a gray scale opening operation, and the image f is first eroded and then dilated using the same structural unit b,

[0020]

[0021] In the formula, f⊙b is a gray scale erosion, and is the minimum value of the region of the image f coinciding with the structural unit b,

[0022]

[0023] is a gray scale dilation, and is the maximum value of the region of the image f coinciding with the structural unit b,

[0024]

[0025] In the formula, x and y are the coordinates of the pixel points of the gray scale image, and s and t are the coordinates of the structural unit.

[0026] The gray scale opening operation adopts a 3*3 rectangular structure unit multiple erosion, multiple expansion mode instead of n*n rectangular structure unit single erosion, expansion, n>3, and is an odd number.

[0027] The image enhancement adopts gamma transformation for contrast adjustment, specifically, through nonlinear transformation, the gray value of the darker area in the two-part area divided according to the threshold value in the image is enhanced, and the gray value of the brighter area is reduced,

[0028] g=cr γ

[0029] In the formula, r is the gray value of a pixel point of a gray image, the value range is [0, 1], g is the gray value after gamma transformation, c is the gray scaling coefficient, the value is 1, and γ is a gamma factor, which is used to control the scaling degree of gamma transformation;

[0030] When γ>1, the contrast of the brighter area of the image is enhanced;

[0031] When γ<1, the contrast of the darker area of the image is enhanced;

[0032] When γ=1, the image is linearly transformed, and the contrast is kept unchanged;

[0033] After normalizing the image gray value, g is,

[0034]

[0035] The smoothing processing adopts mean filtering, and the gray average value of the window pixel is used to replace the current pixel gray value; the gray value of the (x, y) pixel point in the image is I(x, y), the pixel point set A in the window is centered on the pixel point, and the number of pixels in the window is n A After mean filtering, the gray value of the point is I' i (x, y),

[0036]

[0037] Wherein, m and k are the coordinates of the window pixel point set A.

[0038] The binarization processing adopts a local adaptive threshold, and the binarization threshold at the pixel position is determined according to the pixel value distribution of the pixel neighborhood block; different local binarization thresholds are obtained for different local image regions with different brightness, contrast and texture;

[0039] The calculation steps of the local adaptive threshold are as follows:

[0040] 1) Divide the whole image into N*N sub-windows;

[0041] 2) For each sub-window W i,j , first calculate its mean value of gray scale

[0042]

[0043] where n i,j is the number of pixels in the sub-window W i,j , then the window value of the sub-window W i,j

[0044]

[0045] where α is a proportional coefficient and β is a constant;

[0046] 3) The binarization threshold of the pixel point (x, y) in the image is determined by bilinear interpolation, according to the W 0,0 , W 1,0 , W 0,1 , W 1,1 of the four windows adjacent to the sub-window W i,j containing the pixel point (x, y) to determine the corresponding weight, so as to calculate the binarization threshold of the point; first, linear interpolation is performed in the x direction to obtain

[0047] f(x, 0) = f(0, 0) * (1 - x) + f(0, 1) * x

[0048] f(x, 1) = f(1, 0) * (1 - x) + f(1, 1) * x

[0049] Then linear interpolation is performed in the y direction to obtain

[0050] T(x, y) = f(x, 0) * (1 - y) + f(x, 1) * y

[0051] Comprehensively, the binarization threshold T(x, y) of the pixel point (x, y) is obtained

[0052] T(x, y) = (f(0, 0) * (1 - x) + f(0, 1) * x) * (1 - y) + (f(1, 0) * (1 - x) + f(1, 1) * x) * y

[0053] Where f(0, 0), f(0, 1), f(1, 0), f(1, 1) are the gray scale values of the four nearest neighbor pixel points of (x, y).

[0054] The ROI region of the binarized image containing the target is extracted from the binarized image to extract the irregular region where the target is located; according to the target information of the previous frame image, the ROI region of the current frame image is dynamically extracted; ​​

[0055] For the fillet weld, the three feature point pixel coordinates of the previous frame image are FP1(u1, v1), FP2(u2, v2), and FP3(u3, v3), respectively, and an irregular region around the target laser line is calculated, and the pixel coordinates of the ROI region corner points P1 to P6 are,

[0056] P1=(u1, v1-d)

[0057] P2=(u2, v2-d)

[0058] P3=(u3, v3-d)

[0059] P4=(u3, v3+d)

[0060] P5=(u2, v2+d)

[0061] P6=(u1, v1+d)

[0062] 2d is the distance from the ROI region edge to the laser center line.

[0063] The isolated point filtering removes isolated blocks with an area less than a threshold value. Specifically, an eight-neighbor seed filling method is used to extract all connected regions in the binary image ROI region, the number of pixels in each connected region is calculated, and the area threshold T is determined. If the number of pixels in the connected region is less than T, it is considered as noise and is deleted; otherwise, it is retained.

[0064] The eight-neighbor seed filling method selects a pixel point with a pixel value of 255 as a seed, and then combines the pixel points adjacent to the seed point and with a pixel value of 255 into the same set according to the two conditions of the connected region, i.e. the same pixel value and adjacent position, and finally the pixel set obtained is a connected region, which specifically includes the following steps:

[0065] (1) Scan the ROI region image from top to bottom and from left to right until the current pixel point B(x, y) = 255;

[0066] (2) Take B(x, y) as a seed point and assign it a label, and then press the pixel points with a pixel value of 255 in the eight-neighbor domain of the seed point into the stack;

[0067] (3) Pop the top pixel point of the stack and assign it the same label, and then press the pixel points adjacent to the element in the eight-neighbor domain and with a pixel value of 255 into the stack;

[0068] (4) Repeat step (3) until the stack is empty, at which time a connected region is found;

[0069] (5) Repeat steps (1) to (4) until the scanning is completed, and all connected regions are found.

[0070] The refining of the target laser stripe comprises the following steps:

[0071] ① Create an image pyramid, and obtain a group of down-sampling images through mean filtering, that is, the i-th layer mean pyramid obtains the i+1-th layer pyramid image through neighborhood averaging;

[0072] ② For each pyramid image, determine whether a certain pixel point is at the edge of a connected region by setting a determination condition, delete the pixel point determined as the edge, until the width of the connected region is 1 pixel, so as to obtain the center line of the laser stripe.

[0073] The burr removal searches from the end point of the skeleton, that is, the center line of the laser stripe, stops the search when there is a branch point in the eight-neighborhood of the current point, removes the pixel points in the search path, and reduces the branch number of the branch point by 1, so that the branch point becomes a continuous point.

[0074] The present application has the following advantages and beneficial effects:

[0075] 1. The present application can effectively remove the arc light, spatter, smoke and dust and the noise interference of the reflection of the workpiece itself in the weld image, accurately extract the center coordinates of the laser stripe, and has good anti-interference performance.

[0076] 2. The weld type is not constrained, and can be changed at will.

[0077] 3. The present application can process the weld image based on line structured light on line, in real time and automatically, extract the center coordinates of the laser stripe, has high speed and high precision, and lays a solid foundation for subsequent weld feature extraction. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 is the overall flowchart of the present application;

[0079] Figure 2 is a schematic diagram of bilinear interpolation;

[0080] Figure 3 is an irregular ROI diagram of the fillet weld;

[0081] Figure 4 is a schematic diagram of eight-neighborhood. DETAILED DESCRIPTION

[0082] The present application will be further described in detail below in combination with the drawings and examples.

[0083] As Figure 1As shown, the weld seam image processing method based on line structured light of the present application carries out top-hat transformation on the weld seam image to eliminate the phenomenon of uneven background illumination; carries out image enhancement on the weld seam image after top-hat transformation to enhance the bright gray contrast; carries out smoothing processing on the weld seam image after image enhancement to blur the edges of sharp gray changes and suppress noise; carries out local adaptive threshold binarization on the weld seam image to segment the target region; dynamically extracts the ROI region of the binary image to remove noise interference and improve processing speed; carries out isolated point filtering and acquires the connected region of the binary image by using the eight-neighborhood seed filling method to remove small isolated blocks with an area less than a threshold value, further remove noise interference, and obtain the target laser stripe; carries out thinning processing on the laser stripe and removes burrs, and then obtains the center coordinates of the laser stripe, thereby laying a solid foundation for subsequent extraction of weld seam features. Figure 1 As shown, the method specifically comprises the following steps:

[0084] 1. Top-hat transformation

[0085] During the welding process, there are a large number of arc light, spatter, and smoke dust interference, which makes the background illumination of the weld seam image uneven, so top-hat transformation is adopted for correction to improve the image quality.

[0086] Top-hat transformation is defined as the subtraction of the opening operation of a gray image f, and its expression is,

[0087]

[0088] In the formula, b is a 3x3 rectangular structure unit,

[0089]

[0090] The gray level opening operation can smooth the object contour, disconnect the relatively narrow connection between objects, and eliminate the sharp part of the object edge. The image f is first eroded and then dilated using the same structure unit b,

[0091]

[0092] In the formula, f⊙b is the gray level erosion, which can simplify the object structure, and is the minimum value of the region coinciding with the structure unit b in the image f,

[0093]

[0094] The gray level dilation can fill small holes and narrow gaps in the object to increase the size of the object, and is the maximum value of the region coinciding with the structure unit b in the image f,

[0095]

[0096] In this paper, the gray level opening operation is realized by multiple corrosion and multiple expansion, and the operation speed is improved.

[0097] 2. Image enhancement

[0098] After the top-hat transformation, the high gray area of the image becomes dark. In order to facilitate subsequent image binarization, gamma transformation is used for contrast enhancement. Specifically, through nonlinear transformation, the gray value of the dark area in the image is enhanced, and the gray value of the bright area is reduced,

[0099] g = cr γ (6)

[0100] In the formula, r is the gray value of a certain place in the gray image, and the value range is [0, 1], g is the gray value after gamma transformation, c is the gray scaling factor, usually taking the value of 1, and γ is the gamma factor, which controls the scaling degree of gamma transformation,

[0101] When γ>1, the contrast of the high gray area of the image is enhanced;

[0102] When γ<1, the contrast of the low gray area of the image is enhanced;

[0103] When γ=1, the image is linearly transformed, and the contrast remains unchanged.

[0104] Because the range of the power operation value domain is relatively large, the image gray value is usually normalized first, and then the formula (6) is transformed into,

[0105]

[0106] 3. Image smoothing

[0107] In order to blur the edges of the image with sharp gray changes and suppress noise, mean filtering is used to replace the current pixel gray value with the average gray value of the window pixels. The gray value of the pixel point (x, y) in the image is I(x, y), and the number of pixels in the window pixel set A centered on it is n A After mean filtering, the gray value of the point is I' i (x, y),

[0108]

[0109] 4. Image binarization

[0110] Due to the instability and unpredictability of the welding environment, each frame of the welding image collected during the welding process needs to be analyzed to determine the optimal segmentation threshold. Local adaptive threshold is used to determine the binary threshold value at the pixel position according to the pixel value distribution of the pixel neighborhood block. Different local image regions with different brightness, contrast and texture have different local binary threshold values, which improves the stability and accuracy. The specific calculation steps are as follows:

[0111] (1) Divide the entire image into N×N sub-windows;

[0112] (2) For each sub-window W i,j , first calculate its gray mean

[0113]

[0114] In the formula, n i,j is the number of pixels in the window W i,j , and the window value of the sub-window W i,j is

[0115]

[0116] In the formula, α is the proportion coefficient, and β is the constant.

[0117] (3) The binary threshold value of the (x, y) pixel point in the image adopts the bilinear interpolation idea, as shown in Figure 2 , according to the four windows W 0,0 , W 1,0 , W 0,1 , W 1,1 adjacent to the window W i,j where the (x, y) pixel point is located, the corresponding weights are determined, and the binary threshold value of the point is calculated. First, linear interpolation is performed in the x direction to obtain

[0118] f(x, 0) = f(0, 0)*(1-x) + f(0, 1)*x (11)

[0119] f(x, 1) = f(1, 0)*(1-x) + f(1, 1)*x (12)

[0120] Then linear interpolation is performed in the y direction to obtain

[0121] T(x, y) = f(x, 0)*(1-y) + f(x, 1)*y (13)

[0122] Comprehensively, the binary threshold value T(x, y) of the (x, y) pixel point is obtained

[0123] T(x,y) = (f(0,0)*(1-x)+f(0,1)*x)*(1-y)+(f(1,0)*(1-x)+f(1,1)*x)*y (14)

[0124] 5. Dynamic ROI extraction

[0125] The irregular region where the target is located is extracted from the binary image, unnecessary parts are removed, the operation speed is improved, and meanwhile various interferences outside the ROI region can be removed to improve the precision. In the welding process, the change between the two adjacent images is very small, and the ROI region of the current image can be dynamically extracted according to the target information of the previous image.

[0126] As shown in Figure 3 , taking the fillet weld as an example, the pixel coordinates of the three feature points of the previous image are FP1(u1, v1), FP2(u2, v2), and FP3(u3, v3), from which an irregular region around the laser line can be calculated. Assuming that the distance between the ROI edge and the laser center line is 2d, the pixel coordinates of the ROI corner points P1 to P6 are,

[0127] P1 = (u1, v1-d)

[0128] P2 = (u2, v2-d)

[0129]

[0130] P5 = (u2, v2+d)

[0131] P6 = (u1, v1+d)

[0132] 6. Isolated point filtering

[0133] There are some small isolated blocks in the binary image, and the area is much smaller than that of the target laser stripe, so the eight-neighbor seed filling method is used to extract the connected region, as shown in Figure 4 , the pixel number of each connected region is calculated, and the area threshold T is determined. If the number of pixel points of the connected region is less than T, it is considered as noise and is deleted.

[0134] The connected region is an image region composed of pixel points with pixel values of 255 and adjacent positions in the binary image.

[0135] The eight-neighbor seed filling method selects a pixel point with a pixel value of 255 as a seed, and then according to the two basic conditions of the connected region, i.e. the same pixel value and adjacent positions, the pixel points adjacent to the seed point and with a pixel value of 255 are merged into the same set, and finally the pixel set obtained is a connected region. Specifically,

[0136] (1) Scan the image from top to bottom and from left to right until the current pixel point B(x, y) = 255;

[0137] (2) Take B(x, y) as a seed point and assign it a label, and then push all the pixel points with a value of 255 in the eight-neighborhood of the seed point into the stack;

[0138] (3) Pop the top pixel point from the stack and assign it the same label, and then push all the pixel points adjacent to the element in the eight-neighborhood and with a value of 255 into the stack;

[0139] (4) Repeat step (3) until the stack is empty, at which time a connected region is found;

[0140] (5) Repeat steps (1) to (4) until the scanning is complete, and all connected regions are found.

[0141] 7. Image thinning and burr removal

[0142] Generally, the laser stripe has a certain width, and in order to facilitate subsequent extraction of the weld feature, the weld image needs to be thinned first. In order to improve the speed, this paper first creates an image pyramid for the weld image, and obtains a series of down-sampled images through mean filtering, i.e. the i-th layer mean pyramid obtains the i+1-th layer pyramid image through neighborhood averaging. For each layer image of the pyramid, the medial axis transformation method is used for thinning, and through a judgment condition, it is judged whether a certain pixel point is at the edge of the region or not, and the pixel points judged as the edge are deleted until the width of the region is 1 pixel, and finally the center line of the laser stripe is obtained.

[0143] For most weld images, the noise interference such as arc light, spatter, smoke, etc. can be removed in the early image processing stage, and the thinned laser stripe generally does not have burrs. However, for a small number of weld images with particularly serious interference, there is a situation of spatter overlapping with the laser stripe, and the early image processing cannot completely filter out the interference of spatter, arc light, smoke, etc., and there will still be some residues, causing burrs on the thinned laser stripe, affecting the accuracy of subsequent weld feature point extraction, so the weld image needs to be burr-removed to improve the algorithm accuracy.

[0144] The burr removal is to start searching from the skeleton endpoint, stop searching when there is a branch point in the eight-neighborhood of the current point, remove the pixel points in the search path, and reduce the branch number of the branch point by 1, and the branch point becomes a continuous point. Continue searching until there is no branch point in the laser line, and stop searching.

[0145] The endpoints and branch points are defined as follows:

[0146] Endpoint: It is a skeleton pixel itself; there is only one continuous skeleton pixel region in its eight-neighborhood; it is not part of a tumor;

[0147] Branch point: is a skeleton pixel itself; there are at least three regions of consecutive skeleton pixels within its eight-neighborhood, or there are two regions of consecutive skeleton pixels, and one of them is a region of four or more consecutive pixels.

[0148] The lump is a 2x2 or larger skeleton pixel in the skeleton.

Claims

1. A method for processing a weld seam image based on line structured light, characterized in that The method comprises the following steps: Top-hat transformation is performed on the weld image to correct the influence of uneven illumination; Image enhancement is performed on the weld image after top-hat transformation to enhance the contrast of bright gray scale; The weld image after image enhancement is subjected to smoothing processing; The weld image after smoothing processing is subjected to binarization processing to obtain a binary image to separate the target from the background; The ROI region of the binary image containing the target is dynamically extracted; Isolated point filtering is performed, that is, isolated blocks existing in the ROI region of the binary image are removed to obtain the target laser stripe; The target laser stripe is subjected to thinning and burr removal to obtain the coordinates of the center points of the laser stripe; The dynamic extraction of the ROI region of the binary image containing the target is to extract an irregular region where the target is located from the binary image; according to the target information of the previous frame image, the ROI region of the current frame image is dynamically extracted; For a fillet weld, the pixel coordinates of three feature points of the previous frame image are FP1(u1, v1), FP2(u2, v2) and FP3(u3, v3), an irregular region around the target laser line is calculated, and the pixel coordinates of the corner points P1 to P6 of the ROI region are as follows: P1=(u1, v1-d) P2=(u2, v2-d) P3=(u3, v3-d) P4=(u3, v3+d) P5=(u2, v2+d) P6=(u1, v1+d) 2d is the distance from the edge of the ROI region to the center line of the laser.

2. The line structured light based weld seam image processing method according to claim 1, characterized in that, The top-hat transformation is used to correct the influence of uneven illumination, and the expression is as follows: In the formula, f is a gray scale image, that is, a weld image, b is a structure unit, For the opening operation of gray level, the image f is first eroded and then dilated using the same structure element b, In the formula, f⊙b is a gray scale erosion, and is the minimum value of the region coinciding with the structure unit b in the image f, for the gray scale dilation, the maximum value of the region in the image f that coincides with the structure element b, Wherein, x and y are pixel point coordinates of the gray scale image, and s and t are structure unit coordinates.

3. The line structured light based weld seam image processing method according to claim 2, characterized in that, The gray scale opening operation adopts a 3*3 rectangular structure unit multiple erosion and multiple expansion mode instead of an n*n rectangular structure unit single erosion and expansion mode, n>3, and n is an odd number.

4. The line structured light based weld seam image processing method of claim 1, wherein, The image enhancement adopts gamma transformation for contrast adjustment. Specifically, through nonlinear transformation, the gray scale values of the darker regions in the two regions divided according to the threshold in the image are enhanced, and the gray scale values of the brighter regions are reduced, g = cr γ In the formula, r is the gray scale value of a pixel point of the gray scale image, the value range is [0, 1], g is the gray scale value after gamma transformation, c is a gray scale scaling coefficient, the value is 1, and γ is a gamma factor for controlling the scaling degree of gamma transformation; When γ>1, the contrast of the brighter regions of the image is enhanced; When γ<1, the contrast of the darker regions of the image is enhanced; When γ=1, the image is linearly transformed, and the contrast remains unchanged; After the image gray scale value is normalized, g is as follows:

5. The line structured light based weld seam image processing method of claim 1, wherein, The smoothing processing adopts mean filter, and the gray average value of window pixels is used to replace the gray value of current pixel; the gray value of (x, y) pixel point in the image is I(x, y), the pixel point is taken as the center, the number of pixel points in the window pixel set A is n A , and the gray value of the point after mean filter is I' i (x, y), Wherein, m and k are the coordinates in the window pixel point set A.

6. The line structured light based weld seam image processing method of claim 1, wherein, The binarization processing adopts a local adaptive threshold, and the binarization threshold at the pixel position is determined according to the pixel value distribution of the pixel neighborhood block; different local binarization thresholds are obtained for different local image regions with different brightness, contrast and texture; The calculation steps of the local adaptive threshold are as follows: 1) The entire image is divided into N*N sub-windows; 2) For each sub-window W i,j , first compute its mean gray value where n i,j is the number of subwindows W i,j pixels, then the window value of the subwindow W i,j is , Wherein, a is a proportional coefficient, β is a constant, I(x, y) is the gray value of (x, y) pixel point in the image; 3) The binarization threshold at the (x, y) pixel in the image is bilinearly interpolated according to the sub-windows W i,j W 0,0 , W 1,0 , W 0,1 , W 1,1 The corresponding weights are determined, so as to calculate the binarization threshold of the point; first, linear interpolation is performed in the x direction to obtain f(x, 0) = f(0, 0)*(1-x) + f(0, 1)*x f(x, 1) = f(1, 0)*(1-x) + f(1, 1)*x Then linear interpolation is carried out in the y direction, and T(x, y) = f(x, 0)*(1-y) + f(x, 1)*y is obtained. Comprehensively, the binarization threshold T(x, y) of (x, y) pixel point is obtained, T(x, y) = (f(0, 0)*(1-x) + f(0, 1)*x)*(1-y) + (f(1, 0)*(1-x) + f(1, 1)*x)*y Wherein, f(0, 0), f(0, 1), f(1, 0), f(1, 1) are the gray values of the four nearest neighbor pixel points of (x, y) pixel point. The isolated point filtering removes isolated blocks with an area less than a threshold value, specifically, an eight-neighbor seed filling method is used to extract all connected regions in the ROI region of the binary image, the number of pixels of each connected region is calculated, and the area threshold T is determined, if the number of pixels of the connected region is less than T, the noise is considered to be removed; 7. The line structured light based weld seam image processing method of claim 1, wherein, Otherwise, it is retained. The eight-neighbor seed filling method selects a pixel point with a pixel value of 255 as a seed, then according to the two conditions of the connected region, i.e. the same pixel value and adjacent position, the pixel points adjacent to the seed point and with a pixel value of 255 are merged into the same set, and finally the pixel set obtained is a connected region, which specifically includes the following steps: (1) scan the ROI region image from top to bottom and from left to right until the current pixel point B(x, y) = 255; (2) take B(x, y) as a seed point and assign it a label, then press all pixel points with a pixel value of 255 in the eight-neighbor domain of the seed point into the stack; (3) pop the top pixel point of the stack and assign it the same label, then press all pixel points adjacent to the pixel point in the eight-neighbor domain and with a pixel value of 255 into the stack; (4) repeat step (3) until the stack is empty, at this time a connected region is found; (5) repeat steps (1) to (4) until the scanning is completed, and all connected regions are found. The target laser stripe is thinned, including the following steps:

8. The line structured light based weld seam image processing method of claim 1, wherein, ① Create an image pyramid, obtain a group of down-sampling images through mean filtering, i.e. the i-th layer mean pyramid obtains the i+1-th layer pyramid image through neighborhood averaging; ② For each pyramid image, determine whether a certain pixel point is at the edge of a certain connected region by setting a judgment condition, delete the pixel point determined as the edge, until the width of a certain connected region is 1 pixel, thereby obtaining the center line of the laser stripe. The burr is removed, the search is stopped when there is a branch point in the eight-neighbor domain of the current point, the pixel points in the search path are removed, and the branch number of the branch point is reduced by 1, and the branch point becomes a continuous point.

9. The line structured light based weld seam image processing method of claim 1, wherein, ​

Images