Laser welding seam tracking system and image processing method

Abstract

The invention provides a laser welding seam tracking system and an image processing method. The laser welding seam tracking system specifically comprises a laser transmitter, a high-speed camera, a shell body, a reflective mirror, an environment lamp and a light barrier. Wherein the laser transmitter is used for transmitting laser lines for determining the position of a welding point; the high-speed camera is used for collecting weld position image information in real time; the shell main body is a main component of the welding seam tracking system and is used for mounting and bearing the system and components and providing physical protection for the high-speed camera; the angle and the path of line laser emitted by the laser emitter are adjusted through the reflective mirror, the line laser is irradiated to a needed position, meanwhile, the high-speed camera carries out image collection in real time, welding seam space position feature judgment is carried out according to collected images, and therefore welding seam tracking is achieved. The method is used for identifying the space position of the welding seam and tracking the welding seam in automatic welding, can effectively weaken the influence of welding laser on the welding seam image and the damage to the camera, and is beneficial to effectively extracting the welding seam characteristics at a high speed.

Description

Technical Field

[0001] The present application relates to the technical field of weld tracking, and in particular to a laser weld tracking system and an image processing method. Background Art

[0002] As an important processing technology in modern manufacturing, laser welding technology has been widely used in many manufacturing fields such as automobiles, aerospace, petrochemicals, etc. With the rapid development of the manufacturing industry, the research on intelligent welding technology is becoming increasingly urgent. The traditional welding process has a lot of problems such as arc light, smoke, spatter and poor welding environment. In contrast, laser welding, as a more advanced welding process, has the advantages of small welding heat input, fast welding speed, large weld depth-to-width ratio, small workpiece deformation and easy realization of intelligent welding. The combination of laser welding and robots will greatly improve welding efficiency, make welding methods more flexible and reduce costs. The weld tracking system is the key core component of robot intelligent welding, and will also become an inevitable trend in the development of laser welding and intelligent manufacturing industries in the future.

[0003] Laser welding has the advantages of high energy density and fast welding speed, but due to the small spot diameter and narrow weld seam, it is easy to cause positioning deviation, welding defects, processing size and assembly accuracy errors during welding. Therefore, the real-time extraction and tracking control of weld points during laser welding is the key to ensuring welding quality, and it is also an urgent problem to be solved in engineering. Summary of the invention

[0004] In view of the defects in the prior art, the present invention aims to provide a laser weld tracking system and an image processing method for identifying the spatial position of the weld during automatic welding and improving the welding quality.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows.

[0006] A laser weld tracking system and an image processing method, the tracking system comprises a laser transmitter, a high-speed camera, a shell body, a reflector, two ambient lights, and a light shield; the laser transmitter, the reflector, the ambient light, and the light shield are all mounted on the shell body; the high-speed camera is mounted inside the shell body.

[0007] In the laser weld tracking system, the laser emitter is fixed to the upper end of the shell body by bolts, and the emitted laser is controlled to irradiate the weld in the camera shooting area by adjusting the rotation angle of the reflector.

[0008] The shell body of the laser weld seam tracking system is fixed to the front end of the welding gun and is used to protect and install the high-speed camera, and also serves as the installation body of other components.

[0009] In the laser weld tracking system, the ambient light is installed at the bottom of the shell body to supplement the ambient brightness in an environment with insufficient light, so as to improve the quality of the image taken by the camera.

[0010] In the laser weld tracking system, the light shield is installed on the lower side of the shell body and is located between the line laser and the welding gun. It can block the strong light generated during welding, reduce the influence of the welding laser on the weld feature image, and weaken the image noise.

[0011] Furthermore, the present invention also provides an image processing method for a weld tracking system, and the method comprises the following steps.

[0012] S1. The initial image projected on the workpiece and captured by the camera is extracted using a dynamic region of interest extraction method to extract the target region image.

[0013] S2. Perform median filtering on the target area image to improve the peak interference effect, eliminate isolated noise points and remove interference.

[0014] S3. A grayscale threshold-based segmentation method is used for the preprocessed weld image. A fixed threshold segmentation method or a maximum inter-class variance method is selected for segmentation. The weld image and the laser stripe image are obtained by adjusting the threshold.

[0015] S4. The segmented laser stripe image and weld image are processed by the morphological expansion followed by corrosion method to eliminate the weld contour burrs and noise caused by the defects of the fixed threshold segmentation algorithm.

[0016] S5. The extracted laser stripe image is fitted using the least square method, and the center point is located, wherein the intersection of the laser stripe image and the weld feature image is the weld feature point.

[0017] S6. The histogram averaging method is used to adaptively adjust the image, and the image performance is automatically adjusted according to the brightness, contrast and other characteristics of the weld image to make it clearer, more real and realistic.

[0018] The weld image processing method adopts a dynamic region of interest extraction method, which first extracts the laser stripe region of interest, then extracts the weld region of interest, and finally the two jointly determine the final weld feature region of interest.

[0019] The weld image processing method adopts the Gaussian filtering method in image preprocessing to smooth the image by weighted average of each pixel and its surrounding pixels, and processes it according to the following formula: G(x,y)=med{f(x+i,y+i),(i,j∈W)}, where W is a convolution template with a size of (2k+1)×(2k+1), where k is a natural number. The mean filtering method smoothes the image by calculating the average value of the pixel values ​​in the neighborhood around each pixel, and its formula is: Where M is the filter template and m is the total number of pixels in the template.

[0020] The weld image processing method is an image threshold segmentation process. For the laser stripe image, the maximum inter-class variance method is used, and the calculation formula is: g = W0W1 (M0-M1) 2 For weld images, the fixed range method is used, and the formula is: Among them, T is the specified threshold. If the grayscale of the image pixel is greater than T, it is black, and if it is less than T, it is white.

[0021] The weld image processing method is based on a morphological expansion-first-and-then-corrosion method, wherein the morphological expansion processing can make the weld area shape and edge shape more complete, which can be defined as follows: Among them, A and B are the sets of Z, A is the target structural element, that is, the weld area that has not been expanded. B is the structural element that has not been set. is the reflection of set B. Morphological corrosion processing can eliminate isolated and discontinuous noise points and has the following definition: Among them, A and B are the sets of Z, and A is the weld area after expansion.

[0022] The image processing method adopts the least squares fitting method, and the formula is: Where (x0+y0) is the plane coordinate of the weld feature point in the dynamic area of ​​interest, k h 、b j are the slope and intercept of the laser stripe line after least squares fitting.

[0023] The image processing method adopts the histogram averaging method, and the formula is: P s (s) is the probability of gray level s after transformation, P r (r) is the probability of gray level r before transformation, dr and ds correspond to the differentials of gray levels r and s before transformation, respectively.

[0024] Compared with the existing technology, the above-mentioned laser weld tracking system and image processing method have the following beneficial effects.

[0025] ① Strong applicability. The designed system device has a simple structure, small size, easy installation, high universality, and can be applied to most welding scenes and welding devices.

[0026] ② Strong adjustment capability. Aiming at the problem that the weld image features captured by the high-speed camera are not obvious due to insufficient ambient light during welding, the present invention adds two ambient lights to the system to provide brightness supplement in an environment with insufficient light.

[0027] ③ Welding noise reduction. A light shield is installed on the lower side of the system, located between the welding gun and the line laser, to block the strong light generated during welding, reduce the impact of the welding laser on the weld feature image, reduce image noise, and prevent the welding laser reflection from damaging the camera chip.

[0028] ④ Algorithm optimization. The proposed weld image processing method can extract weld features more quickly and accurately, and has the ability to continuously extract weld feature points, which can reduce the cost of automated welding equipment, reduce the complexity of the image processing algorithm, and improve the continuity and accuracy of welding. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0031] Figure 2 Schematic diagram of a laser vision sensing system for weld tracking provided in an embodiment of the present invention.

[0032] Figure 3 It is a flowchart of an image processing method of the present invention.

[0033] Description of reference numerals:

[0034] 1. Laser transmitter, 2. High-speed camera, 3. Shell body, 4. Reflector, 5. Ambient light, 6. Light baffle, 7. Welding seam, 8. Camera shooting area, 9. Line laser. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0036] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0037] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0038] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0039] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0040] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only implementation method.

[0041] A laser weld tracking system and an image processing method, the system comprising a laser transmitter 1, a high-speed camera 2, a housing body 3, a reflector 4, two ambient lights 5, and a light shield 6. The laser transmitter 1, the reflector 4, the two ambient lights 5, and the light shield 6 are all mounted on the housing body 3. The high-speed camera 2 is mounted inside the housing body 3.

[0042] In this embodiment, preferably, with the welding direction as the reference direction, the housing is mounted 3 to the front end of the welding gun so that the distance between the visual sensor and the front end of the welding gun is 52 mm, and the distance between the high-speed camera 2 and the weld is 150 mm. Turn on the laser transmitter 1, adjust the angle of the reflector 4 so that the line laser can irradiate the center of the camera shooting area, such as Figure 2 As shown, images are continuously collected and photographed by a high-speed camera 2, and then image processing is performed.

[0043] An image processing method described in the present invention includes the following steps.

[0044] In this embodiment, preferably, the first step is to extract the target area, use the dynamic region of interest extraction method for the initial image collected by the high-speed camera, first grayscale the image, then use the Gamma correction method to standardize the color space of the input image, and use nonlinear operations or inverse operations to calculate the gradient of each pixel of the image. Divide the image into small areas, count the gradient histogram of each area, and then form a region block with several areas. Connect the directional gradients of all the region blocks in series to obtain the directional gradient histogram feature of the image, and finally extract the weld feature area to be processed next.

[0045] In this embodiment, preferably, the dynamic region of interest extraction method is to first extract the laser stripe region of interest, then extract the weld region of interest, and finally obtain the weld feature region of interest by extracting the common area of ​​the laser stripe region of interest and the weld region of interest.

[0046] In this embodiment, preferably, the second step is to adopt a smoothing filtering method of median filtering on the extracted target area image in the weld image preprocessing, so as to improve the peak interference effect, eliminate isolated noise points, remove the interference of noise in the image, and improve the signal-to-noise ratio.

[0047] In this embodiment, preferably, the median filtering method is to smooth the image by replacing each pixel value with the median of the pixel value, and its formula is: G(x,y)=med{f(x+i,y+i),(i,j∈W)}, where W is the convolution template with a size of (2k+1)×(2k+1), and k is a natural number.

[0048] In this embodiment, preferably, the third step is to segment the weld image by using a gray threshold-based segmentation method on the preprocessed image, select a fixed threshold method to segment the weld features, and use a maximum between-class variance method to segment the laser stripes, and obtain the weld image and the laser stripe image by adjusting the threshold.

[0049] In this embodiment, preferably, the image threshold segmentation process adopts the maximum inter-class variance method for the laser stripe image. This method can automatically obtain the global optimal threshold by calculation. Let the total number of pixels in the image I (x, y) be N, the average grayscale of the image be M, set a threshold T, and divide the image pixels into two categories, C0 and C1. Assuming that the number of C0 pixels is N0, the proportion of the total number of image pixels is W0, and the grayscale of the average pixel value is M0, the number of all pixels in C1 is N1, the proportion of the total number of pixels is W1, and the average grayscale value in the region is M1, the inter-class variance calculation formula can be derived as follows: g = W0W1 (M0-M1) 2 For weld images, the fixed range method is used, and the formula is: Among them, T is the specified threshold. If the grayscale of the image pixel is greater than T, it is black, and if it is less than T, it is white.

[0052] In this embodiment, preferably, the fourth step is to perform noise reduction processing on the weld image, and use a morphological expansion followed by corrosion method to process the segmented laser stripe image and weld image to eliminate burrs and noise points that appear on the weld edge contour due to defects in the fixed threshold segmentation algorithm.

[0053] In this embodiment, preferably, a morphological method of first dilating and then corroding is adopted to make the shape of the weld area and the edge shape more complete, which can be defined as follows: Among them, A and B are the sets of Z, A is the target structural element, that is, the weld area that has not been expanded. B is the structural element that has not been set. is the reflection of set B. Morphological corrosion processing can eliminate isolated and discontinuous noise points, which can be defined as follows: Among them, A and B are the sets of Z, and A is the weld area after expansion.

[0054] In this embodiment, preferably, the fifth step is to extract weld feature points, and the extracted laser stripe image is fitted using the least squares method to obtain its center point. The intersection of the center point of the laser stripe image and the weld feature image is the weld feature point.

[0055] In this embodiment, preferably, the least squares fitting formula is: Where (x0+y0) is the plane coordinate of the weld feature point in the dynamic area of ​​interest, k h 、b j are the slope and intercept of the laser stripe line after least squares fitting.

[0056] In this embodiment, preferably, the last step is the adaptive adjustment of the weld image, which uses the histogram averaging method to adaptively adjust the image, and automatically adjusts the image performance according to the image's brightness, contrast and other characteristics to make it clearer and more realistic.

[0057] In this embodiment, preferably, the principle formula of the histogram averaging method is: P s Pr(r) is the probability of gray level s after transformation, dr and ds are the differentials of gray levels r and s before transformation, respectively.

[0058] The contents explained in the above embodiments should be understood as these embodiments are only used to more clearly illustrate the present invention, and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

Claims

1. A laser weld tracking system and image processing method, characterized in that: The system comprises a laser emitter (1), a high-speed camera (2), a housing body (3), a reflector (4), two ambient lights (5), and a light shield (6); the laser emitter (1), the reflector (4), the two ambient lights (5), and the light shield (6) are all mounted on the housing body (3); and the high-speed camera (2) is mounted inside the housing body (3).

2. A laser weld seam tracking system according to claim 1, characterized in that: The laser emitter (1) is fixed to the upper end of the shell body (3) by bolts, and the emitted laser is controlled to irradiate the weld by adjusting the rotation angle of the reflector (4); the shell body (3) is fixed to the front end of the welding gun and is used to protect and install the high-speed camera (2), and also serves as the installation body of the line laser emitter (1), the high-speed camera (2), the reflector (4), the ambient light (5), and the light shielding plate (6).

3. A laser weld seam tracking system according to claim 1, characterized in that: The ambient light (5) is mounted on the bottom of the housing body (3) and can supplement the ambient brightness in an environment with insufficient light, so as to improve the quality of the image captured by the camera; the light shield (6) is mounted on the lower side of the housing body (3), located between the line laser and the welding gun, and can block the strong light generated during welding, reduce the influence of the welding laser on the characteristic image of the weld, and weaken the image noise.

4. A weld image processing method, characterized in that: The method comprises the following steps: S1. Target area extraction: The dynamic region of interest extraction method is used for the initial image of the weld captured by the high-speed camera. The weld image is first grayed, and then the Gamma correction method is used to standardize the color space of the input weld image. The gradient of each pixel of the weld image is calculated using nonlinear operations or inverse operations; the weld image is divided into small areas, the gradient histogram of each area is counted, and then several adjacent areas are combined into a region block; the directional gradients of all the region blocks are connected in series to obtain the directional gradient histogram features of the weld image; finally, the weld feature area is extracted; S2. Weld image preprocessing: Gaussian filtering, median filtering or mean filtering is used on the extracted target area image to improve the peak interference effect, eliminate isolated noise points, remove the interference of noise in the weld image, and improve the signal-to-noise ratio; S3. Weld image segmentation: The preprocessed weld image is segmented using a grayscale threshold-based segmentation method. Fixed threshold segmentation or maximum inter-class variance method can be used for segmentation. The weld image and laser stripe image are obtained by adjusting the threshold. S4, weld image denoising: using a morphological expansion followed by corrosion method to process the segmented laser stripe image and weld image, eliminating burrs and noise points on the weld edge contour caused by defects in the fixed threshold segmentation algorithm; S5. Extraction of weld feature points: The extracted laser stripe image is fitted by the least square method to obtain its center point. The intersection of the center point of the laser stripe image and the weld feature image is the weld feature point. S6. Adaptive adjustment of weld image: The histogram averaging method is used to adaptively adjust the image, and the image performance is automatically adjusted according to the brightness, contrast and other characteristics of the weld image to make it clearer and more realistic.

5. The weld image processing method according to claim 4, characterized in that: The dynamic region of interest extraction method is to first extract the laser stripe region of interest, then extract the weld region of interest, and finally the final weld feature area is jointly determined by the laser stripe region of interest and the weld region of interest.

6. The weld image processing method according to claim 4, characterized in that: The Gaussian filter in the image preprocessing method is used to smooth the image by weighted averaging each pixel and its surrounding pixels. The formula is: Where G(x,y) represents the gray value of the image at (x,y), and σ represents the mean square error of the Gaussian function in the template; the median filter method smoothes the image by replacing each pixel value with the median of the pixel value, and its formula is: G(x,y)=med{f(x+i,y+i),(i,j∈W)}, where W is the convolution template with a size of (2k+1)×(2k+1), where k is a natural number; the mean filter method smoothes the image by calculating the average value of the pixel values ​​in the neighborhood around each pixel, and its formula is: Where M is the filter template and m is the total number of pixels in the template.

7. The weld image processing method according to claim 4, characterized in that: Image threshold segmentation is used. For laser stripe images, the maximum inter-class variance method is used. This method can automatically obtain the global optimal threshold by calculation. Let the total number of pixels in the image I (x, y) be N, the average grayscale of the image be M, set a threshold T, and divide the image pixels into two categories, C0 and C1. Assume that the number of C0 pixels is N0, the proportion of the total number of image pixels is W0, and the grayscale of the average pixel value is M0. The number of all pixels in C1 is N1, the proportion of the total number of pixels is W1, and the average grayscale value in the area is M1. The inter-class variance calculation formula can be derived as follows: g = W0W1 (M0-M1) 2 ; For the weld image, the fixed range method is used, and the formula is: Among them, T is the specified threshold. If the grayscale of the image pixel is greater than T, it is black, and if it is less than T, it is white.

8. The weld image processing method according to claim 4, characterized in that: The method of morphological expansion followed by corrosion is adopted, in which the morphological expansion treatment can make the shape of the weld area and the edge shape more complete, which can be defined as follows: Among them, A and B are the sets of Z, A is the target structural element, that is, the weld area that has not been expanded; B is the structural element that has not been set, is the reflection of set B; morphological corrosion processing can eliminate isolated and discontinuous noise points and has the following definition: Among them, A and B are the sets of Z, and A is the weld area after expansion.

9. The weld image processing method according to claim 4, characterized in that: The least squares fitting method is used, and the formula is: Where (x0+y0) is the plane coordinate of the weld feature point in the dynamic area of ​​interest, k h , b j are the slope and intercept of the laser stripe line after least squares fitting.

10. The weld image processing method according to claim 4, characterized in that: The histogram averaging method is used, and the formula is: P s (s) is the probability of gray level s after transformation, P r (r) is the probability of gray level r before transformation, dr and ds correspond to the differentials of gray levels r and s before transformation, respectively.

Images