Detection method and device for identifying and positioning welding seam of steel grating
Through laser projection and grayscale image acquisition, combined with welding joint detection model and OSTU algorithm, the problems of high labor intensity and poor welding quality consistency in steel grating welding are solved, and the rapid and accurate identification and positioning of steel grating welding joints are achieved, improving the Welding quality and efficiency.
Patent Information
- Application Number
- CN202411131841.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
AI Technical Summary
The prior art has problems in steel grating welding with high labor intensity, harsh working environment and poor welding quality consistency. The difference between the steel grating and background under conventional lighting conditions is small, which affects the accuracy of welding joint identification and positioning.
Laser projection and grayscale image acquisition are used, and solder joint detection model is used to conduct solder joint detection, filtering, contrast enhancement, binarization and shape template matching are used to optimize binary processing through Blob analysis and OSTU algorithm to improve solder joint recognition accuracy , and the structural characteristics of the steel grating are displayed through laser projection to achieve rapid and accurate welding joint positioning.
It improves the accuracy of welding joint identification and positioning, reduces the impact of uneven light, enhances the consistency and reliability of welding quality, and improves the efficiency and accuracy of automated welding.
Smart Images

Figure CN119027398A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of steel grating weld identification and positioning, and specifically relates to a detection method and device for steel grating weld identification and positioning. Background Art
[0002] With the development of modern industry, steel structures are widely used in the fields of construction, bridges, shipbuilding, machinery manufacturing, etc. Welding is one of the main connection methods of steel structures, and its quality directly affects the safety and reliability of the entire structure. Steel grating is a steel structure with a grid in the middle, which is made of flat steel arranged crosswise at a certain spacing and cross bars and welded. At present, steel grating welding is mainly done manually, which has problems such as high labor intensity, poor working environment, and poor welding quality consistency, which seriously affects the welding quality. Automated welding can not only improve the quality and efficiency of steel grating welding, meet the development requirements of modern industry for high-quality and high-precision welding, but also ensure the consistency and reliability of the product. The accuracy of weld identification and positioning is an important prerequisite for achieving high-quality automated welding.
[0003] In the steel grating images collected under normal lighting conditions, the difference and contrast between the steel grating and the background are small, and its structural features are not obvious, which affects the feature extraction of the steel grating and results in low accuracy in weld spot recognition and positioning. Summary of the invention
[0004] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to provide a detection method and device for identifying and locating welds of steel gratings.
[0005] The present invention solves the technical problem by adopting the following technical solution:
[0006] In one aspect, the present invention provides a detection method for identifying and locating a weld of a steel grating, comprising the following steps:
[0007] Step 1: Project laser onto the steel grating and collect the grayscale image and depth image of the steel grating under the laser projection condition;
[0008] Step 2: Based on the grayscale image of the steel grating, use the weld detection model to perform weld detection and obtain the pixel coordinates of the welds to be welded;
[0009] In the welding spot detection model, the grayscale image of the steel grating is filtered, contrast enhanced, binarized, opened and matched with shape templates to obtain the ROI area. Through Blob analysis, all the pixels in the ROI area are screened according to the area and long axis length of the connected domain. When the area and long axis length of the connected domain are greater than their respective thresholds, all the pixels in the connected domain are retained. For all the retained pixels, the two pixels with larger coordinates in the long side direction of the steel grating are used for straight line fitting to obtain a straight line. The remaining pixels are perpendicular to the straight line, and the foot of the perpendicular is the point to be welded, thereby obtaining the pixel coordinates of each point to be welded.
[0010] For the binarization process, firstly, the grayscale values of all pixels in the row and column where a certain pixel is located are extracted, and the OSTU thresholds of all extracted pixels are calculated using the traditional OSTU algorithm; then, the adaptive threshold fine-tuning amount P is calculated; finally, the OSTU threshold is added to the adaptive threshold fine-tuning amount to obtain the binarization threshold of the pixel, and the pixel is binarized according to the binarization threshold; the above process is repeated to binarize all pixels;
[0011]
[0012] In the formula, Q max and Q min is the maximum and minimum grayscale values of the row and column where a pixel is located, P max and P min is the threshold fine-tuning amount of the image under the strongest and weakest illumination, which is an empirical value; x0 is a constant;
[0013] Step 3: According to the pixel coordinates of the welding point and the depth information of the welding point obtained from the depth image of the steel grating, the position of the welding point in the robot coordinate system is obtained through coordinate conversion, and the robot performs the welding operation according to this position.
[0014] Furthermore, the shape template matching includes: firstly, a cross line area formed by intersecting laser lines is cut out from the image as a template, and a pyramid layer-by-layer search is used to perform template matching. For the source image generated by each slide, the similarity between the template and the source image is measured according to formula (4). If formula (4) holds, it indicates that the matching is impossible, and the matching is terminated in advance.
[0015]
[0016] In the formula, k j is the sum of the cosine values of the angle differences between the template and the first j edge points of the source image, k min is the similarity threshold, n is the number of edge points in the template, θ i is the angle difference between the template and the i-th edge point of the source image.
[0017] Furthermore, in the first step, two line lasers are symmetrically distributed, one projecting laser along the long side of the steel grating and tilted downward at 45°, and the other line laser is located above the steel grating and projects laser in a direction perpendicular to the grid.
[0018] On the other hand, the present invention provides a detection device for identifying and locating steel grating welds, comprising a detection system, a laser projection module, an action execution module and an image acquisition module; the laser projection module is located at the end of the action execution module, and the image acquisition module is located on the laser projection module, and the laser projection module, the action execution module and the image acquisition module are all communicatively connected to the detection system.
[0019] Compared with the prior art, the present invention has the following beneficial effects:
[0020] 1. Perform laser projection on the steel grating, where two line lasers are projected in a direction parallel to the long side of the steel grating and tilted downward at 45°, and a line laser located in the middle of the two line lasers is projected in a direction perpendicular to the grid of the steel grating. The grayscale image of the steel grating under the laser projection condition is collected, so that the structure of the steel grating appears in the image in the form of laser lines, highlighting the structural features of the steel grating, which is conducive to feature extraction of the weld spot detection model, and realizes fast and accurate identification and positioning of weld spots.
[0021] 2. The segmentation threshold of the traditional OSTU algorithm is for the entire image, and has a good effect on the binarization processing of a single foreground target. However, the grayscale image of the steel grating is collected under laser projection conditions, and is affected by uneven lighting. The use of the traditional OSTU algorithm for binarization processing cannot achieve ideal results. Therefore, the present invention calculates the OSTU threshold based on the grayscale values of all pixels in the rows and columns where the pixel points are located, and simultaneously calculates the adaptive threshold fine-tuning amount. The OSTU threshold and the adaptive threshold fine-tuning amount are superimposed to obtain the binarization threshold of the pixel point. Each pixel point has its own binarization threshold, which avoids the presence of more background or target missing in the binarized image due to uneven lighting, thereby improving the positioning accuracy of the weld point.
[0022] 3. In the shape template matching process, an improved similarity metric standard is proposed, that is, to judge whether the sum of the squares of the cosine values of the angle differences between the template and the first j edge points of the source image meets the termination condition. If so, the matching is terminated in advance. This similarity metric standard can speed up the template matching process and improve the efficiency of template matching and welding. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 It is a schematic diagram of the structure of the detection device;
[0024] Figure 2 It is the structural diagram of the laser projection module;
[0025] Figure 3 is a flow chart of the detection method;
[0026] Figure 4 It is the grayscale image of the steel grating collected without laser projection;
[0027] Figure 5 It is the grayscale image of the steel grating collected under the laser projection condition;
[0028] Figure 6 It is a flow chart of solder joint inspection;
[0029] Figure 7 It is a schematic diagram of the pixel coordinate position of the welding point to be welded;
[0030] In the figure, 1-detection system; 2-laser projection module; 3-action execution module; 4-image acquisition module. DETAILED DESCRIPTION
[0031] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the protection scope of the present application.
[0032] The present invention provides a detection device for steel grating weld identification and positioning, comprising a detection system 1, a laser projection module 2, an action execution module 3 and an image acquisition module 4; wherein the laser projection module 2 is installed at the end of the action execution module 3, the image acquisition module 4 is installed on the laser projection module 2, and the laser projection module 2, the action execution module 3 and the image acquisition module 4 are all connected to the detection system 1 in communication. The laser projection module 2 is used to project laser onto the steel grating, so that the structural features of the steel grating are more prominent under the laser projection condition; the image acquisition module 4 is used to collect the grayscale image and depth image of the steel grating under the laser projection condition; the action execution module 3 is used to perform operations such as automatic welding, adjusting the laser projection and image acquisition position; the detection system 1 comprises a processor and a memory, the processor is responsible for receiving instructions and performing corresponding tasks according to the instructions, a weld spot detection model for weld spot identification and positioning is deployed in the processor, and the memory is responsible for storing data and programs.
[0033] like Figure 2 As shown, the laser projection module 2 includes a connecting bracket and three line lasers; the connecting bracket is fixed at the end of the action execution module 3, two of the line lasers are fixed at both ends of the connecting bracket to project lasers along the long side of the steel grating and tilted downward by 45°, and the other line laser is fixed in the middle of the connecting bracket to project lasers in a direction perpendicular to the grid.
[0034] The image acquisition module 4 is a 3D industrial camera, which can be a surface structured light camera or other industrial cameras. The action execution module 3 is a six-axis robot arm with a welding gun.
[0035] The present invention provides a detection method for identifying and locating a weld of a steel grating, comprising the following steps:
[0036] Step 1: The laser projection module 2 projects laser onto the steel grating, and the image acquisition module 4 is used to acquire the grayscale image and depth image of the steel grating under the laser projection condition. The imaging plane of the image acquisition module 4 should be parallel to the steel grating;
[0037] Figure 4 This is a grayscale image of the steel grating collected without laser projection conditions. In this image, the steel grating is not much different from the background, the difference between the light and dark areas of the image is small, the contrast is small, and the image brightness is uneven, making it difficult to locate the position of the steel grating in the image. Figure 5 This is a grayscale image of the steel grating collected under laser projection conditions. There are many bright spots in the image. These bright spots are the structure of the steel grating that appears in the image in the form of laser lines. The contrast between the bright spots and the image background is large and the significance is high. They are easy to extract and are conducive to the positioning of welds.
[0038] Step 2: Based on the grayscale image of the steel grating, use the weld detection model to perform weld detection and obtain the pixel coordinates of the welds to be welded;
[0039] The grayscale image of the steel grating is input into the weld detection model. It is first subjected to median filtering to filter out the noise in the image while preventing the edge information of the steel grating from being blurred. The contrast of the filtered grayscale image is enhanced to strengthen the contrast between the laser part and the background in the image and weaken the influence of the background. The formula for contrast enhancement is:
[0040] g(i,j)=αf(i,j)+β (1)
[0041] Wherein, α>0 is a gain parameter, β is a bias parameter, f(i,j) and g(i,j) are the pixel values before and after contrast enhancement, respectively; in this embodiment, α is 1.5 and β is 17; α is multiplied by the pixel value before contrast enhancement as a coefficient, which will enlarge the difference between the maximum pixel value and the minimum pixel value, thereby improving the contrast; β is used as an addend to directly increase the pixel value before contrast enhancement, which can improve the brightness;
[0042] The improved OSTU algorithm is used to binarize the contrast-enhanced grayscale image. The traditional OSTU algorithm is to find the segmentation threshold of the entire image and segment the pixels of the entire image according to the segmentation threshold. The segmentation threshold is global and can only segment a single foreground target. Since the grayscale image of the steel grating is collected under laser projection conditions, uneven illumination can easily lead to a large amount of background or target missing in the binarized image, and the traditional OSTU algorithm does not have a good effect. Therefore, first, the grayscale values of all pixels in the row and column where a certain pixel is located are extracted, and the traditional OSTU algorithm is used to calculate the OSTU threshold of all extracted pixels. Then, the adaptive threshold fine-tuning amount P is calculated according to equations (2) and (3). Finally, the OSTU threshold is added to the adaptive threshold fine-tuning amount to obtain the binarization threshold of the pixel, and the pixel is binarized according to the binarization threshold. All pixels are traversed, and each pixel has a corresponding binarization threshold to reduce the influence of uneven illumination.
[0043]
[0044] In the formula, Q max and Q min is the maximum and minimum grayscale values of the row and column where a pixel is located, P max and P min is the threshold fine-tuning amount of the image under the strongest and weakest illumination, which is an empirical value. In this embodiment, it is 19 and 5 respectively; x0 is a constant, and its value range is [Q min -60,Q min -50];
[0045] Perform an opening operation on the binary grayscale image, first erode the binary grayscale image, then dilate the eroded image to obtain the grayscale image after the opening operation; the width and height of the eroded structure are both 15 pixels;
[0046] The grayscale image after the opening operation is subjected to pyramid-based template matching. First, a crosshair area formed by intersecting laser lines is intercepted from the image as a template. Template matching is performed using a pyramid layer-by-layer search. Each slide generates a source image of the same size as the template. The similarity between the template and the source image is measured according to formula (4). If formula (4) holds, it indicates that the matching is impossible, and the matching is terminated in advance.
[0047]
[0048] In the formula, k j is the sum of the cosine values of the angle differences between the template and the first j edge points of the source image, k min is the similarity threshold, n is the number of edge points in the template, θ i is the angle difference between the template and the i-th edge point of the source image;
[0049] After the template traverses the entire image, multiple source images are obtained; the source image with the greatest similarity to the template is used as the template matching result, and the template matching result is positionally corrected to obtain the ROI area; through Blob analysis, all pixel points in the ROI area are screened according to the area and major axis length of the connected domain. When the area and major axis length of the connected domain are both greater than their respective thresholds, all pixel points in the connected domain are retained. For all retained pixel points, the two pixel points with larger coordinates in the long side direction of the steel grating are used for straight line fitting to obtain a straight line, which is the edge sealing of the steel grating. Perpendicular lines are drawn from the remaining pixel points to the straight line, and the foot of the perpendicular is the point to be welded, thereby obtaining the pixel coordinates of each point to be welded.
[0050] Step 3: According to the pixel coordinates of the points to be welded, the depth information of the points to be welded is obtained from the depth image of the steel grating; according to the depth information of the points to be welded and the intrinsic parameter matrix of the camera, the position of the points to be welded in the camera coordinate system is obtained; according to the hand-eye calibration matrix, the position of the points to be welded in the robot coordinate system is obtained, and the robot performs the welding operation according to the position of the points to be welded in the robot coordinate system.
[0051] Any matters not described in the present invention are applicable to the prior art.
Claims
1. A detection method for identifying and locating welds of steel gratings, characterized in that: The method comprises the following steps: Step 1: Project laser onto the steel grating and collect the grayscale image and depth image of the steel grating under the laser projection condition; Step 2: Based on the grayscale image of the steel grating, use the weld detection model to perform weld detection and obtain the pixel coordinates of the welds to be welded; In the welding spot detection model, the grayscale image of the steel grating is filtered, contrast enhanced, binarized, opened and matched with shape templates to obtain the ROI area. Through Blob analysis, all the pixels in the ROI area are screened according to the area and long axis length of the connected domain. When the area and long axis length of the connected domain are greater than their respective thresholds, all the pixels in the connected domain are retained. For all the retained pixels, the two pixels with larger coordinates in the long side direction of the steel grating are used for straight line fitting to obtain a straight line. The remaining pixels are perpendicular to the straight line, and the foot of the perpendicular is the point to be welded, thereby obtaining the pixel coordinates of each point to be welded. For the binarization process, firstly, the grayscale values of all pixels in the row and column where a certain pixel is located are extracted, and the OSTU thresholds of all extracted pixels are calculated using the traditional OSTU algorithm; then, the adaptive threshold fine-tuning amount P is calculated; finally, the OSTU threshold is added to the adaptive threshold fine-tuning amount to obtain the binarization threshold of the pixel, and the pixel is binarized according to the binarization threshold; the above process is repeated to binarize all pixels; In the formula, Q max and Q min is the maximum and minimum grayscale values of the row and column where a pixel is located, P max and P min is the threshold fine-tuning amount of the image under the strongest and weakest illumination, which is an empirical value; x0 is a constant; Step 3: According to the pixel coordinates of the welding point and the depth information of the welding point obtained from the depth image of the steel grating, the position of the welding point in the robot coordinate system is obtained through coordinate conversion, and the robot performs the welding operation according to this position.
2. A detection method for identifying and locating welds of steel gratings according to claim 1, characterized in that: The shape template matching includes: firstly, a cross-line area formed by intersecting laser lines is cut out from the image as a template, and the template matching is performed by layer-by-layer pyramid search. For each source image generated by sliding, the similarity between the template and the source image is measured according to formula (4). If formula (4) is established, it indicates that the matching is impossible, and the matching is terminated in advance. In the formula, k j is the sum of the cosine values of the angle differences between the template and the first j edge points of the source image, k min is the similarity threshold, n is the number of edge points in the template, θ i is the angle difference between the template and the i-th edge point of the source image.
3. A detection method for identifying and locating a steel grating weld according to claim 1 or 2, characterized in that: In the first step, two line lasers are symmetrically distributed, one projecting laser along the long side of the steel grating and tilted downward at 45°, and the other line laser is located above the steel grating and projects laser in a direction perpendicular to the grid.
4. A detection device for identifying and locating the weld seam of a steel grating, which uses the method of claim 1 to identify and locate the weld seam of a steel grating, characterized in that: The device includes a detection system, a laser projection module, an action execution module and an image acquisition module; the laser projection module is located at the end of the action execution module, the image acquisition module is located on the laser projection module, and the laser projection module, the action execution module and the image acquisition module are all communicatively connected with the detection system.
5. A detection device for identifying and locating welds of steel gratings according to claim 4, characterized in that: The laser projection module includes three line lasers, the image acquisition module is an industrial camera, and the action execution module is a six-axis robotic arm with a welding gun.
Citation Information
Patent Citations
Weld bead trajectory and attitude real-time tracking detecting method, electronic equipment and medium
CN109822216A
Cylindrical surface longitudinal weld seam feature area extraction and weld seam tracking method and system based on structured light
CN110064819A
Finger vein feature recognition device for intelligent blood sampling and recognition method thereof
CN112883824A
Portable high-speed turnout detection trolley and method based on line laser technology
CN115077423A
Welding laser visual image segmentation method, electronic equipment and storage medium
CN116275388A
Cited By
Welding method and system suitable for multi-dimensional force sensor
CN120551663A
A welding method and system suitable for multi-dimensional force sensors
CN120551663B
