Method for eliminating multi-source interference and real-time tracking of weld seam in hollow net rack ball welding process

Abstract

The present application relates to the field of automatic welding of hollow net rack ball, and the ball cannot be guaranteed to enter the equipment in the center state of the weld after loading, and the size and thickness of the ball itself have tolerances, and the welding gun needs to be adjusted in real time to follow the offset of the weld during welding, the present application provides a method for eliminating multi-source interference and tracking weld in real time during the welding process of hollow net rack ball, the video stream image captured by the line laser camera is removed by using multi-frame synthesis to remove the influence of multi-source interference generated during the welding process on the captured weld image, the weld is identified in real time, and the position of the welding gun is adjusted in real time according to the position of the weld, the multi-source interference is eliminated, the position of the weld is tracked in real time, the deviation of the welding position and the existence of defects in the weld are avoided, and the welding efficiency is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of automatic welding technology for hollow space frame spheres, specifically to a method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres. Background Technology

[0002] Space frame structures, as a new type of building structure, are widely used in large-scale construction projects. Among various space frame structures, welded spherical space frame structures are favored for their advantages such as light weight, low material consumption, and high structural rigidity, offering broad application prospects and numerous advantages. They can be widely used in construction, aerospace, rail transportation, industrial equipment, stadiums, exhibition halls, and cultural and entertainment venues. Compared with traditional space frame structures, welded spherical space frame structures have several advantages: First, they allow for modular design and manufacturing, enabling rapid assembly and disassembly, accelerating construction speed and reducing costs. Second, the spherical design of the nodes allows for the connection of steel pipes of different sizes and angles through core holes in the spheres, significantly saving materials. Finally, welded spherical space frame structures have an aesthetically pleasing, modern, and stylish appearance, and the spheres exhibit excellent performance in corrosion resistance, oxidation resistance, and other properties, making them well-suited to various climatic conditions. In conclusion, welded spherical space frame structures have a very broad application prospect, and their numerous advantages make them a leader among various space frame structures.

[0003] Currently, the manufacturing method for welded spheres involves first creating two hollow hemispheres, then assembling and welding them together, and finally performing a final weld to fill in the gaps. However, during material loading, it's impossible to guarantee that the weld seam will be centered after the sphere enters the equipment. Furthermore, the size and thickness of the sphere itself have tolerances. During assembly, the angle of the machined joint and the flatness of the assembly also have varying degrees of tolerance. Therefore, the welding torch must be adjusted in real-time during the welding process to follow any shifts in the weld seam position.

[0004] In view of the problems existing in the above welding process, a real-time weld tracking method is invented to prevent multi-source interference in the image during the welding of hollow mesh ball. It does not require manual operation and has important practical significance for improving welding efficiency and eliminating production problems such as welding position errors, weld defects, and internal cavities caused by misalignment of the welding torch and the weld. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres. This invention is mainly used to eliminate multi-source interference during the welding process of hollow space frame spheres, realize real-time tracking of weld seam position, avoid welding position deviation and weld seam defects, and effectively improve welding efficiency.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for eliminating multi-source interference and real-time weld tracking during the welding of hollow grid spheres involves using multi-frame synthesis to remove the influence of multi-source interference generated during the welding process on the captured weld images from a line laser camera. The method also includes real-time weld identification and real-time adjustment of the welding torch position based on the weld location. The method comprises the following steps:

[0008] S1. Rotate the hollow mesh sphere counterclockwise around the X-axis at a uniform speed. A line laser camera captures a 1080p or higher resolution video stream of the weld seam, which is then transmitted to the Raspberry Pi for processing. All images in the video stream are read and converted to grayscale. An image pyramid downsampling method is then used to obtain the original pixel count 1 / p. 2 The video stream image IMG1 is a multiple of the image size, where p is the scaling factor of the image's side length.

[0009] S2. After preprocessing the video stream image IMG1 obtained in step S1, we obtain the video stream image IMG2;

[0010] S3. Multi-frame synthesis of the video stream image IMG2 after pyramid downsampling and scaling by a ratio of p. By subtracting each pair of several video stream images IMG2, the fast and continuous motion parts are obtained. The contour extraction method is used to extract the contours of the fast and continuous motion parts and superimpose them together, which are marked as motion noise M.

[0011] A new thread is started simultaneously, all video stream images IMG2 are normalized, optical flow is estimated using the LK method, multi-frame images are superimposed and synthesized, the synthesized image is normalized again to prevent brightness from exceeding the limit, welding arc light interference is removed, and video stream image IMG3 is obtained.

[0012] S4. Remove the motion noise M mentioned in step S3 from the video stream image IMG3 to obtain a weld feature image that has been freed from multi-source interference. First, use the adaptive template selection method to select a suitable weld template, then use the NCC template matching method to obtain the weld feature coordinates (x1, y1), and then expand the feature template outward to twice the pixel size based on the weld feature coordinates. Finally, crop a new ROI region video stream image IMG4 on the original resolution image at a ratio of p times the pixel size.

[0013] S5. Repeat steps S2 to S3 on the newly cropped video stream image IMG4 from step S4, and remove motion noise M2 from the video stream image IMG32 after multi-frame synthesis and superposition of video stream image IMG4 according to the corresponding ratio p. Then, use the Zhang-Suen algorithm to process the video stream image IMG32 after removing motion noise M2 to produce the centerline image IMG5 of the weld feature image under line laser irradiation.

[0014] S6. Apply the NCC template matching method again to the centerline image IMG5 to match the left edge of the weld with feature coordinates (x2, y2) and the right edge of the weld with feature coordinates (x3, y3). Substitute the feature coordinates of the two edges and the overall feature coordinates of the weld mentioned in S5 into the following formula to obtain the high-precision weld feature center coordinates (x, y).

[0015] x = x1 * p + (x2 + x3) / 2

[0016] y = y1*p + (y2 + y3) / 2;

[0017] S7. Store the weld feature center coordinates (x, y) as a dictionary, with the key being the current rotation angle γ of the hollow grid sphere. N The value is the coordinate of the weld feature center (x N y N );

[0018] S8. Based on the dictionary described in step S7, at the current rotation angle γ of the hollow net ball... N Next, solve for the distance parameter Dis that the welding torch should be adjusted to:

[0019] Dis=F(D(γ N -β),r,l)

[0020] Where β is the angle between the line connecting the welding torch and the center of the ball and the line containing the laser axis of the line laser camera; D(*) is a dictionary lookup, γ N -β is the key value; r is the radius of the sphere; l is the distance from the camera to the sphere; F is the conversion function between pixel coordinates and real-world coordinates.

[0021] Furthermore, in step 1, the posture of the hollow grid ball is adjusted so that the plane containing the weld ring coincides with the vertical plane S passing through a point on the center of the ball. The welding torch is placed on plane S and directly above the hollow grid ball. The line laser camera is also placed at the intersection of plane S and the equipment frame, and is located in the clockwise direction of the welding torch. The power source for the rotation of the ball is a servo motor, and the current rotation angle is determined from the encoder.

[0022] Furthermore, in step S2, the image preprocessing includes: FFT processing to remove high-frequency noise, then using IFFT to restore the image and performing Gaussian filtering to remove image noise caused by dark current, resulting in the preprocessed image IMG2.

[0023] Furthermore, in step S3, the number of repetitions shall not be less than 2.

[0024] Furthermore, the template source for the NCC template matching method is the actual photos taken by a line laser camera of hollow grid spheres of different sizes and thicknesses. After processing in steps S2-S5, the centerline image is obtained, and after Gaussian blur processing, it is stored as a template library.

[0025] Furthermore, the adaptive template selection method is as follows: before the sphere is welded and before the sphere is rotated, three clean weld feature images with noise M removed based on steps S1-S4 are selected, and all templates in the template library are used for matching. The matching confidence of each template is added together, and the template with the highest confidence is taken as the template used for subsequent weld recognition.

[0026] Furthermore, the template in step S6 corresponds one-to-one with the template in step S4.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] (1) Compared with traditional methods, the present invention uses multi-frame synthesis to specifically remove the influence of rapidly moving slag, sparks and arc light on the image during the welding process. It can identify the weld seam in real time during the welding process and adjust the welding gun according to the weld seam position, and has a better tracking effect for multi-circle welding.

[0029] (2) The present invention uses adaptive NCC template matching, which improves the accuracy of weld identification;

[0030] (3) The image algorithm used in this invention actually identifies the left and right edges of the weld, which reduces the identification errors caused by the influence of weld points, stains and other factors in the weld of the hollow grid ball and improves the robustness of the identification.

[0031] (4) The image processing algorithm used in this invention can reduce the number of pixels in the image to be processed to 1 / p of the original image. 2 It can quickly determine the initial position of the weld; on the other hand, after initially determining the initial position of the weld, it divides a smaller ROI area and uses the original image resolution to improve the accuracy of weld position recognition. This greatly reduces the computational pressure while ensuring high-precision weld recognition, and can run smoothly on low-power processing devices such as Raspberry Pi. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the welding torch position and the line laser camera position of the present invention;

[0033] Figure 2 This is a schematic diagram of image preprocessing and Fourier transform in this invention;

[0034] Figure 3 This is a schematic diagram of the multi-frame synthesis of the dynamic and static parts of the present invention;

[0035] Figure 4 This is a schematic diagram of NCC template matching, ROI selection, and final positioning in this invention;

[0036] Figure 5 This is a schematic diagram of the present invention, showing the rotation of the sphere and the recording of the corresponding weld position coordinates.

[0037] In the diagram: 1-Welding torch; 2-Line laser camera. Detailed Implementation

[0038] The present invention will now be further described with reference to the accompanying drawings.

[0039] like Figure 1 As shown, welding torch 1 is placed on plane S and directly above the hollow mesh sphere. Line laser camera 2 is also placed at the intersection of plane S and the equipment frame, clockwise from welding torch 1. The sphere's rotation is powered by a servo motor, with the current rotation angle determined by an encoder. Ideally, the hollow mesh sphere's posture should be adjusted so that the plane containing the weld ring coincides with the vertical plane S passing through the sphere's center. However, due to the inherent tolerances of the hollow sphere, controlling its posture is difficult, and the equipment installation also has tolerances. Therefore, it is challenging to adjust the weld seam of the hollow mesh sphere to coincide with the vertical plane S passing through the sphere's center. This invention utilizes real-time weld seam image recognition to adjust the welding torch position in real-time based on the weld seam location, improving the welding efficiency of the hollow sphere and significantly reducing the impact of multi-source interference on the welding process.

[0040] This invention discloses a method for eliminating multi-source interference and real-time weld tracking during the welding of hollow mesh spheres. The interference includes: dark current interference from the welding torch, sparks and slag spatter during welding, and arc smoke with rapidly changing brightness. Since multi-source interference affects weld image recognition, the method utilizes multi-frame synthesis of video stream images captured by a line laser camera to remove the influence of multi-source interference generated during welding on the captured weld image. The method then identifies the weld in real time and adjusts the welding torch position in real time based on the weld location. The specific steps are as follows:

[0041] S1. Rotate the hollow mesh sphere counterclockwise around the X-axis at a uniform speed. A line laser camera captures a 1080p or higher resolution video stream of the weld seam, which is then transmitted to the Raspberry Pi for processing. All images in the video stream are read and converted to grayscale. An image pyramid downsampling method is then used to obtain the original pixel count 1 / p. 2 The video stream image IMG1 is a multiple of the image size, where p is the scaling factor for the side length. For example, if 1080p is downsampled to 270p, p equals 4. Figure 2 As shown in the upper part.

[0042] S2. After preprocessing the video stream image IMG1 described in S1, a preprocessed video stream image IMG2 is obtained. The preprocessing process is as follows: FFT processing is used to remove high-frequency noise, then IFFT is used to restore the image, and Gaussian filtering is performed to remove image noise caused by the dark current of the camera due to the large current interference when the welding torch is turned on. Figure 2 The lower half is shown.

[0043] S3. The video stream image IMG2, after pyramid downsampling and scaling by a factor of p, is synthesized into multiple frames. By subtracting each pair of IMG2 images, the fast and continuous motion components can be obtained. Using contour extraction, the contours of these fast and continuous motion components are extracted and superimposed, labeled as motion noise M. Figure 3 As indicated by the left arrow.

[0044] A new thread is started simultaneously to normalize the video stream image IMG2, estimate optical flow using the LK method, and perform multi-frame image overlay synthesis. The synthesized image is then normalized again to prevent brightness exceeding limits and welding arc light interference is removed to obtain the video stream image IMG3, as shown below. Figure 3 As indicated by the right arrow.

[0045] S4. Remove the motion noise M mentioned in step S3 from the video stream image IMG3 to obtain a clean weld feature image free from multi-source interference such as spatter, sparks, and smoke. First, use an adaptive template selection method to select a suitable weld template, then use the NCC template matching method to obtain the weld feature coordinates (x1, y1). Then, based on the weld feature coordinates, expand the feature template outward to twice the pixel size, and crop a new ROI region video stream image IMG4 from the original resolution image at twice the pixel ratio. Figure 4 As shown on the left. Motion noise M is a region defined by a set of coordinates, mainly used to remove the influence of weld spatter.

[0046] S5. Repeat steps S2-S3 on the newly cropped video stream image IMG4 from step S4, and remove motion noise M2 from the multi-frame composite video stream image IMG32 of IMG4 according to the corresponding ratio p. Then, process the video stream image IMG32 after removing motion noise M2 using the Zhang-Suen algorithm to produce the centerline image IMG5 of the weld feature image under laser illumination, as shown below. Figure 4 As shown. In this step, the video stream image IMG32 is synthesized from cropped high-resolution images, with a higher resolution than the video stream image IMG3 after pyramid processing. M2 is the region selected by M in S4 according to the corresponding ratio p, which can greatly reduce the amount of computation.

[0047] S6. Apply the NCC template matching method again to the centerline image IMG5, matching the left edge of the weld with feature coordinates (x2, y2) and the right edge of the weld with feature coordinates (x3, y3). Substitute the feature coordinates of the left and right edges with the overall feature coordinates of the weld described in S4 into the following formula to obtain the high-precision weld feature center coordinates (x, y), as follows. Figure 4 As shown.

[0048] x = x1 * p + (x2 + x3) / 2

[0049] y = y1*p + (y2 + y3) / 2

[0050] S7. Store the weld feature center coordinates (x, y) as a dictionary, with the key being the current rotation angle γ of the hollow grid sphere. N The value is the coordinate of the weld feature center (x N y N ),like Figure 5 As shown.

[0051] S8. Based on the dictionary described in S7, at the current rotation angle γ of the hollow net ball... N Next, solve for the distance parameter Dis that the welding torch should be adjusted to, Dis = F(D(γ) N -β),r,l); β is the angle between the line connecting welding torch 1 and the center of the ball and the line containing the laser axis of the laser camera 2; D is a dictionary lookup, (γ) N -β) is the key value; r is the radius of the sphere; l is the distance from the camera to the sphere; F is the conversion function between pixel coordinates and real coordinates.

[0052] In this invention, the template source for the NCC template matching method is the actual image taken by a line laser camera 2 of hollow grid spheres of different sizes and thicknesses. After processing in steps S2-S5, the center line image is obtained. At this time, the image already has clean weld features. Then, Gaussian blur processing is used to expand the fault tolerance range of the matching, and then it is stored as a template library.

[0053] The adaptive template selection method of the present invention is as follows: Before the sphere is welded and before the sphere is rotated, three clean weld feature images with noise M removed based on steps S1-S4 are selected, and all templates in the template library are used for matching. The matching confidence of each template is added together, and the template with the highest confidence is taken as the template used for subsequent weld recognition. The template in step S6 corresponds one-to-one with the template selected in S4, so there is no need to re-match all templates and then select.

[0054] In this embodiment, when performing multi-frame synthesis, 5 frames of images are selected for synthesis. The video stream image IMG4 is cropped from the original image in step S1 according to the proportion. If multi-frame synthesis is defined as 5 frames, the video stream image IMG4 appears from the 5th frame of the program. The first image of the video stream image IMG4 is cropped from the position determined by frames 1-5, and the second image is cropped from frames 2-6. Then, when the program runs to the 10th frame, the image IMG5 is calculated from the five images IMG4. It is equivalent to the image IMG5 being processed from the original images 5-9, and the second image IMG5 being processed from frames 6-10. However, the camera frame rate is very fast, at 30-60fps. After power-on, the program first turns on the camera and then performs mechanical movement, and the time to enter the loop processing is very short.

[0055] Image IMG3 is a composite of multiple frames from 1 to 5, appearing after the fifth frame. Image IMG4 is the region located based on IMG3, which is also confirmed based on the original frames from 1 to 5. The position of motion noise M is consistent for both images IMG3 and IMG4, only the scale p is different.

[0056] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres, characterized in that, The video stream images captured by the line laser camera are synthesized using multi-frame synthesis to remove the influence of multi-source interference generated during the welding process on the captured weld seam images. The weld seam is identified in real time, and the welding torch position is adjusted in real time according to the weld seam position. The process includes the following steps: S1. Rotate the hollow mesh sphere counterclockwise around the X-axis at a uniform speed. A line laser camera captures a 1080p or higher resolution video stream of the weld seam, which is then transmitted to the Raspberry Pi for processing. All images in the video stream are read and converted to grayscale. An image pyramid downsampling method is then used to obtain the original pixel count 1 / p. 2 The video stream image IMG1 is a multiple of the image size, where p is the scaling factor of the image's side length. S2. After preprocessing the video stream image IMG1 obtained in step S1, we obtain the video stream image IMG2; S3. Multi-frame synthesis of the video stream image IMG2 after pyramid downsampling and scaling by a ratio of p. By subtracting each pair of several video stream images IMG2, the fast and continuous motion parts are obtained. The contour extraction method is used to extract the contours of the fast and continuous motion parts and superimpose them together, which are marked as motion noise M. A new thread is started simultaneously, all video stream images IMG2 are normalized, optical flow is estimated using the LK method, multi-frame images are superimposed and synthesized, the synthesized image is normalized again to prevent brightness from exceeding the limit, welding arc light interference is removed, and video stream image IMG3 is obtained. S4. Remove the motion noise M mentioned in step S3 from the video stream image IMG3 to obtain a weld feature image that has been freed from the influence of multi-source interference. First, use the adaptive template selection method to select a suitable weld template, then use the NCC template matching method to obtain the weld feature coordinates (x1, y1), and then expand the feature template outward to twice the pixel size based on the weld feature coordinates. Finally, crop a new ROI region video stream image IMG4 on the original resolution image at a ratio of p times the pixel size. S5. Repeat steps S2 to S3 on the newly cropped video stream image IMG4 from step S4, and remove motion noise M2 from the video stream image IMG32 after multi-frame synthesis and superposition of video stream image IMG4 according to the corresponding ratio p. Then, use the Zhang-Suen algorithm to process the video stream image IMG32 after removing motion noise M2 to produce the centerline image IMG5 of the weld feature image under line laser irradiation. S6. Apply the NCC template matching method again to the centerline image IMG5 to match the left edge of the weld with feature coordinates (x2, y2) and the right edge of the weld with feature coordinates (x3, y3). Substitute the feature coordinates of the two edges and the weld feature coordinates (x1, y1) obtained in S4 into the following formula to obtain the high-precision weld feature center coordinates (x, y). x=x1 p+(x2+x3) / 2 y=y1 p+(y2+y3) / 2; S7. Store the weld feature center coordinates (x, y) as a dictionary, with the key being the current rotation angle γ of the hollow grid sphere. N The value is the coordinate of the weld feature center (x N y N ); S8. Based on the dictionary described in step S7, at the current rotation angle γ of the hollow net ball... N Next, solve for the distance parameter Dis that the welding torch should be adjusted to: Dis=F(D(γ N -b), r,l) Where β is the angle between the line connecting the welding torch (1) and the center of the sphere and the line containing the laser axis of the laser camera (2); D ( ) represents a dictionary lookup, γ N -β is the key value; r is the radius of the sphere; l is the distance from the camera to the sphere; F is the conversion function between pixel coordinates and real-world coordinates.

2. The method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres according to claim 1, characterized in that: In step 1, the posture of the hollow grid ball is adjusted so that the plane of the weld ring coincides with the vertical plane S passing through the center of the ball. The welding torch (1) is placed on plane S and directly above the hollow grid ball. The line laser camera (2) is also placed at the intersection of plane S and the equipment frame, and is located in the clockwise direction of the welding torch (1). The power source for the rotation of the ball is a servo motor, and the current rotation angle is determined from the encoder.

3. The method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres according to claim 1, characterized in that: In step S2, the image preprocessing includes: FFT processing to remove high-frequency noise, then using IFFT to restore the image and performing Gaussian filtering to remove image noise caused by dark current, resulting in the preprocessed image IMG2.

4. The method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres according to claim 1, characterized in that: In step S3, the number of repetitions shall not be less than 2.

5. The method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres according to claim 1, characterized in that: The template source for the NCC template matching method is the actual photos taken by a line laser camera (2) of hollow net frame spheres of different sizes and thicknesses. After processing in steps S2-S5, the center line image is obtained, and after Gaussian blur processing, it is stored as a template library.

6. The method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres according to claim 5, characterized in that: The adaptive template selection method is as follows: before the sphere is welded and before the sphere is rotated, three clean weld feature images with noise M removed based on steps S1-S4 are selected, and all templates in the template library are used for matching. The matching confidence of each template is added together, and the template with the highest confidence is taken as the template used for subsequent weld recognition.

7. The method for eliminating multi-source interference and real-time tracking of weld seams during the welding process of hollow space frame spheres according to claim 6, characterized in that: The template in step S6 corresponds one-to-one with the template in step S4.

Images