Intelligent welding guiding method and device based on dynamic vision deviation correction and medium

Abstract

The invention relates to an intelligent welding guiding method and device based on dynamic vision correction and a medium. The guiding method comprises the steps that a depth map and a grey-scale map are obtained; segmenting the welding gun to obtain a mask of the welding gun and a point cloud of the welding gun; through an ICP algorithm, the poses of the welding gun in a camera coordinate system and a world coordinate system are obtained through the point cloud of the welding gun and a preloaded welding gun CAD model; segmenting a to-be-welded workpiece by the instance to obtain a mask of the workpiece, and calculating to obtain a point cloud of the workpiece; acquiring poses of the workpiece in a camera coordinate system and a world coordinate system through an ICP (Inductively Coupled Plasma) algorithm by utilizing the point cloud of the workpiece and a preloaded workpiece CAD (Computer Aided Design) model; and the difference between the welding wire position on the welding gun and the workpiece target welding spot position is obtained and serves as a deviation correction value. Compared with the prior art, the actual alignment capability of the intelligent welding guide system on the welding gun welding wire and the workpiece welding spot is improved, and the alignment effect in the real world is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent welding in 3D vision, and particularly to an intelligent welding guidance method, device, and medium based on dynamic vision correction. Background Art

[0002] In terms of the actual alignment ability of the intelligent welding guidance system for the welding torch wire and the workpiece welding point, the prior art has at least one of the following defects: Precision limitation: The prior art has certain precision limitations in the actual alignment ability. Due to various factors interfering during the welding process, such as welding fumes, light changes, etc., the actual alignment precision of the welding torch wire and the workpiece welding point is limited. Lack of real-time performance: The prior art has certain deficiencies in real-time performance. During the welding process, the positions of the welding torch wire and the workpiece welding point may change, and real-time adjustment of alignment is required, but the prior art still needs to be improved in terms of real-time performance. Poor adaptability to complex environments: During the welding process, the environment may be complex and changeable, such as the complex shape of the welded workpiece and the difficult-to-access welding position. These complex environments pose challenges to the actual alignment ability of the intelligent welding guidance system for the welding point. Insufficient system stability: The prior art has certain deficiencies in system stability. Due to factors such as vibration and temperature changes during the welding process, the stability of the intelligent welding guidance system has a greater impact on the actual alignment ability. Poor adaptability to different welding tasks: The prior art has certain differences in adaptability to different welding tasks. Different welding tasks may require different alignment methods and algorithms, and the flexibility and adaptability of the prior art in this regard still need to be improved. Summary of the Invention

[0003] The purpose of the present invention is to overcome the above-mentioned defects existing in the prior art and provide an intelligent welding guidance method, device, and medium based on dynamic vision correction. Through dynamic vision correction, the actual alignment ability of the intelligent welding guidance system for the welding torch wire and the workpiece welding point is improved, and the alignment effect in the real world is enhanced.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] The first aspect of the present invention provides an intelligent welding guidance method based on dynamic vision correction, including the following steps:

[0006] S1: Obtain the depth map and grayscale map in the current scene output by a 3D camera with the eye outside the hand;

[0007] S2: Instance segment the welding torch in the grayscale map to obtain the mask of the welding torch, and calculate the point cloud of the welding torch;

[0008] S3: Using the ICP algorithm and the point cloud of the welding torch and the pre-loaded CAD model of the welding torch, obtain the pose of the welding torch in the camera coordinate system and the world coordinate system;

[0009] S4: Instance segment the workpiece to be welded in the grayscale image to obtain the mask of the workpiece, and calculate the point cloud of the workpiece;

[0010] S5: Using the ICP algorithm and the point cloud of the workpiece and the pre-loaded CAD model of the workpiece, obtain the pose of the workpiece in the camera coordinate system and the world coordinate system;

[0011] S6: Obtain the difference between the position of the welding wire on the welding torch and the position of the target welding point on the workpiece, and use it as the deviation correction value. Based on the deviation correction value, output a guiding instruction to guide the welding wire on the welding torch to move to the actual welding point position.

[0012] Further, in S1, the off-axis 3D camera includes a structured light camera and an adjustable bracket connected to the structured light camera.

[0013] Further, in S1, the structured light camera is used to obtain a depth map and obtain a de-distorted grayscale image through processing. The pixels in the depth map and the de-distorted grayscale image correspond to each other. By processing the depth map, the depth information can be mapped onto the grayscale image to more intuitively observe the texture and details of the scene. In this way, the depth map and the grayscale image can be obtained simultaneously, so as to perform subsequent image processing and analysis.

[0014] Further, in S2, the point cloud of the welding torch is specifically obtained by the following method:

[0015] Obtain the segmentation mask of the welding torch from the grayscale image through a visual segmentation algorithm, then use the mask to filter and optimize the depth map, and finally generate the optimized point cloud of the welding torch using the internal parameters of the camera.

[0016] Further, in S3, the process of obtaining the pose of the welding torch in the camera coordinate system and the world coordinate system includes:

[0017] Through the iterative closest point ICP algorithm and based on the CAD model of the welding torch, and then continuously iterate and optimize the alignment with the point cloud of the welding torch until the iteration termination condition is reached, to obtain the final estimated pose of the welding torch.

[0018] Further, in S4, the point cloud of the workpiece is specifically obtained by the following method:

[0019] Obtain the segmentation mask of the workpiece from the grayscale image through a visual segmentation algorithm, then use the mask to filter and optimize the depth map, and finally generate the optimized point cloud of the workpiece using the internal parameters of the camera.

[0020] Further, in S5, the process of obtaining the poses of the workpiece in the camera coordinate system and the world coordinate system includes:

[0021] By using the Iterative Closest Point (ICP) algorithm and based on the CAD model of the workpiece, and then continuously iterating and optimizing the alignment with the workpiece point cloud until the iteration termination condition is reached, the final estimated pose of the workpiece is obtained.

[0022] Further, in S6, the method for obtaining the deviation correction value is:

[0023] Based on the estimated poses of the welding torch and the workpiece, the positions of the welding wire on the welding torch and the target welding point of the workpiece in the world coordinate system are calculated respectively, so as to obtain the spatial position difference between the welding wire on the welding torch and the target welding point of the workpiece in the world coordinate system, and a guiding deviation correction value for the displacement of the welding torch is generated based on this spatial position difference.

[0024] The second aspect of the present invention provides an electronic device, including a memory and a processor, and the processor is used to execute the program in the memory to implement the intelligent welding guidance method based on dynamic vision deviation correction as described above.

[0025] The third aspect of the present invention provides a storage medium containing computer-executable instructions, and when the computer-executable instructions in the storage medium are executed by a computer processor, they are used to execute the intelligent welding guidance method based on dynamic vision deviation correction as described above.

[0026] Compared with the prior art, the present invention has the following technical advantages:

[0027] 1) Improve the precision and accuracy of welding: By the method of dynamic vision deviation correction, the pose information of the welding torch and the workpiece can be obtained in real time, and the welding torch can be guided to move to the actual welding point position according to the deviation correction value, thereby improving the precision and accuracy of welding.

[0028] 2) Improve the welding efficiency: By the automated guidance method, the workload of welding operators can be reduced, and the welding efficiency can be improved.

[0029] 3) Suitable for complex environments: This method can adapt to welding tasks in different scenarios, including cases where the shape of the welded workpiece is complex and the welding environment is complex, etc., and the flexibility and adaptability are significantly improved. Description of the Drawings

[0030] Figure 1 It is a flow schematic diagram of the intelligent welding guidance method based on dynamic vision deviation correction in the present invention. Detailed Embodiments

[0031] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. In the technical solution, features such as component models, material names, connection structures, control methods, algorithms, etc. that are not clearly described are regarded as common technical features disclosed in the prior art.

[0032] Embodiment 1

[0033] Step 1. Refer to Figure 1 , the present invention uses a structured light camera to obtain a depth map and a rectified grayscale map, with pixels corresponding to each other. The function of Step 1 is to obtain the depth map and grayscale map in the current scene output by the 3D camera with the eye outside the hand. The depth map is used to obtain the distance information of objects in the scene, while the grayscale map is used for subsequent instance segmentation and image processing. These image data will be used for the subsequent positioning and pose estimation of the welding torch and the workpiece.

[0034] Step 2. The present invention uses a segmentation algorithm to obtain the segmentation mask M of the welding torch from the grayscale map welder . Then, the depth map D is filtered using the mask, and finally, the optimized point cloud is generated using the internal parameters K of the camera. The function of Step 2 is to instance-segment the welding torch in the grayscale map, obtain the mask of the welding torch, and calculate the point cloud of the welding torch. Through the instance segmentation algorithm, the welding torch can be segmented from the grayscale map to obtain the binary mask of the welding torch, which is used for subsequent welding torch positioning and pose estimation. At the same time, through the depth map and the binary mask of the welding torch, the point cloud of the welding torch can be calculated for subsequent point cloud registration and pose estimation. These data will be used for subsequent welding torch guidance and deviation correction operations.

[0035] Step 3. In the iterative closest point (ICP) algorithm of the present invention, the CAD model of the welding torch is used, and then through continuous iterative optimization of the alignment with the point cloud of the welding torch, the final estimated pose of the welding torch is obtained. The function of Step 3 is to obtain the pose of the welding torch in the camera coordinate system and the world coordinate system through the ICP algorithm and using the point cloud of the welding torch and the pre-loaded CAD model of the welding torch. The ICP algorithm is an iterative closest point algorithm used to register two point clouds, that is, to find the best transformation relationship between them. In this step, by registering the point cloud of the welding torch with the pre-loaded CAD model of the welding torch, the pose information of the welding torch in the camera coordinate system and the world coordinate system can be obtained. These pose information will be used for subsequent welding torch guidance and deviation correction operations to ensure the accurate position and pose of the welding torch.

[0036] Step 4. The present invention uses a segmentation algorithm to obtain the segmentation mask M of the workpiece from the grayscale map obj。Then, filter the depth map D using the mask, and finally generate the optimized point cloud using the camera's internal parameters K. The function of Step 4 is to perform instance segmentation on the workpiece to be welded in the grayscale image, obtain the mask of the workpiece, and calculate the point cloud of the workpiece. Through the instance segmentation algorithm, the workpiece to be welded can be segmented from the grayscale image to obtain the binary mask of the workpiece, which is used for subsequent workpiece positioning and pose estimation. At the same time, through the depth map and the binary mask of the workpiece, the point cloud of the workpiece can be calculated, which is used for subsequent point cloud registration and pose estimation. These data will be used for subsequent torch guidance and deviation correction operations to ensure the accurate position and pose of the welding point.

[0037] The purpose of filtering and optimizing the depth map using the mask is to remove the areas in the depth map that are irrelevant to the torch or the workpiece, so as to obtain the optimized point cloud of the torch or the workpiece. The specific steps are as follows: First, obtain the segmentation mask of the torch or the workpiece from the grayscale image through the instance segmentation algorithm. The segmentation mask is a binary image, in which the area of the torch or the workpiece is marked as the foreground (white), and other areas are marked as the background (black). Next, apply the segmentation mask to the depth map and set the areas in the depth map that are irrelevant to the torch or the workpiece to invalid values (such as setting to 0 or NaN). In this way, only the areas related to the torch or the workpiece are retained in the depth map. Finally, according to the camera's internal parameters, convert the optimized depth map into the point cloud of the torch or the workpiece. By converting each pixel point in the depth map into a three-dimensional point in the camera coordinate system, the point cloud data of the torch or the workpiece can be obtained.

[0038] Through the above steps, filtering and optimizing the depth map using the mask can obtain the optimized point cloud of the torch or the workpiece, thus providing an accurate data basis for subsequent pose estimation and deviation correction operations.

[0039] Step 5. In the iterative closest point (ICP) algorithm of the present invention, the CAD model of the workpiece is used, and then the final estimated pose of the workpiece is obtained by continuously iteratively optimizing the alignment with the workpiece point cloud. In the ICP algorithm, the pose estimated using the ArUco code is used as the initial pose of the workpiece, and then the final estimated pose of the workpiece is obtained by continuously iteratively optimizing. The function of Step 5 is to obtain the pose of the workpiece in the camera coordinate system and the world coordinate system through the ICP algorithm and using the point cloud of the workpiece and the pre-loaded CAD model of the workpiece. Similar to Step 3, by registering the point cloud of the workpiece with the pre-loaded CAD model of the workpiece through the ICP algorithm, the pose information of the workpiece in the camera coordinate system and the world coordinate system can be obtained. These pose information will be used for subsequent torch guidance and deviation correction operations to ensure the accurate position and pose of the welding point.

[0040] Step 6. Based on the estimated poses of the welding torch and the workpiece, the present invention calculates the positions of the welding wire and the target welding point of the workpiece in the world coordinate system, as well as the difference between the two. By using the difference as the deviation correction value, the robotic arm is guided to move the welding wire to the welding point. The function of Step 6 is to perform welding torch guidance and deviation correction operations according to the pose information of the welding torch and the workpiece. According to the pose information of the welding torch and the workpiece, the relative pose relationship of the welding torch with respect to the workpiece can be calculated. By controlling the movement of the welding torch, the guidance of the welding torch can be achieved, enabling it to perform welding operations along a predetermined path and pose. At the same time, according to the pose information of the welding torch and the workpiece, deviation correction operations can be carried out, that is, according to the deviation between the actual welding point and the predetermined welding point, the position and pose of the welding torch are adjusted to ensure the accuracy and quality of welding. These operations are used to achieve an automated welding process.

[0041] This embodiment also proposes an intelligent welding guidance device based on dynamic vision correction. The device includes a processor and a memory, which are coupled. The memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the above task management method is implemented. The processor can be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it can also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components; the memory may include a random access memory (RAM for short), and may also include a non-volatile memory, such as at least one disk memory. The memory can be an internal memory of the random access memory (RAM) type, and the processor and the memory can be integrated into one or more independent circuits or hardware, such as: an application specific integrated circuit (ASIC). It should be noted that when the computer program in the above memory is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention.

[0042] This embodiment also provides a computer-readable storage medium, which stores computer instructions for causing a computer to execute the above-mentioned intelligent welding guidance method based on dynamic vision correction. The storage medium can be an electronic medium, a magnetic medium, an optical medium, an electromagnetic medium, an infrared medium, or a semiconductor system or a propagation medium. The storage medium can also include semiconductor or solid-state memories, magnetic tapes, removable computer disks, random access memories (RAMs), read-only memories (ROMs), hard disks, and optical disks. The optical disks can include compact disk-read only memories (CD-ROMs), compact disk-read / write (CD-RWs), and digital versatile disks (DVDs).

[0043] The above description of the embodiments is provided to enable those of ordinary skill in the art to understand and use the invention. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative efforts. Therefore, the present invention is not limited to the above embodiments, and all improvements and modifications made by those skilled in the art without departing from the scope of the present invention as disclosed should be within the protection scope of the present invention.

Claims

1. An intelligent welding guidance method based on dynamic vision correction, characterized in that, it includes the following steps: S1: Obtain the depth map and grayscale map in the current scene; S2: Instance segment the welding torch in the grayscale map to obtain the mask of the welding torch, and calculate the point cloud of the welding torch; S3: Through the ICP algorithm, and using the point cloud of the welding torch and the pre-loaded CAD model of the welding torch, obtain the pose of the welding torch in the camera coordinate system and the world coordinate system; S4: Instance segment the workpiece to be welded in the grayscale map to obtain the mask of the workpiece, and calculate the point cloud of the workpiece; S5: Through the ICP algorithm, and using the point cloud of the workpiece and the pre-loaded CAD model of the workpiece, obtain the pose of the workpiece in the camera coordinate system and the world coordinate system; S6: Obtain the difference between the position of the welding wire on the welding torch and the position of the target welding point on the workpiece, and use it as the correction value, and output a guidance instruction based on the correction value to guide the welding wire on the welding torch to move to the actual welding point position.

2. The intelligent welding guidance method based on dynamic vision correction according to claim 1, characterized in that, in S1, obtain the depth map and grayscale map in the current scene output by an eye-in-hand 3D camera, and the eye-in-hand 3D camera includes a structured light camera and an adjustable bracket connected to the structured light camera.

3. The intelligent welding guidance method based on dynamic vision correction according to claim 2, characterized in that, in S1, the structured light camera is used to obtain the depth map and process it to obtain a de-distorted grayscale map, and the pixels in the depth map and the de-distorted grayscale map correspond to each other.

4. The intelligent welding guidance method based on dynamic vision correction according to claim 1, characterized in that, in S2, the point cloud of the welding torch is specifically obtained by the following method: Obtain the segmentation mask of the welding torch from the grayscale map through a visual segmentation algorithm, then use the mask to filter and optimize the depth map, and finally generate the optimized point cloud of the welding torch using the internal parameters of the camera.

5. The intelligent welding guidance method based on dynamic vision correction according to claim 1, characterized in that, in S3, the process of obtaining the pose of the welding torch in the camera coordinate system and the world coordinate system includes: Through the iterative closest point ICP algorithm, and based on the CAD model of the welding torch, continuously iterate and optimize the alignment with the point cloud of the welding torch until the iteration termination condition is reached, and obtain the final estimated pose of the welding torch.

6. The intelligent welding guidance method based on dynamic vision correction according to claim 1, characterized in that, in S4, the point cloud of the workpiece is specifically obtained by the following method: Obtain the segmentation mask of the workpiece from the grayscale map through a visual segmentation algorithm, then use the mask to filter and optimize the depth map, and finally generate the optimized point cloud of the workpiece using the internal parameters of the camera.

7. The intelligent welding guidance method based on dynamic vision correction according to claim 1, characterized in that, in S5, the process of obtaining the pose of the workpiece in the camera coordinate system and the world coordinate system includes: By using the Iterative Closest Point (ICP) algorithm and based on the CAD model of the workpiece, continuously iterate and optimize the alignment with the workpiece point cloud until the iteration termination condition is reached to obtain the final estimated pose of the workpiece.

8. An intelligent welding guidance method based on dynamic vision correction according to claim 1, wherein, in S6, the method for obtaining the correction value is: Based on the estimated poses of the welding torch and the workpiece, calculate the positions of the welding wire on the welding torch and the target welding point of the workpiece in the world coordinate system respectively, so as to obtain the spatial position difference between the welding wire on the welding torch and the target welding point of the workpiece in the world coordinate system, and generate a guidance correction value for the displacement of the welding torch based on this spatial position difference.

9. An electronic device, including a memory and a processor, wherein, the processor is used to execute the program in the memory to implement the intelligent welding guidance method based on dynamic vision correction according to any one of claims 1 to 8.

10. A storage medium containing computer-executable instructions, wherein, when the storage medium of the computer-executable instructions is executed by a computer processor, it is used to execute the intelligent welding guidance method based on dynamic vision correction according to any one of claims 1 to 8.

Images