Welding gun conveying posture acquisition system of binocular vision and IMU (Inertial Measurement Unit) composite system

Abstract

The invention discloses a welding gun conveying posture collecting system of a binocular vision and IMU composite system. The welding gun conveying posture collecting system of the binocular vision and IMU composite system comprises a welding gun, a fixing clamp, an IMU sensor, a feature marker, a binocular camera, an arc light baffle and a computer. The welding gun is located on the welding gun frame or in the hand of a welding operator, the fixing clamp is arranged on the welding gun, the IMU sensor and the feature marker are fixed above the welding gun through the fixing clamp, the binocular camera is installed in front of the welding gun, the arc light baffle is installed between the binocular camera and the welding gun, and the computer is in communication with the IMU sensor and the binocular camera through data lines. A welding gun posture calculation algorithm, an image processing algorithm and a three-dimensional coordinate reconstruction algorithm are stored in the computer. According to the welding gun conveying posture acquisition system of the binocular vision and IMU composite system, special mark points and a matched recognition algorithm are designed according to the welding environment, and the mark points are recognized and positioned through RGB colors and geometrical shapes.

Description

Technical Field

[0001] This invention belongs to the field of welding, and in particular relates to a welding torch movement posture acquisition system of binocular vision and IMU composite system. Background Technology

[0002] Currently, welding robots used in industrial production are mainly "teach-and-playback robots," whose welding paths and process parameters are preset, and welding quality can only be guaranteed by strictly controlling the working conditions. However, in actual welding processes, when faced with factors such as workpiece machining accuracy errors and workpiece deformation due to stress and heat, existing welding robots lack the ability to interact and adjust with the external environment and molten pool in real time, thus failing to adjust the welding process parameters and leading to welding defects.

[0003] However, skilled human welders can comprehensively assess the current stable state of the molten pool and the weld formation characteristics formed after solidification based on information such as the penetration of the molten pool, the three-dimensional morphology changes of the groove, and the dynamics of the electric arc. They can also dynamically adjust the molten pool by changing the welding torch posture to avoid the generation of welding defects.

[0004] Therefore, there is an urgent need to design a welding torch movement posture acquisition system that combines binocular vision and IMU to solve the problems mentioned above. Summary of the Invention

[0005] To address the technical problem mentioned in the background art, where existing welding robots lack the ability to interact and adjust with the external environment and molten pool in real time, thus preventing the adjustment of welding process parameters and leading to welding defects, a welder's torch posture acquisition system based on binocular vision and IMU is provided to solve the problem of interfering with the welder's normal welding process.

[0006] To achieve the above objectives, the specific technical solution of the welder's gun movement posture acquisition system of the binocular vision and IMU composite system of the present invention is as follows: A welding torch posture acquisition system combining binocular vision and IMU includes a welding torch, a fixing fixture, an IMU sensor, feature markers, a binocular camera, an arc light baffle, and a computer. The welding torch is located on the torch holder or in the hand of the welding operator. The fixing clamp is set on the welding torch. The IMU sensor and feature markers are fixed above the welding torch by the fixing clamp. The binocular camera is installed in front of the welding torch. The arc baffle is installed between the binocular camera and the welding torch. The computer communicates with the IMU sensor and the binocular camera through a data cable. The computer stores welding torch attitude calculation algorithm, image processing algorithm and three-dimensional coordinate reconstruction algorithm.

[0007] Furthermore, before welding begins, the welding torch is placed on the torch holder and kept in place.

[0008] Furthermore, the fixing fixture is fixed to the welding torch nozzle by a pipe clamp bracket, with the top of the pipe clamp bracket and the top of the welding torch nozzle being flush.

[0009] Furthermore, the IMU sensor is fixed on a fixture, and its Z-axis coordinate system is parallel to the central axis of the welding torch nozzle.

[0010] Furthermore, the feature markers are affixed to the fixing fixture and consist of patterns of specific shapes and spacing, with their longitudinal axis coinciding with the central axis of the welding torch nozzle.

[0011] Furthermore, the binocular camera consists of a left camera and a right camera spaced apart, with the left and right cameras placed horizontally.

[0012] Furthermore, the left and right cameras are oriented perpendicular to the weld direction and at the same height as the feature marker. During the welding process, the feature marker is located at the overlapping field of view of the left and right cameras throughout the entire process.

[0013] Furthermore, the arc light baffle is located between the binocular camera and the weld bead, blocking the welding arc light from directly hitting the binocular camera throughout the welding process.

[0014] Furthermore, the welding torch's angular velocity and triaxial acceleration collected by the IMU sensor are used to obtain the welding torch's two-axis attitude angles pitch and roll.

[0015] The welder's gun movement posture acquisition system of the binocular vision and IMU composite system of the present invention has the following advantages: The position and attitude information of the welding torch is collected by a combination of binocular camera and IMU system. A special fixture was designed to fix the IMU and visual markers to the welding torch without affecting the welder's operation. According to the welding environment, special markers and matching recognition algorithms were designed to identify and locate the markers through RGB color and geometric shape. An IMU module was also introduced to obtain welding torch attitude with higher accuracy. Attached Figure Description

[0016] Figure 1 This is a flowchart of the welder's gun movement posture acquisition system of the binocular vision and IMU composite system of the present invention; Figure 2 This is a structural diagram of the welder's gun movement posture acquisition system of the binocular vision and IMU composite system of the present invention; Figure 3 This is a schematic diagram showing the position and field of view of the binocular camera in the welder's gun movement posture acquisition system of the binocular vision and IMU composite system of the present invention. Figure 4 This is a three-dimensional coordinate-time diagram of the weld seam and the tip of the welding torch in the welder's torch posture acquisition system of the binocular vision and IMU composite system of the present invention. Figure 5 This is a schematic diagram of the time-welding gun attitude angle output by the welding gun attitude acquisition system of the binocular vision and IMU composite system of the present invention.

[0017] Explanation of markings in the diagram: 1. Computer; 2. Welding torch; 3. Pipe clamp support; 4. Nozzle; 5. IMU sensor; 6. Feature marker; 7. Binocular camera; 8. Fixture; 9. Arc baffle. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0020] The following is a reference to the appendix. Figure 1 To be continued Figure 5 This invention describes a welding torch movement posture acquisition system based on a binocular vision and IMU composite system.

[0021] like Figure 1 and Figure 2 As shown, the welding torch posture acquisition system of the binocular vision and IMU composite system of the present invention includes a welding torch 2, a fixing fixture 8, an IMU sensor 5, a feature marker 6, a binocular camera 7, an arc light baffle 9, and a computer 1. The welding torch 2 is located on the welding torch holder or in the hand of the welding operator. The fixing fixture 8 is set on the welding torch 2. The IMU sensor 5 and the feature marker 6 are fixed above the welding torch 2 by the fixing fixture 8. The binocular camera 7 is installed in front of the welding torch 2. The arc light baffle 9 is installed between the binocular camera 7 and the welding torch 2. The computer 1 communicates with the IMU sensor 5 and the binocular camera 7 through a data cable. The computer 1 stores the welding torch posture calculation algorithm, the image processing algorithm, and the three-dimensional coordinate reconstruction algorithm.

[0022] Furthermore, such as Figure 2As shown, before welding begins, the welding torch 2 is placed on the welding torch holder and kept fixed; the fixing fixture 8 is fixed to the welding torch nozzle 4 by the pipe clamp bracket 3, and the top of the pipe clamp bracket 3 is flush with the top of the welding torch nozzle 4; the IMU sensor 5 is fixed on the fixing fixture 8, and its Z-axis coordinate system is parallel to the central axis of the welding torch nozzle 4.

[0023] Preferably, the welding torch 2 needs to be placed on the welding torch holder before welding begins, and the welder operates the welding torch 2 to perform welding after welding begins; the nozzle 4 is part of the welding torch 2, and the nozzle 4 is a detachable structure used to control the direction of wire feeding and gas delivery of the welding torch 2.

[0024] The fixing fixture 8 is fixed to the nozzle 4 by the pipe clamp bracket 3, and the upper end of the pipe clamp bracket 3 is flush with the upper end of the nozzle 4. The feature marker 6 is attached to the fixing fixture 8, and its longitudinal center line is the same straight line as the central axis of the nozzle 4.

[0025] The IMU sensor 5 is fixed to the fixing fixture 8 with screws, and the binocular camera 7 is fixed to the welding platform with a bracket, and their positions must meet certain requirements.

[0026] Furthermore, such as Figure 2 As shown, the feature marker 6 is attached to the fixed fixture 8 and consists of a pattern of a specific shape and spacing. Its longitudinal axis coincides with the central axis of the welding torch nozzle 4. The binocular camera 7 consists of a left camera and a right camera with a fixed distance of 12cm. It is placed horizontally, oriented perpendicular to the direction of the weld bead, and its height is as consistent as possible with the feature marker 6. The distance between it and the weld bead is maintained at 30-60cm. During the welding process, the feature marker 6 must be located at the overlapping field of view of the left camera and the right camera throughout the entire process.

[0027] The shape of the arc baffle 9 is not fixed and is determined by the welding conditions, but it must ensure that it blocks the direct welding arc throughout the welding process and does not block the feature marker 6.

[0028] Furthermore, such as Figures 2 to 5 As shown, the arc light baffle 9 is located between the binocular camera 7 and the weld bead, and blocks the welding arc light from directly hitting the binocular camera 7 throughout the welding process; the angular velocity and triaxial acceleration of the welding torch 2 collected by the IMU sensor 5 are used to obtain the two-axis attitude angles pitch and roll of the welding torch 2.

[0029] In a preferred embodiment, before welding, the system is installed on the welding platform according to its position, and the welding torch 2 must be placed on the welding torch holder and kept stationary. Then, the welder posture acquisition system is started on computer 1. IMU sensor 5 and binocular camera 7 begin operation. IMU sensor 5 acquires the angular velocity and three-axis acceleration of the welding torch 2's movement through a gyroscope and accelerometer, and outputs a high-precision timestamp through a crystal oscillator timer. The left and right cameras of binocular camera 7 simultaneously capture images.

[0030] Computer 1 records the angular velocity output by the gyroscope of IMU sensor 5 in a stationary state before welding begins, and records it as the gyroscope's zero bias. Then the welder begins to operate welding torch 2 to weld. During the welding process, the data acquisition system runs continuously and constantly uploads data to computer 1.

[0031] After welding, the data collected by the binocular camera 7 and the IMU sensor is processed by computer 1. The images captured simultaneously by the left and right cameras of the binocular camera 7 are processed as follows: First, by finding the largest color block within a specific RGB range, the region containing the feature marker 6 is selected, cropped, rotated, and then designated as the Region of Interest (ROI). Then, corner detection is performed on the ROI to detect feature points on the feature marker 6. Finally, the feature points of the left and right images captured simultaneously are matched according to the permutation matrix and the Y-coordinate in the pixel coordinate system.

[0032] After matching, the depth value Z of each feature point is further calculated by calculating their disparity, i.e., the difference in X coordinate values ​​in the pixel coordinate system. Finally, the X and Y coordinates of the feature points in the camera coordinate system are calculated by normalizing the plane, and the coordinates of the tip of the welding torch 2 are calculated based on the positional relationship of the feature points. The angular velocity output by IMU sensor 5 is subtracted from the zero bias, and then pre-integrated with time. The integration result is Kalman filtered with the gravity direction determined by the three-axis acceleration to obtain the two-dimensional attitude angles roll and pitch.

[0033] Furthermore, computer 1 synchronizes the IMU sensor 5 and the binocular camera 7 via CPU time control. This allows for the calculation of the three-dimensional coordinates and two-axis attitude angles of the welding torch 2, thus determining the welder's torch-moving posture at each moment.

[0034] In the preferred embodiment, under the conditions of 60° bevel, 1.5mm gap, and 45° oblique butt welding of DH36 ship plate steel, the position and attitude angle of the welding torch were collected when the welder held the welding torch 2. The welding parameters were: MIG welding, welding current 120A, shielding gas 98%Ar+2%CO2, and gas flow rate 14L / min.

[0035] The image processing algorithm of the welding torch posture acquisition system of the binocular vision and IMU composite system in this invention is used to identify and process the image of the feature marker 6 acquired by the binocular camera 7, and obtain the pixel coordinates of the feature points on the feature marker 6.

[0036] The pixel coordinates of the feature points on the feature marker 6 are obtained by using a 3D reconstruction algorithm for image processing, thus obtaining the 3D coordinates of the feature points on the feature marker 6 and the tip of the welding torch 2.

[0037] This invention relates to a welding torch posture acquisition system based on a binocular vision and IMU composite system. It acquires the position and posture information of the welding torch 2 using a binocular camera 7 and an IMU composite system. A special fixture is designed to fix the IMU and visual markers to the welding torch 2 without affecting the welder's operation. Based on the welding environment, special markers and a matching recognition algorithm are designed to identify and locate the markers using RGB color and geometric shape. Furthermore, an IMU module is introduced to obtain more accurate welding torch 2 posture data.

[0038] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A welding torch movement posture acquisition system based on a binocular vision and IMU composite system, characterized in that, Includes welding torch, fixture, IMU sensor, feature marker, binocular camera, arc blind, and computer; The welding torch is located on the torch holder or in the hand of the welding operator. The fixing clamp is set on the welding torch. The IMU sensor and feature markers are fixed above the welding torch by the fixing clamp. The binocular camera is installed in front of the welding torch. The arc baffle is installed between the binocular camera and the welding torch. The computer communicates with the IMU sensor and the binocular camera through a data cable. The computer stores welding torch attitude calculation algorithm, image processing algorithm and three-dimensional coordinate reconstruction algorithm.

2. The welder's torch movement attitude acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, Before welding begins, the welding torch is placed on the torch holder and kept in place.

3. The welder's torch movement attitude acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, The fixing fixture is fixed to the welding torch nozzle by the pipe clamp bracket, and the top of the pipe clamp bracket is flush with the top of the welding torch nozzle.

4. The welder's torch movement attitude acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, The IMU sensor is fixed on a fixture, and its Z-axis coordinate system is parallel to the central axis of the welding torch nozzle.

5. The welder's gun movement posture acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, The feature markers are affixed to the fixed fixture and consist of patterns of specific shapes and spacing, with their longitudinal axis coinciding with the central axis of the welding torch nozzle.

6. The welder's torch movement attitude acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, The binocular camera consists of a left camera and a right camera spaced apart, with the left and right cameras placed horizontally.

7. The welder's torch movement attitude acquisition system of the binocular vision and IMU composite system according to claim 6, characterized in that, The left and right cameras are oriented perpendicular to the weld direction and at the same height as the feature marker. During the welding process, the feature marker is located at the overlapping field of view of the left and right cameras throughout the entire process.

8. The welder's torch movement attitude acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, The arc light baffle is located between the binocular camera and the weld bead, blocking the welding arc light from directly hitting the binocular camera throughout the welding process.

9. The welder's gun movement posture acquisition system of the binocular vision and IMU composite system according to claim 1, characterized in that, The welding torch's angular velocity and triaxial acceleration are collected by the IMU sensor, which yields the welding torch's two-axis attitude angles: pitch and roll.

Images