Omnidirectional Weld Tracking Laser Vision Sensor, Sensing Method and Welding Device

Abstract

The present invention discloses an omnidirectional weld tracking laser vision sensor, a sensing method and a welding device, which relate to the field of welding automation. The sensor includes a sensor housing, a mounting bracket, a filter, a laser and at least three cameras. The cameras are arranged around the welding torch mounting hole and are evenly distributed circumferentially. The cameras are used to capture the laser closed stripe images formed by the laser emitted by the laser. The filter is located at the front end of the camera and allows the light with the same wavelength as the laser emitted by the laser to pass through. The sensor further includes an image processing module disposed in the sensor housing, which is used to connect to the cameras and receive the image information captured by the cameras, and splice the images captured by all the cameras to form an output including a complete laser closed stripe image. The welding device applying the omnidirectional weld tracking laser vision sensor and the sensing method can realize the tracking detection of complex spatial welds, and can realize omnidirectional automatic welding without adjusting the direction of the welding torch.

Description

Technical Field

[0001] The present invention relates to the field of welding automation, and more specifically, to an all-directional weld tracking laser vision sensor, a sensing method and a welding device. Background Art

[0002] Weld tracking technology is to automatically detect and adjust the position of the welding torch during the welding process, so as to always follow the weld position for welding, thereby ensuring the welding quality, realizing adaptive control and improving the welding efficiency. The weld tracking system mainly consists of three parts: a sensor, a control system and a motion actuator. Among them, the sensor is the most important component. At present, the laser vision sensor has good application prospects. The weld tracking laser vision sensor uses the principles of optical propagation and imaging to obtain the position information of each point within the laser scanning area, and completes the on-line real-time detection of common welds through complex program algorithms. The device calculates the deviation between the detected weld and the welding torch, outputs deviation data, and the motion actuator corrects the deviation in real time to accurately guide the welding torch to automatically weld, so as to realize real-time communication with the robot control system to track the weld for welding.

[0003] At present, the commonly used laser vision sensor in industry is linear, which has the problem of being relatively large in volume and easy to collide with the workpiece. At the same time, the linear laser sensor can only be placed at the front or rear end of the welding torch and is suitable for long straight weld tracking. For example, Chinese Patent Application, Publication No. CN106984926 A, Publication Date July 28, 2017, discloses a weld tracking system and a weld tracking method. The weld tracking system includes a vision sensing device, a bracket, a welding torch, a motion mechanism, a computer and a baffle. The vision sensing device includes a dot laser, a line laser and a area array camera; the area array camera acquires an image and transmits it to the computer, and the computer issues an instruction to control the motion mechanism to move in three dimensions. Such a weld tracking sensing device has a separated position between the vision sensing device and the welding torch, forming a front-back guiding relationship on a straight line. When tracking spatial complex curves such as curved, 90° fold angle welds and circular welds, the robotic arm needs to drive the welding torch and the sensor to rotate and continuously change the posture. Especially when tracking a 90° fold angle weld, when reaching the weld fold angle, the one with the prior position among the welding torch and the sensor will reach the blank area without weld on the straight-ahead direction before the one with the posterior position. At this time, a blind area appears in the weld detection, the welding process needs to stop, wait for the welding torch and the sensor to rotate and then adjust the starting position, and start the front-back cooperation again in the new direction, so continuous welding cannot be achieved, resulting in welding quality problems.

[0004] Chinese Patent Application, Publication No. CN110153602A, discloses a multi-directional laser vision tracking sensor device and its tracking and control method. The device includes a laser vision sensor, a special-shaped welding torch, and a computer control system; the laser vision sensor is installed at the end of the manipulator, and the special-shaped welding torch is installed at the lower center of the laser vision sensor. The laser vision sensor includes a flange frame, four cameras, three lasers, and an integrated pipeline. The four cameras are evenly distributed around the upper end of the welding torch. Among them, the first to third cameras are respectively installed in pairs with the first to third lasers on the diagonal side of the flange frame to measure the front, rear, and side welds before tracking; the fourth camera is arranged in another direction to observe and monitor the weld. The multi-directional laser vision tracker can achieve the measurement and tracking of intersecting welds and realize the switching tracking of vertical, horizontal, and vertical welds. It solves the welding of intersecting welds in complex structures such as the steel plate grid structure and corrugated plates due to space limitations and easy interference at the starting and ending ends. The drawback is that this invention requires the computer control system to pre-read the workpiece drawing to obtain the theoretical data of the workpiece structure and the starting and ending positions of the welds, which is used to control the moving scan of multiple lasers and the cooperation of multiple cameras to complete image acquisition. It cannot complete the all-directional weld tracking detection in real time; in terms of the installation method, this device uses the method of hanging multiple cameras and laser modules outside the flange frame, which is flexible but not stable enough. This invention uses four different sensors to detect different directions and needs to control the switching between multiple sensors to achieve tracking, and cannot truly achieve the all-directional monitoring of an automatic fusion closed loop. The solution is too complex. Summary of the Invention

[0005] 1. Technical Problem to be Solved

[0006] Aiming at the problems that the existing weld tracking laser vision sensor is prone to collide with the workpiece and there are limitations in the detection of complex curve welds, the present invention proposes an all-directional weld tracking laser vision sensor. The lasers and cameras are arranged around the welding torch, reducing the system volume and avoiding collision with the workpiece; it can real-time collect the closed laser stripe images around the 360° working area of the welding torch, and through image comparison, it can complete the all-directional detection of complex path welds such as curved and 90° folded corner welds. During the full-automatic welding process of the above complex welds, there is no need to pause the welding to adjust the direction, improving the welding efficiency and quality.

[0007] 2. Technical Solution

[0008] The object of the present invention is achieved by the following technical solutions.

[0009] One aspect of the present invention provides an omnidirectional weld tracking laser vision sensor, which includes a sensor housing and a mounting bracket disposed inside the sensor housing. The sensor housing is a cylinder, one end of the cylinder is provided with a cylinder cover, and a welding torch mounting hole is provided on the cylinder cover and is used for sleeving a welding torch; a filter is provided at the other end of the cylinder, and a through hole is provided on the filter. During use, the welding torch is sleeved into the welding torch mounting hole, and the welding tip of the welding torch passes through the through hole on the filter, forming a mounting method in which the sensor is sleeved on the outer ring of the welding torch.

[0010] A laser and at least three cameras are provided on the mounting bracket. The cameras are evenly distributed circumferentially around the welding torch mounting hole, and the cameras are used to capture the laser closed stripe images formed by the laser emitted by the laser. During use, the welding torch has been sleeved into the sensor, and the welding torch has blocked the laser stripe formed by the laser emitted by the laser and the lens of the camera to a certain extent. At this time, it is necessary to adjust the position of the laser so that the laser stripe formed by the laser emitted by the laser on the workpiece surface or the connection of multiple laser stripes can form a closed stripe around the working area of the welding torch, that is, the laser stripe can form a closed curve in all directions of 360° around the working area of the welding torch, so that after the welding torch completes the current work, when the weld changes and bends at different angles, it intersects with the laser closed stripe, realizing omnidirectional detection of the weld.

[0011] The filter is located at the front end of the camera, and the wavelength of the light allowed to pass through is the same as the wavelength of the laser emitted by the laser. The filter can also be used to block the soot and spatter generated during the welding process and filter out the arc light interference.

[0012] An image processing module is also provided inside the cylinder. The image processing module is connected to the camera and is used to receive the image information captured by the camera, and splice the images captured by all cameras to form an output including a complete laser closed stripe image. This is mainly because in actual use, due to the blockage of the welding torch, it is difficult for each camera to capture a complete laser closed stripe image, and there is a certain overlapping area in the images captured by the three cameras. Through image processing algorithms, the pictures captured by each camera can be integrated to obtain an image including a complete laser stripe.

[0013] Furthermore, the omnidirectional weld tracking laser vision sensor further includes a cable. The cable includes a power line and a signal line, and the image processing module is connected to the camera through the signal line. The sensor also supports a power supply and signal transmission method without a cable, which can be adjusted according to the actual working scenario.

[0014] Furthermore, the method for splicing the images captured by all cameras by the image processing module using an image splicing algorithm is as follows: Use an image processing algorithm to remove noise and discrete dot patterns, and extract the pixel positions of the incomplete stripes in all images. Splice the stripes according to the obtained stripe pixel positions. Preferably, the least squares method or the like is used to find the position with the smallest variance after splicing to complete the splicing.

[0015] Preferably, the laser is a single ring laser that emits ring-shaped laser light, and the ring laser surrounds the welding torch mounting hole for one week.

[0016] In some embodiments, the laser can also be three ring lasers, which are circumferentially spaced at an angle of 120° around the welding torch mounting hole and are staggered with the camera. As long as the position, quantity, and the morphology of the laser stripes emitted by the lasers can form a laser closed stripe that surrounds the working area of the welding torch at 360° during use.

[0017] Preferably, the camera is a pinhole camera.

[0018] In some embodiments, the sensor is also provided with a light reduction system.

[0019] In the second aspect of the present invention, a full-direction weld tracking welding device is provided, which includes a welding torch, an information processing module, and an instruction execution mechanism, and also includes a full-direction weld tracking laser vision sensor according to the first aspect of the present invention. The welding torch is inserted into the welding mounting hole and passes through the through hole, and is sleeved and installed with the weld tracking sensor.

[0020] The weld tracking sensor outputs a real-time laser closed stripe image surrounding the working area of the welding torch to the information processing module;

[0021] The information processing module analyzes the weld and weld bead information based on the real-time laser closed stripe image, generates a welding torch operation control instruction, and then sends the welding torch operation control instruction to the instruction execution mechanism; the specific method is: comparing the real-time laser closed stripe image with a reference laser closed stripe image, extracting the stripe deviation characteristics, and calculating to obtain the weld and weld bead information. The reference laser closed stripe image is the laser closed stripe image formed by the laser emitted by the laser on a flat plate without a weld.

[0022] The instruction execution mechanism receives the control instruction issued by the information processing module and controls the welding torch to complete the welding work according to the instruction.

[0023] In the third aspect of the present invention, a full-direction weld tracking laser vision sensing method is also provided, which is used for a full-direction weld tracking welding device described in the second aspect of the present invention, and includes the following steps:

[0024] Step S1: Scanning a flat plate using a full-direction weld tracking laser vision sensor sleeved with a welding torch. The specific method is: placing the flat plate directly below the full-direction weld tracking laser vision sensor sleeved with a welding torch, the laser emits laser light to the surface of the flat plate, and adjusting the position of the laser to form laser stripes, so that one laser stripe or the continuation of multiple laser stripes forms a laser closed stripe around the projection area of the welding torch on the flat plate.

[0025] Step S2: The image processing module outputs the flat laser closing fringe image to the information processing module as the reference laser closing fringe image.

[0026] Step S3: Use the omnidirectional weld tracking laser vision sensor sleeved with a welding torch to scan the workpiece to be welded in real time. The laser closing fringe intersects with the weld around the working area of the welding torch. The image processing module outputs the real-time laser closing fringe image to the information processing module.

[0027] Step S4: The information processing module compares the reference laser closing fringe image and the real-time laser closing fringe image, extracts the characteristics of the fringe deviation, obtains the weld and weld bead information, and uses it to generate the welding torch operation control instruction.

[0028] Repeat steps S3 and S4 until the welding is completed.

[0029] Preferably, before the image processing module outputs the laser closing fringe image, it further includes the step of finding the intersection points of the laser fringes to obtain the outer contour of the laser closing fringe, and the output laser closing fringe image only includes the outer contour.

[0030] Furthermore, the weld and weld bead information obtained by the information processing module includes the weld information that has been welded.

[0031] 3. Beneficial effects

[0032] Compared with the prior art, the advantages of the present invention are as follows:

[0033] 1. The laser sensor of the omnidirectional weld tracking laser vision sensor cooperates with at least 3 cameras, and the emitted laser forms a laser closing fringe, forming a 360° omnidirectional closed curve around the working area of the welding torch, so that various complex welds intersect with the laser closing fringe when turning in any direction, eliminating the detection blind area when the weld direction suddenly changes; at the same time, through at least 3 cameras, the information collection of the complete laser closing fringe is completed, and the laser closing fringe image obtained after splicing can realize the omnidirectional real-time tracking detection of the weld. Even for space complex curve welds such as curved path welds and 90° broken line fillet welds, it can be detected in time. Without the need to rotate the welding torch direction by the robotic arm, the weld direction can be judged, and continuous welding can be achieved, avoiding the situation where welding has to be stopped during the process of adjusting the welding torch and the sensor, thus improving the welding quality.

[0034] 2. Compared with the existing weld tracking sensing devices and detection methods, the omnidirectional weld tracking laser vision sensor of the present invention adopts a cylindrical structure and is sleeved on the welding torch during use, forming a tightly surrounding connection relationship with the welding torch. During the working process of the welding torch, it moves synchronously with the welding torch, which can avoid the collision between the laser sensor and the workpiece.

[0035] 3. The sensor uses a pinhole camera and a ring laser, with a compact structure, surrounding the welding torch; the overall volume of the system is small, suitable for the welding operation of the flexible welding torch in a narrow space.

[0036] 4. The all-directional weld seam tracking laser vision sensing method of the present invention realizes the splicing of images taken by different cameras through an image processing algorithm, calculates the weld bead information of the weld seam by continuously comparing the reference laser closed stripe image and the real-time laser closed stripe image, can predict the trend of the weld seam, and further combines the running speed of the welding torch to achieve more accurate automatic welding control. Brief Description of the Drawings

[0037] Figure 1 It is a schematic diagram of the application scenario of the weld seam tracking sensing device in the prior art;

[0038] Figure 2 It is a schematic diagram of the structure of the all-directional weld seam tracking laser vision sensor;

[0039] Figure 3 It is a sectional view taken along line A-A of the all-directional weld seam tracking laser vision sensor;

[0040] Figure 4 It is a sectional view taken along line B-B of the all-directional weld seam tracking laser vision sensor;

[0041] Figure 5 It is a schematic diagram of the laser emission and camera image splicing;

[0042] Figure 6 It is a flow chart of the all-directional weld seam tracking laser vision sensing method;

[0043] Figure 7 It is a schematic diagram of the effect of the image processing algorithm;

[0044] Figure 8 It is a schematic diagram of the working condition of a 90-degree broken line weld seam;

[0045] Figure 9 It is a schematic diagram of different weld regions;

[0046] Figure 10 It is the image obtained by the sensor when the weld seam in the P1 region is not welded;

[0047] Figure 11 It is the image obtained by the sensor during the welding process of the weld seam in the P1 region;

[0048] Figure 12 It is the image obtained by the sensor when the weld seam in the P3 region is not welded;

[0049] Figure 13 It is the image obtained by the sensor during the welding process of the weld seam in the P3 region;

[0050] Figure 14 The image acquired by the sensor when the weld in the P2 area is not welded;

[0051] Figure 15 The image acquired by the sensor during the welding process of the weld in the P2 area.

[0052] Explanation of the labels in the figure: 1. Laser; 2. Camera; 3. Filter; 31. Through hole; 4. Cable; 5. Sensor housing; 51. Welding torch mounting hole; 6. Mounting bracket; 7. Image processing module; 8. Welding torch; 9. Hose clamp; 10. Workpiece 1; 11. Workpiece 2; 12. P1 area; 13. P2 area; 14. P3 area; 15. 90-degree broken line weld; 16. Horizontal substrate; 17. Image processing algorithm. Specific embodiments

[0053] The present invention will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0054] The related concepts involved in the present invention are explained as follows:

[0055] Weld: In this application document, a weld refers to a preset area that connects two or more workpieces after welding by a welding torch.

[0056] The weld tracking system is used in the automatic welding scenarios of industrial robots or dedicated welding machines.

[0057] Workpiece: The workpiece in this application document refers to the base material with a preset weld.

[0058] Flat plate: It is a planar substrate without a weld. When the laser of the laser scans across the surface of the flat plate, the laser ripple is a regular pattern corresponding to the laser.

[0059] Welding torch: The welding torch referred to in the context of the present invention can be an arc welding torch, or other welding torches such as laser welding, ultrasonic welding torches, resistance welding, plasma welding, or electron beam welding that meet the required morphological specifications for automatic welding applications.

[0060] Camera: It is used to acquire image information and can be various types of optical cameras. The acquired image can be a photographed picture or obtained by selecting a video frame from a video stream. The acquired image is preferably a digital signal, or can be an analog signal converted into a digital signal.

[0061] Filter: The wavelength of the light allowed to pass through the filter is the same as the wavelength of the laser emitted by the laser. It is used to filter out the arc light interference during the welding process and at the same time block the smoke and spatter generated during the welding process.

[0062] Image processing module: It includes a memory and a processor. The processor can execute certain image recognition and processing algorithms to complete tasks such as stripe image stitching, closed contour recognition, and stripe image comparison.

[0063] Laser: A laser is a device that emits a highly concentrated beam of light.

[0064] As described in the Chinese patent application document with the publication number CN 107020449 A, refer to Figure 1 As shown, when a traditional welding torch works, a weld seam tracking sensor and the welding torch are arranged on a welding execution mechanism, such as a robot or a welding trolley, etc. The sensor is located at a certain distance in front of the welding torch. During welding, the sensor transmits the observed weld seam information to a computer or other mobile terminal. After processing, the weld seam trajectory is obtained. The computer or mobile terminal can then query in real time the distance by which the welding torch deviates from the weld seam, thereby controlling the execution mechanism to adjust the position of the welding torch and achieving the tracking of the weld seam. This method is applicable to long straight weld seam operations, but there are many limitations for complex curves.

[0065] The all-directional weld seam tracking laser vision sensor and weld seam detection method provided by the present invention can detect all possible directions of the welding torch during the process of weld seam tracking, complete the continuous tracking of a weld seam trajectory with a complex curve, and achieve uninterrupted welding.

[0066] Embodiment 1

[0067] As Figure 2 As shown, one aspect of the embodiments of this specification provides an all-directional weld seam tracking laser vision sensor, which includes a laser 1, at least 3 cameras 2, a filter 3, a cable 4, a sensor housing 5, and an image processing module 7 is arranged inside the sensor housing 5.

[0068] The sensor housing 5 is a cylinder. One end of the cylinder is provided with a cylinder cover, and a welding torch mounting hole 51 is arranged on the cylinder cover and is used for sleeving a welding torch 8; the other end of the cylinder is provided with a filter 3, and a through hole 31 is arranged on the filter 3. During use, the welding torch 8 is sleeved into the welding torch mounting hole 51, and the welding tip of the welding torch 8 passes through the through hole 31 on the filter 3, forming a mounting method in which the sensor is sleeved outside the welding torch.

[0069] The sensor sleeving the welding torch 8 makes the sensor closely surround the welding torch 8 and move along with the working welding torch 8, avoiding the situation of collision between the welding torch 8 and the sensor. In some embodiments, the positional relationship between the welding torch 8 and the sensor is not limited to this sleeved installation method, and the sensor can also be independently arranged outside. For example, the laser or the sensor can also be placed on a table, and this purpose can be achieved through a reflection device on the welding torch 8, but this is costly and can only be applied under ideal conditions.

[0070] The laser 1 and the camera 2 are arranged on the mounting bracket inside the sensor housing 5. In some other embodiments, the laser 1 and the camera 2 can also be installed on the surface of the sensor housing 5 and distributed circumferentially around the welding torch mounting hole 51. When the laser emitted by at least one laser 1 is projected onto the surface of the workpiece, one or more laser stripes are joined continuously to form a closed curve that surrounds the working area of the welding torch 8 for one week, resulting in a laser closed stripe. As Figure 15 shown by the intermittent small notches in the left curve in, in practical applications, through experimental verification, due to the influence of design, process, and on-site adjustment, it is difficult for the laser emitted by the laser to achieve complete closure. In this case, the degree of closed enclosure can meet the requirements of actual weld seam tracking. The camera 2 is used to collect the image of the laser closed stripe formed by the laser stripes.

[0071] The types, quantities, and installation positions of the laser 1 and the camera 2 are not limited. The laser can also be a laser that emits a straight line or other laser stripes. As long as it can theoretically meet the requirement that the laser stripes emitted by some or all of the lasers 1 can individually or continuously complete the closed enclosure around the welding torch. Between the laser 1 and the camera 2, it is sufficient to detect the weld seam in the direction and position where the welding torch 8 may move after the laser emitted by the laser cooperates with the image collected by the camera 2.

[0072] The image processing module 7 is connected to the camera 2 through the cable 4, receives the image information collected by the camera 2, and completes image processing to generate an image containing a closed loop formed by complete laser stripes, and further completes closed contour recognition and stripe image comparison. In some embodiments, the image processing module 7 can also be connected to the camera 2 wirelessly as long as it can complete the timely transmission of image information.

[0073] The filter 3 is installed at the front end of the camera 2 to block the soot and spatter generated during welding and filter out the arc light interference. Preferably, there are 3 lasers 1, which are evenly distributed circumferentially around the welding torch mounting hole at an interval of 120°. Preferably, the laser 1 is a ring laser that emits ring laser. The 3 cameras 2 are arranged staggered with the lasers circumferentially around the welding torch mounting hole.

[0074] Preferably, the camera 2 is a pinhole camera with a small volume. This can further compress the size of the sensor and facilitate cooperation with the flexible welding torch to enter a narrow space for work. Other cameras or cameras that meet the volume and performance requirements of the application scenario can also be considered.

[0075] Preferably, the filter 3 is a band-pass filter that allows the wavelength of the transmitted light to be the same as the wavelength of the laser emitted by the laser.

[0076] Furthermore, the material of the filter is selected as glass. Other materials that meet the requirements of light filtering and adaptation to the scenario, such as resin materials, can also be selected.

[0077] Further, the cable 4 includes a signal line and a power line.

[0078] Further, in some embodiments, the sensing device further includes a light reduction filter for reducing the input amount of light and filtering out strong arc light during the welding process.

[0079] As Figures 2 - 4 shown, a specific embodiment of the present invention is as follows: The sensor includes three ring lasers 1, three pinhole cameras 2, a band-pass filter 3, a cable 4, a sensor housing 5, a mounting bracket 6, and an image processing module 7.

[0080] The overall shape of the sensor is cylindrical and is used outside the welding torch 8. The sensor housing 5 corresponds to the device housing.

[0081] As Figure 3 shown, the laser 1 is arranged inside the sensor housing 5. The 3 lasers 1 are arranged circumferentially around the welding torch 8, and the adjacent two lasers are spaced 120°. Due to the occlusion of the welding tip of the welding torch 8, the ring laser emitted by the ring laser cannot be completely projected onto the workpiece surface. Therefore, it is necessary to adjust the positions of the three lasers 1 so that they cooperate and stagger to form a closed loop.

[0082] The 3 pinhole cameras 2 are arranged inside the sensor housing 5 and are arranged in a staggered ring around the welding torch at the front end of the sensor, with an interval of 120° between the adjacent two cameras 2. Similarly, due to the occlusion of the welding tip of the welding torch 8, a single camera 2 cannot capture the complete ring laser stripe morphology. Therefore, it is necessary to splice the images captured by the three cameras 2 into a complete image.

[0083] The wavelength of the light allowed to pass through by the band-pass filter 3 is consistent with the wavelength of the laser emitted by the laser. The band-pass filter 3 is fixed on the mounting bracket 6 at the front end of the sensor housing 5, and is used to block the soot and spatter generated during the welding process and filter out the arc light interference. A light reduction filter is also provided before the glass filter material for reducing the input amount of light.

[0084] Due to hardware limitations, in this embodiment, three single-point emission lasers are used to emit ring laser stripes to form a closed loop, instead of using a single ring laser. When the device process is satisfied, a single ring laser is also applicable.

[0085] Embodiment 2

[0086] In the second aspect of the present invention, a full-directional weld tracking welding device is provided, which includes a welding torch 8, an information processing module, and an instruction execution mechanism, and further includes a full-directional weld tracking laser vision sensor according to the first aspect of the present invention. The welding torch 8 is inserted into the welding installation hole 51 and passes through the through hole 31, and is sleeved and installed with the weld tracking sensor.

[0087] The weld seam tracking sensor outputs a real-time laser closed fringe image around the working area of the welding torch to the information processing module;

[0088] The information processing module analyzes the weld seam and weld bead information from the real-time laser closed fringe image, generates a welding torch operation control instruction, and then sends the welding torch operation control instruction to the instruction execution mechanism; The specific method is: comparing the real-time laser closed fringe image with the reference laser closed fringe image, extracting the fringe deviation feature, and calculating the weld seam and weld bead information. The reference laser closed fringe image is the laser closed fringe image formed by the laser emitted by the laser on a flat plate without a weld seam.

[0089] The instruction execution mechanism receives the control instruction sent by the information processing module and controls the welding torch to complete the welding work according to the instruction.

[0090] Figure 5 A specific embodiment of a welding device using an omnidirectional weld seam tracking laser vision sensor is shown. The sensor of Embodiment 1 is used outside the welding torch 8. Since the nozzle of the welding torch 8 will block the action area of a single laser 1. Adjust the position of the laser 1 so that the laser emitted by the laser 1 can form a closed loop by adjacent connection on the horizontal substrate 16 to obtain a laser closed fringe. At the same time, due to the presence of the welding torch 8, a single camera 2 cannot obtain a complete fringe image, and the images detected by the three cameras 2 need to be synthesized in the image processing module 7 through the image processing algorithm 17, and the feature information is extracted. During the welding process, the image processing algorithm 17 in the image processing module 7 processes the real-time image obtained by the camera 2, completes the stitching of the fringe image, further finds the intersection points of the circular fringes, obtains the closed outer contour, and then through the comparison of the fringe images, obtains the tracked weld seam information and outputs it to the welding torch control system for weld seam detection and welding torch control.

[0091] Embodiment 3

[0092] As Figure 6 shown, the present invention also provides an omnidirectional weld seam tracking laser vision sensing method for an omnidirectional weld seam tracking welding device described in the second aspect of the present invention. The steps are as follows:

[0093] Step S1: Adjust the position of the laser 1 to form a laser closed fringe: Use an omnidirectional weld seam tracking laser vision sensor sleeved with a welding torch to scan the flat plate. The specific method is: Place the flat plate directly below the omnidirectional weld seam tracking laser vision sensor sleeved with a welding torch, the laser emits laser to the flat plate surface, adjust the position of the laser to form a laser fringe, so that one laser fringe or a plurality of laser fringes are connected in succession around the projection area of the welding torch on the flat plate to form a laser closed fringe.

[0094] Optionally, an annular laser is provided in the device and can be installed around the welding torch mounting hole 51 for one week. The emitted laser stripes can form an annular closed stripe around the working area of the welding torch 8; or at least two lasers 1 are provided, and the lasers 1 partially or all emit lasers that converge and continue to form a laser closed stripe around the welding torch. Preferably, the laser is an annular laser. In some embodiments, there are 3 lasers 1.

[0095] Step S2: Obtain a reference laser closed stripe image: The image processing module outputs the flat laser closed stripe image to the information processing module as the reference laser closed stripe image.

[0096] Specifically, the camera 2 collects the original image of the laser closed stripe formed by the laser 1 on the flat plate in step S1. The camera 2 sends the collected original image to the image processing module 7 through the signal line in the cable 4; the image processing module 7 receives the original image and completes image processing to obtain the reference laser closed stripe image containing the complete laser stripe formation.

[0097] Optionally, three cameras 2 respectively collect partial images, and some or all of the partial images can be spliced to generate an image including the complete laser closed stripe. In some embodiments of the present invention, a filter is provided at the front end of the camera lens, which is used to filter out interfering light during operation. Preferably, a light reduction part is further provided at the front end of the filter for reducing the amount of light input.

[0098] In some embodiments of the present invention, the image processing module obtains an image including the complete laser closed stripe by splicing partial images. Further, during splicing, the image processing module searches for the intersection points of the laser stripes to obtain the outer contour of the laser closed stripe for facilitating comparison in subsequent steps.

[0099] In this embodiment, there are two key devices inside the sensor. One is a laser emitter that emits laser stripes, and the focus is on the focal plane of the laser; the other is a camera for taking stripe images, and the focus is on the camera focal length. Oblique incidence will cause graphic distortion and sensor shooting distortion, but it can be corrected by precise image processing algorithms. Therefore, there is no limited range for the distance and angle between the flat plate and the workpiece and the sensor, but it is necessary to ensure that the oblique incidence does not exceed the allowable range of the laser focal plane. For example, when the laser obliquely irradiates the flat plate, if the circle close to the laser is on the focal plane, the distal end of the circle will be away from the focal plane, and the laser stripe will become wider.

[0100] Due to assembly errors, it is very difficult to unify the distances between different laser emitters and the flat plate. The actually obtained image is not an ideal and regular closed loop. At this time, the process of obtaining the reference laser closed fringe image of the flat plate is necessary if the position, size information of the laser closed fringe on the flat plate and the mapping relationship of occlusion are known in advance through calculation. This step S2 can be omitted under ideal conditions, and subsequent comparisons can also be based on real-time images and calculation data. Currently, the algorithm that retains step S2 is suitable for practical applications.

[0101] Step S3: Real-time collect the laser closed fringe image of the weld of the workpiece to be welded: Use an omnidirectional weld tracking laser vision sensor sleeved with a welding torch to scan the workpiece to be welded in real time. The laser closed fringe intersects with the weld around the working area of the welding torch, and the image processing module outputs the real-time laser closed fringe image to the information processing module.

[0102] Specifically, using the same method as in step S2, use an omnidirectional weld tracking laser vision sensor to scan the workpiece to be welded with a weld in real time to obtain a real-time laser closed fringe image. Preferably, the outer contour of the real-time laser closed fringe image can be obtained by the same method described in S2.

[0103] Step S4: The information processing module compares the reference laser closed fringe image and the real-time laser closed fringe image, extracts the characteristics of the fringe deviation, obtains the weld and weld bead information, and is used to generate a welding torch operation control instruction. Specifically, the information processing module calculates the center position of the weld and / or the weld bead size according to the characteristics of the fringe deviation.

[0104] Repeat the above steps S3 and S4 until the welding is completed. In a specific example of the present invention, the image processing module splices the partial images of the laser fringes formed by the laser emitted by the three pinhole cameras 2 respectively to form a complete morphology of the laser closed fringe.

[0105] Further, the method for the image processing module 7 to splice the images taken by all the cameras 2 using the image stitching algorithm 17 is: Use the image processing algorithm 17 to remove noise and discrete dot patterns, and extract the pixel positions of the incomplete fringes in all the images. Splice the fringes according to the obtained fringe pixel positions. Preferably, the least squares method etc. is used to find the position with the smallest variance after splicing to complete the splicing.

[0106] Preferably, for the convenience of comparison, the image processing module 7 uses the image processing algorithm 17 to find the intersection points of the three circular laser fringes to obtain the closed contour of the fringe; as Figure 7 shown. Comparing the fringe image obtained in real time with the fringe image obtained on the flat plate can realize the identification of the omnidirectional weld center position.

[0107] The process of the all-directional weld tracking welding device in Embodiment 2 of the present invention for automatic welding using the all-directional weld tracking laser vision sensing method is specifically implemented as follows Figure 8 and includes the following steps:

[0108] The information processing module analyzes to obtain the weld track information, generates an instruction for adjusting the working state of the welding torch 8, and sends it to the instruction execution mechanism, which controls the working of the welding torch 8.

[0109] Figure 8 As shown in the figure, there is a 90-degree folded weld 15 between workpiece one 10 and workpiece two 11. The welding torch 8 is sleeved in the sensor housing 5, and the hose clamp 9 is used to fix the welding torch 8. A band-pass glass filter 3 is provided at the position where the sensor housing 5 abuts against the welding tip of the welding torch. The cable 4 is used to transmit the image information obtained by the sensing detection device to the image processing module 7. The weld tracking sensor and sensing method using a ring laser are applied to the tracking of a 90° folded corner weld, and the weld can be divided into 3 regions: P1 region 12, P2 region 13, and P3 region 14.

[0110] The P1 region 12 is a long straight weld, and the stripe morphology before welding is as Figure 10 shown. By comparing with the reference laser closed stripe image obtained at the flat plate position Figure 7 , at Figure 10 the upper and lower ends of the laser closed stripe image, corresponding to the front and rear directions of the welding torch, there are two notches corresponding to the unwelded welds, indicating that there are unwelded welds in both the front and rear directions of the welding torch. At this time, the welding torch 8 starts to work. The morphology during the welding process is as Figure 11 shown. At the upper end of the closed loop in the image, corresponding to the front of the welding torch 8, there is still a sunken notch corresponding to the unwelded weld. At the lower end of the closed loop in the image, corresponding to the rear of the welding torch, there is a protrusion formed after welding, indicating that the weld at the rear of the welding torch has been welded. The sensor is mainly responsible for tracking the position of the unwelded front section, which can be achieved by calculating the image position, and the completed part can be used for welding quality evaluation.

[0111] The stripe morphology of the P3 region 14 before welding is as Figure 12 shown. By comparing the closed loop stripe image in the figure with the original image obtained at the flat plate position Figure 6 , similarly, it can be seen that the unwelded welds exist below and to the right of the welding torch, and there is a 90° fold angle in the weld here. The morphology of the P3 region 14 during the welding process is as Figure 13 shown. The lower end has been welded. After the welding torch completes the welding of the lower end, it should continue to weld to the right. In fact, there is still a certain distance between the two detected notches in the image and the position of the welding wire in the middle of the welding torch, resulting in a blind area. Only through Figure 13 the bulging information below, it is not known Figure 9Whether the middle weld reaches the inflection point and when it reaches the inflection point is recorded by the sensor when detecting in the P1 area 12. Specifically, when the sensor is working in the P1 area 12 and the upper edge of the stripe just touches the inflection point, the upper edge of the stripe image changes. At this time, the vertical weld is not yet completed. The actuator calculates the distance between the upper edge of the stripe and the center of the welding torch divided by the welding speed to obtain the delay time. After waiting for the delay time, the welding torch 8 reaches the inflection point position. At this time, the notch in the P3 area 14 appears directly to the right of the center of the welding torch 8. When the actuator moves within the delay time, the corresponding right-side notch of the stripe gradually transitions from the upper right to directly to the right. That is to say, after the center of the welding torch reaches the inflection point, the direction of travel should be determined according to the change of the stripe within the delay time. In this case, it travels directly to the right.

[0112] At this time, since the tracking sensor realizes the all-directional weld detection, when a 90° fold angle appears in the weld, there is no need to stop welding and rotate the welding torch 8, and the turning and continuous welding can be realized.

[0113] After completing the welding of the P1 area 12 and the P3 area 14, the welding torch 8 can continuously weld to the P2 area 13 without interruption. The stripe morphology before welding in the P2 area 13 is as Figure 14 shown. Similar to the analysis of the P1 area 12 and the P3 area 14, there are unwelded welds in both the left and right directions of the welding torch 8. At this time, the welding torch 8 advances in the left and right directions, and the morphology during the welding process is as Figure 15 shown. The left side has been welded, and the right side is the direction of the weld to be welded. At this time, we observe that in the detection image of the welded area, due to the occlusion of the welding torch, a small notch appears in the image that should be closed. Further explanation, during the actual installation, debugging, and use of the equipment, due to the limitations of the actual conditions, a complete closed loop cannot always be presented, as long as it does not affect the completion of the welding task. It can be seen that by comparing the above different stripe images with the original image Figure 7 obtained from the flat position, the control system can judge the three-dimensional morphology information of the current detection position. During the detection process, the welding torch 8 and the sensor do not need to rotate to realize all-directional weld tracking. At the same time, when the welding torch 8 runs to the turning point of the P3 area 14, the welding direction can also be immediately switched, and there is no visual blind area. Therefore, the sensor and detection method of the present invention can realize all-directional real-time weld detection.

[0114] At the same time, the images before and after welding can be compared to obtain the net forming contour of the current weld bead. Through the solution recorded in the Chinese patent application with the publication number CN116973540 A, the height and width information of the weld bead can be obtained.

[0115] The detection method of the omnidirectional weld tracking laser vision sensor using a circular laser stripe can be applied to the omnidirectional tracking detection of complex space curve welds, and the robotic arm does not need to adjust the direction of the welding torch during the automatic welding process. At the same time, the detection device disclosed in the present invention has a compact structure and is suitable for narrow spaces; therefore, the device is also applicable to the dimensional detection of arc additive forming in narrow spaces. On the basis of the detection method of the present invention, it can be further expanded. By comparing the morphology before and after forming, the actual net deposition forming profile can be obtained. The specific method has exceeded the scope of the present invention and will not be elaborated here.

[0116] The present invention and its implementation manners are schematically described above. The description is not restrictive. Without departing from the spirit or basic features of the present invention, the present invention can be implemented in other specific forms. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Any reference signs in the claims should not limit the claimed claims. Therefore, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments to this technical solution without creative efforts without departing from the purpose of this creation, they should all fall within the protection scope of this application. In addition, the word "comprising" does not exclude other elements or steps, and the word "a" before an element does not exclude including "a plurality of" such elements. The multiple elements stated in the product claims can also be implemented by one element through software or hardware. The step names, as well as the words such as first and second, are only used to represent names and do not represent any specific order.

Claims

1. An omnidirectional weld tracking laser vision sensor, comprising a sensor housing (5) and a mounting frame (6) disposed in the sensor housing (5), characterized in that: The sensor housing (5) is a cylinder, one end of which is provided with a cylinder cover, on which a welding gun mounting hole (51) is provided and which is used to socket the welding gun (8); the other end of the cylinder is provided with a filter (3), on which a through hole (31) is provided; A laser (1) and three cameras (2) are provided on a mounting frame (6); the laser (1) is three ring lasers (1) which are arranged in a staggered manner with the cameras (2) at a circumferential interval of 120° around a welding gun mounting hole (51); a plurality of laser stripes formed on the surface of a workpiece by the laser (1) emitting laser light are connected to form closed stripes around a working area of ​​the welding gun (8); the laser stripes form a closed curve around the working area of ​​the welding gun (8) in all directions at 360°, so that after the welding gun (8) completes the current work, the weld seam intersects with the laser closed stripes when it is bent at different angles; The cameras (2) are arranged around the welding gun mounting hole (51) and are evenly distributed in the circumferential direction. The cameras (2) are used to capture the image of the laser closed stripes formed by the laser emitted by the laser (1). The images captured by the three cameras have a certain overlapping area. The images captured by the cameras are integrated through an image processing algorithm to obtain an image including the complete laser stripes. The filter (3) is located at the front end of the camera (2) and allows the wavelength of light to pass through to be the same as the wavelength of the laser emitted by the laser (1). An image processing module (7) is also provided in the cylinder, and the image processing module (7) is connected to the camera (2) and is used to receive image information captured by the camera (2), and to splice images captured by all cameras (2) to form an output containing a complete laser closed stripe image.

2. According to the omnidirectional weld tracking laser vision sensor of claim 1, the image processing module (7) stitches the images taken by all cameras (2) in the following manner: using an image processing algorithm (17) to remove noise and discrete spots, and extracting the pixel positions of incomplete stripes in all images; stitching the stripes according to the acquired stripe pixel positions, and using the least squares method to find the position with the smallest variance after stitching, and completing the stitching.

3. The omnidirectional weld tracking laser vision sensor according to claim 2, characterized in that: It also includes a cable (4), the cable (4) including a power line and a signal line, and the image processing module (7) is connected to the camera (2) via the signal line.

4. An omnidirectional weld tracking welding device, comprising a welding gun (8), an information processing module and an instruction execution mechanism, characterized in that: It also includes an omnidirectional weld seam tracking laser vision sensor as claimed in any one of claims 1 to 3, wherein the welding gun (8) is inserted into the welding mounting hole (51) and passes through the through hole (31) to form a sleeve installation with the weld seam tracking laser vision sensor; The weld tracking laser vision sensor outputs a real-time laser closed stripe image around the welding gun working area to the information processing module; The information processing module obtains the weld and weld bead information based on the real-time laser closed fringe image analysis, generates the welding gun operation control instruction, and sends the welding gun operation control instruction to the instruction execution mechanism; The instruction execution mechanism receives the control instruction sent by the information processing module and controls the welding gun (8) according to the instruction to complete the welding work.

5. The omnidirectional seam tracking welding device according to claim 4, characterized in that: The method for obtaining the weld seam and weld bead information by analyzing the information processing module is as follows: comparing the real-time laser closed fringe image with the reference laser closed fringe image, extracting the fringe deviation characteristics, and calculating the weld seam and weld bead information, wherein the reference laser closed fringe image is a laser closed fringe image formed on a flat plate without a weld seam by the laser emitted by the laser (1).

6. An omnidirectional seam tracking laser vision sensing method, used in an omnidirectional seam tracking welding device according to claim 4 or 5, comprising the following steps: Step S1: using an omnidirectional weld seam tracking laser vision sensor sleeved with a welding gun (8) to scan a flat plate, the specific method is: placing the flat plate directly below the omnidirectional weld seam tracking laser vision sensor sleeved with the welding gun, emitting laser light to the surface of the flat plate, adjusting the position of the laser to form laser stripes, so that a plurality of laser stripes form closed stripes on the flat plate that are continuous and surround the welding gun (8); Step S2: the image processing module (7) outputs the flat laser closed stripe image to the information processing module as a reference laser closed stripe image; Step S3: using an omnidirectional weld seam tracking laser vision sensor sleeved with a welding gun to scan the workpiece to be welded in real time, the laser closed stripes surround the working area of ​​the welding gun and intersect the weld seam, and the image processing module (7) outputs the real-time laser closed stripe image to the information processing module; In steps S2 and S3, before the image processing module (7) outputs the laser closed stripe image, it also includes the step of finding the intersection of the laser stripes to obtain the outer circle contour of the laser closed stripes, and the output laser closed stripe image only includes the outer circle contour; Step S4: the information processing module compares the reference laser closed fringe image with the real-time laser closed fringe image, extracts the characteristics of fringe deviation, obtains weld and weld bead information, and generates welding gun operation control instructions; in, The method for extracting the characteristics of stripe deviation and obtaining the information of long straight welds and 90° angle welds is as follows: Long straight weld: The stripe morphology before welding is compared with the reference laser closed stripe image obtained at the flat plate position. At the upper and lower ends of the laser closed stripe image before welding, corresponding to the front and rear directions of the welding gun, two gaps corresponding to the unwelded weld appear, indicating that there are unwelded welds in the front and rear directions of the welding gun; the upper end of the closed loop in the image during the welding process, corresponding to the front of the welding gun (8), still shows a concave gap corresponding to the unwelded weld, and the lower end of the closed loop in the image, corresponding to the rear of the welding gun, shows a bulge formed after welding; When the weld has a 90° bend, there is a certain distance between the two notches detected in the image and the position of the welding wire in the middle of the welding gun, resulting in a blind spot. When the upper edge of the stripe just touches the turning point, the upper edge of the stripe image changes. The actuator calculates the distance between the upper edge of the stripe and the center of the welding gun and divides it by the welding speed to obtain the delay time. After the delay time, the welding gun (8) reaches the turning point. At this time, the weld notch appears to the right of the center of the welding gun (8). After the center of the welding gun reaches the turning point, it moves directly to the right. Repeat steps S3 and S4 until the welding is completed.

7. The omnidirectional weld tracking laser vision sensing method according to claim 6 is characterized in that: The weld and weld bead information obtained by the information processing module includes information about welds that have been completed. The judgment method is: the lower end of the closed loop in the image, corresponding to the rear of the welding gun, presents a bulge formed after welding, indicating that the weld at the rear of the welding gun has been completed.

Images