Omnidirectional welding seam tracking laser vision sensor, sensing method and welding device

By distributing lasers and cameras around the welding gun, a 360° closed curve is formed, and combined with the splicing technology of the image processing module, the problem of collision between the weld tracking sensor and workpiece and the limitations of complex curve detection is solved, real-time tracking and efficient omnidirectional welding is achieved welding.

CN119016965AActive Publication Date: 2024-11-26SUZHOU OFIR INTELLIGENT TECH CO LTD +1

Patent Information

Application Number
CN202411510355.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-28
Publication Date
2024-11-26
Estimated Expiration
2044-10-28

AI Technical Summary

Technical Problem

Existing weld tracking laser vision sensors are prone to collision with workpieces, and there are limitations in the detection of complex curve welds, which cannot achieve real-time tracking and detection in all directions, resulting in stopping the direction adjustment during welding, affecting efficiency and quality.

Method used

Design an all-direction weld tracking laser vision sensor, the laser and camera are distributed around the welding gun, forming a 360° closed laser stripe image, and the omnidirectional detection of complex path welds is achieved through image comparison, and the cylindrical structure is used to plug the welding gun to avoid Collision and stitch the images taken by the camera through the image processing module to generate a complete laser closed stripe image.

Benefits of technology

Real-time inspection of all directions of complex welds is realized, avoiding the pause of direction adjustment during welding, improving welding efficiency and quality, and the system is small in size and suitable for narrow spaces.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119016965A_ABST
    Figure CN119016965A_ABST
Patent Text Reader

Abstract

The invention discloses an omnidirectional welding seam tracking laser vision sensor, a sensing method and a welding device, and relates to the field of welding automation. The sensor comprises a sensor shell, a mounting frame, an optical filter, a laser and at least three cameras, the cameras surround the welding gun mounting hole and are uniformly distributed in the circumferential direction, and the cameras are used for shooting laser closed stripe images formed by laser emitted by the laser; the optical filter is located at the front end of the camera, and the wavelength of light allowed to pass is the same as that of laser emitted by the laser; the sensor further comprises an image processing module arranged in the sensor shell, and the image processing module is used for being connected with the cameras, receiving image information shot by the cameras and splicing the images shot by all the cameras so as to form output containing complete laser closed stripe images. According to the welding device, tracking detection of space complex welding seams can be achieved through the omni-directional welding seam tracking laser vision sensor and the sensing method, and omni-directional automatic welding can be achieved without adjusting the direction of a welding gun.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of welding automation, and more specifically to an omnidirectional weld tracking laser vision sensor, a sensing method and a welding device. Background Art

[0002] Weld seam tracking technology is to automatically detect and adjust the position of the welding gun during the welding process so that it can always follow the weld position for welding, thereby ensuring welding quality, achieving adaptive control, and improving welding efficiency. The weld seam tracking system is mainly composed of three parts: sensor, control system, and motion actuator. Among them, the sensor is the most important component. At present, the laser vision sensor has the best application prospects. The weld seam tracking laser vision sensor uses the principles of optical propagation and imaging to obtain the position information of each point in the laser scanning area, and completes the online real-time detection of common welds through complex program algorithms. The equipment calculates the deviation between the detected weld and the welding gun, outputs the deviation data, and the motion actuator corrects the deviation in real time, accurately guides the welding gun to automatically weld, thereby realizing real-time communication with the robot control system to track the weld for welding.

[0003] At present, the laser vision sensors commonly used in industry are linear, which have the problem of large size and easy collision with workpieces. At the same time, linear laser sensors can only be placed at the front or rear end of the welding gun, and are suitable for long straight weld tracking. For example, the Chinese patent application, publication number CN106984926 A, publication date July 28, 2017, records a weld tracking system and a weld tracking method. The weld tracking system includes a visual sensor device, a bracket, a welding gun, a motion mechanism, a computer and a baffle. The visual sensor device includes a point laser, a linear laser, and a surface array camera; the surface array camera collects images and transmits them to the computer, and the computer issues instructions to control the motion mechanism to move in three dimensions. This type of weld tracking sensor device has a visual sensor device separated from the welding gun position to form a front and rear guiding relationship on a straight line. When tracking complex spatial curves such as bends, 90° folded corner welds and annular welds, the robot arm needs to drive the welding gun and the sensor to rotate and constantly change the posture. Especially when tracking a 90° angle weld, when reaching the bend of the weld, the welding gun and the sensor that are in the earlier position will reach the blank area without welds in the straight forward direction before the sensor that is in the later position. At this time, a blind spot appears in weld detection and the welding process needs to be stopped. Wait for the welding gun and the sensor to rotate, adjust the starting position, and restart the forward and backward collaboration in the new direction. As a result, continuous welding is impossible, resulting in welding quality problems.

[0004] The Chinese patent application, publication number CN110153602A, describes a multi-directional laser vision tracking sensor device and its tracking and control method. The device includes a laser vision sensor, a special-shaped welding gun, and a computer control system; the laser vision sensor is installed at the end of the manipulator, and the special-shaped welding gun is installed at the lower end of the center of the laser vision sensor. The laser vision sensor includes a flange frame, four cameras, three lasers, and an integrated pipeline. The four cameras are evenly distributed around the upper end of the welding gun, and the first to third cameras are respectively installed in pairs with the first to third lasers on the diagonally opposite sides of the flange frame to measure and track the front, rear, and side welds respectively; the fourth camera is arranged in another direction for observing and monitoring the welds. The multi-directional laser vision tracker can realize the measurement and tracking of cross welds and realize the switching tracking of welds in the longitudinal, transverse, and vertical directions. It solves the welding of cross welds in complex structures such as steel plate tic-tac-toe structures and corrugated plates due to space limitations and easy interference between the starting and end ends. The disadvantage is that the invention requires a computer control system to pre-read the workpiece diagram, obtain theoretical data on the workpiece structure and the start and end positions of the weld, and control the mobile scanning of multiple lasers and the cooperation of multiple cameras to complete image acquisition. It cannot complete the tracking and detection of welds in all directions in real time. In terms of installation, the device uses a flange frame to hang multiple cameras and laser modules. While it is flexible, it is not stable enough. The invention uses four different sensors to detect different directions, and it needs to control switching between multiple sensors to achieve tracking. It cannot truly achieve automatic fusion closed-loop omnidirectional monitoring, and the solution is too complicated. Summary of the invention

[0005] 1. Technical issues to be solved In view of the problems that existing weld tracking laser vision sensors are prone to collision with workpieces and have limitations in detecting complex curved welds, the present invention proposes an omnidirectional weld tracking laser vision sensor. The laser and camera are arranged around the welding gun, which reduces the system volume and avoids collision with the workpiece. It can collect 360° closed laser stripe images around the welding gun working area in real time, and can complete omnidirectional detection of complex path welds such as bends and 90° fold line corner welds through image comparison. During the fully automatic welding process of the above-mentioned complex welds, there is no need to pause welding to adjust the direction, thereby improving welding efficiency and quality.

[0006] 2. Technical solution The purpose of the present invention is achieved through the following technical solutions.

[0007] One aspect of the present invention provides an omnidirectional weld tracking laser vision sensor, comprising a sensor housing and a mounting frame arranged in the sensor housing, wherein the sensor housing is a cylinder, a cylinder cover is arranged at one end of the cylinder, a welding gun mounting hole is arranged on the cylinder cover and used to sleeve the welding gun; a filter is arranged at the other end of the cylinder, and a through hole is arranged on the filter. When in use, the welding gun is inserted from the welding gun mounting hole, and the welding nozzle of the welding gun passes through the through hole on the filter, forming an installation mode in which the sensor is sleeved on the outer ring of the welding gun.

[0008] A laser and at least three cameras are provided on the mounting frame. The cameras are evenly distributed around the welding gun mounting hole. The cameras are used to capture the laser closed stripe image formed by the laser emitted by the laser. When in use, the welding gun has been inserted into the sensor, and the welding gun has caused a certain degree of obstruction to the laser stripes formed by the laser emitting laser and the camera lens. At this time, it is necessary to adjust the position of the laser so that a laser stripe or a plurality of laser stripes formed on the surface of the workpiece by the laser emitting laser can form a closed stripe around the working area of ​​the welding gun, that is, the laser stripes can form a closed curve around the working area of ​​the welding gun in all directions at 360°, so that after the welding gun completes the current work, the weld will intersect with the laser closed stripes when it bends to different angles, thereby realizing omnidirectional detection of the weld.

[0009] The filter is located in front of the camera and allows the wavelength of light to pass through the same as the wavelength of the laser emitted by the laser. The filter can also be used to block the smoke and spatter generated during the welding process and filter out arc interference.

[0010] An image processing module is also provided in the cylinder. The image processing module is connected to the camera and is used to receive the image information captured by the camera, and to splice the images captured by all cameras to form an output containing a complete laser closed stripe image. This is mainly because in actual use, the welding gun is blocked, which makes it difficult for each camera to capture a complete laser closed stripe image, and the images captured by the three cameras have a certain overlapping area. Through the image processing algorithm, the pictures captured by each camera can be integrated to obtain an image including the complete laser stripes.

[0011] Furthermore, the omnidirectional weld tracking laser vision sensor also includes a cable, which includes a power cable and a signal cable, and the image processing module is connected to the camera via the signal cable. The sensor also supports a cable-free power supply and signal transmission method, which can be adjusted according to the actual working scene.

[0012] Furthermore, the image processing module uses an image stitching algorithm to stitch the images captured by all cameras by using an image processing algorithm to remove noise and discrete spots, and extract the pixel positions of incomplete stripes in all images. The stripes are stitched according to the acquired stripe pixel positions. Preferably, the least square method is used to find the position with the smallest variance after stitching to complete the stitching.

[0013] Preferably, the laser is a ring laser that emits a ring laser that surrounds the welding gun mounting hole.

[0014] In some embodiments, the lasers may also be three ring lasers, which are staggered with the camera at an angle of 120° around the welding gun mounting hole. The position, number and morphology of the laser stripes emitted by the lasers are sufficient as long as they can form 360° laser closed stripes around the welding gun working area when in use.

[0015] Preferably, the camera is a pinhole camera.

[0016] In some embodiments, the sensor is further provided with a light reduction system.

[0017] The second aspect of the present invention provides an omnidirectional weld seam tracking welding device, including a welding gun, an information processing module and an instruction execution mechanism, and also includes an omnidirectional weld seam tracking laser vision sensor according to the first aspect of the present invention. The welding gun is inserted into the welding mounting hole and passes through the through hole to form a socket installation with the weld seam tracking sensor.

[0018] The weld tracking sensor outputs a real-time laser closed stripe image around the welding gun working area to the information processing module; The information processing module obtains weld and weld bead information based on the real-time laser closed stripe image analysis, generates a welding gun operation control instruction, and then sends the welding gun operation control instruction to the instruction execution mechanism; the specific method is: compare the real-time laser closed stripe image with the reference laser closed stripe image, extract the stripe deviation characteristics, and calculate the weld and weld bead information, wherein the reference laser closed stripe image is a laser closed stripe image formed by the laser emitted by the laser on a flat plate without a weld.

[0019] The instruction execution mechanism receives the control instructions sent by the information processing module and controls the welding gun according to the instructions to complete the welding work.

[0020] The third aspect of the present invention further provides an omnidirectional weld seam tracking laser visual sensing method, which is used in the omnidirectional weld seam tracking welding device described in the second aspect of the present invention, comprising the following steps: Step S1: Use an omnidirectional weld seam tracking laser vision sensor with a welding gun mounted thereon to scan the flat plate. The specific method is as follows: place the flat plate directly below the omnidirectional weld seam tracking laser vision sensor with a welding gun mounted thereon, emit laser light to the surface of the flat plate, adjust the laser position to form laser stripes, and make one laser stripe or multiple laser stripes connected to form laser closed stripes around the projection area of ​​the welding gun on the flat plate.

[0021] Step S2: the image processing module outputs the flat laser closed stripe image to the information processing module as a reference laser closed stripe image.

[0022] Step S3: Use an omnidirectional weld seam tracking laser vision sensor connected to a welding gun to scan the workpiece to be welded in real time. The laser closed stripes intersect the weld seam around the working area of ​​the welding gun. The image processing module outputs the real-time laser closed stripe image to the information processing module.

[0023] Step S4: The information processing module compares the reference laser closed fringe image with the real-time laser closed fringe image, extracts the characteristics of the fringe deviation, obtains the weld and weld bead information, and generates the welding gun operation control instruction.

[0024] Repeat steps S3 and S4 until welding is completed.

[0025] Preferably, before the image processing module outputs the laser closed stripe image, it also includes the step of finding the intersection of the laser stripes to obtain the outer circle contour of the laser closed stripes, and the output laser closed stripe image only includes the outer circle contour.

[0026] Furthermore, the weld and weld bead information obtained by the information processing module includes information about welds that have been completed.

[0027] 3. Beneficial effects Compared with the prior art, the advantages of the present invention are: 1. The laser sensor of the omnidirectional weld tracking laser vision sensor cooperates with at least three cameras. The emitted laser forms laser closed stripes, forming a 360° omnidirectional closed curve around the working area of ​​the welding gun, so that all kinds of complex welds intersect with the laser closed stripes when turning in any direction, eliminating the detection blind spot when the weld direction changes suddenly; at the same time, the information collection of the complete laser closed stripes is completed by at least three cameras, and the laser closed stripe image obtained after splicing can realize omnidirectional real-time tracking and detection of the weld. Even in the face of curved path welds, 90° fold line corner welds and other spatially complex curve welds, they can be detected in time. The direction of the weld can be determined without the robot arm rotating the welding gun direction, and continuous welding can be achieved, avoiding the situation where welding has to be stopped during the adjustment of the welding gun and the sensor, thereby improving the welding quality.

[0028] 2. Compared with the existing weld tracking sensor devices and detection methods, the omnidirectional weld tracking laser vision sensor of the present invention adopts a cylindrical structure. When in use, it is put on the welding gun to form a tightly surrounding connection relationship with the welding gun. During the operation of the welding gun, it moves synchronously with the welding gun to avoid collision between the laser sensor and the workpiece.

[0029] 3. The sensor uses a pinhole camera and a ring laser with a compact structure surrounding the welding gun. The overall size of the system is small, which is suitable for welding operations of soft welding guns in narrow spaces.

[0030] 4. The omnidirectional weld tracking laser visual sensing method of the present invention realizes the stitching of images taken by different cameras by image processing algorithm, calculates the weld bead information by continuously comparing the reference laser closed stripe image and the real-time laser closed stripe image, and can predict the direction of the weld. It is further combined with the running speed of the welding gun to achieve more accurate automatic welding control. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of an application scenario of a weld tracking sensor device in the prior art; Figure 2 This is a schematic diagram of the structure of the omnidirectional weld tracking laser vision sensor; Figure 3 AA cross-sectional view of the omnidirectional weld tracking laser vision sensor; Figure 4 This is the BB cross-sectional view of the omnidirectional weld tracking laser vision sensor; Figure 5 This is a schematic diagram of laser emission and camera image stitching; Figure 6 Flow chart of laser vision sensing method for omnidirectional weld tracking; Figure 7 This is a schematic diagram of the effect of the image processing algorithm; Figure 8 This is a schematic diagram of the 90-degree fold line weld working condition; Fig. 9 Schematic diagram of different weld areas; Fig.10 This is the image acquired by the sensor when the weld in the P1 area is not welded; Fig.11 This is the image acquired by the sensor during the welding process of the weld in the P1 area; Fig.12 This is the image acquired by the sensor when the weld in the P3 area is not welded; Fig.13 The image obtained by the sensor during the welding process of the weld in the P3 area; Fig.14 This is the image acquired by the sensor when the weld in the P2 area is not welded; Fig.15 This is the image acquired by the sensor during the welding process of the weld in the P2 area.

[0032] Explanation of the numbers in the figure: 1. Laser; 2. Camera; 3. Filter; 31. Through hole; 4. Cable; 5. Sensor housing; 51. Welding gun mounting hole; 6. Mounting bracket; 7. Image processing module; 8. Welding gun; 9. Hose clamp; 10. Workpiece one; 11. Workpiece two; 12. P1 area; 13. P2 area; 14. P3 area; 15. 90-degree fold line weld; 16. Horizontal substrate; 17. Image processing algorithm. DETAILED DESCRIPTION

[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0034] The relevant concepts involved in the present invention are explained as follows: Weld: In this application document, a weld is a predetermined area that connects two or more workpieces after welding with a welding gun.

[0035] The weld seam tracking system is used in automatic welding scenarios of industrial robots or welding machines.

[0036] Workpiece: The workpiece in this application document refers to the parent material with a preset weld.

[0037] Flat plate: It is a flat substrate without welding seams. When the laser light of the laser scans over the surface of the flat plate, the laser ripple is a regular pattern corresponding to the laser.

[0038] Welding gun: The welding gun referred to in the context of the present invention can be an arc welding gun, or it can be a laser welding gun, ultrasonic welding gun, resistance welding, plasma welding or electron beam welding gun, etc. whose shape specifications meet the requirements and other welding guns that meet the automatic welding application scenarios.

[0039] Camera: used to obtain image information. It can be any type of optical camera. The collected image can be a photograph or a video frame selected from a video stream. The collected image is preferably a digital signal, or it can be an analog signal converted into a digital signal.

[0040] Filter: The filter allows the wavelength of light to be consistent with the wavelength of the laser emitted by the laser, which is used to filter out arc interference during the welding process and block the smoke and spatter generated during the welding process.

[0041] Image processing module: includes memory and processor. The processor can execute certain image recognition and processing algorithms to complete tasks such as stripe image stitching, closed contour recognition, stripe image comparison, etc.

[0042] Laser: A laser is a device that emits a highly concentrated beam of light.

[0043] As described in the Chinese patent application document, publication number CN 107020449 A, reference Figure 1As shown, when the traditional welding gun is working, the weld tracking sensor and the welding gun are placed on the welding actuator, such as a robot or a welding trolley, and the sensor is located a certain distance in front of the welding gun. During welding, the sensor transmits the observed weld information to a computer or other mobile terminal, and the weld trajectory is obtained after processing. The computer or mobile terminal can query the distance of the welding gun from the weld in real time, thereby controlling the actuator to adjust the position of the welding gun and realize the tracking of the weld. This method is suitable for long straight weld operations, but there are many limitations for complex curves.

[0044] The omnidirectional weld tracking laser vision sensor and weld detection method provided by the present invention can detect all possible directions of the welding gun during the weld tracking process, complete the continuous tracking of the weld trajectory as a complex curve, and realize uninterrupted welding.

[0045] Example 1 like Figure 2 As shown, one aspect of an embodiment of the present specification provides an omnidirectional weld tracking laser vision sensor, including a laser 1, at least three cameras 2, a filter 3, a cable 4, a sensor housing 5, a mounting frame 6 and an image processing module 7 are arranged in the sensor housing 5.

[0046] The sensor housing 5 is a cylinder, one end of which is provided with a cylinder cover, on which a welding gun mounting hole 51 is provided and used for sleeved welding gun 8; the other end of the cylinder is provided with a filter 3, on which a through hole 31 is provided. When in use, welding gun 8 is inserted from welding gun mounting hole 51, and the welding tip of welding gun 8 passes through the through hole 31 on filter 3, forming an installation mode in which the sensor is sleeved on the outer ring of welding gun.

[0047] The sensor is sleeved with the welding gun 8 so that the sensor closely surrounds the welding gun 8 and moves with the welding gun 8 during operation, thereby avoiding collision between the welding gun 8 and the sensor. In some embodiments, the positional relationship between the welding gun 8 and the sensor is not limited to the sleeve installation, and the sensor can also be independently set outside. For example, the laser or sensor can also be placed on the table, and this purpose can be achieved through the reflection device on the welding gun 8, but this is very costly and can only be applied under ideal conditions.

[0048] The laser 1 and the camera 2 are arranged on a mounting frame in the sensor housing 5. In some other embodiments, the laser 1 and the camera 2 can also be mounted on the surface of the sensor housing 5 and distributed circumferentially around the welding gun mounting hole 51. When the laser emitted by at least one laser 1 is projected onto the surface of the workpiece, one or more laser stripes intersect and connect to enclose the working area of ​​the welding gun 8 for one week, forming a closed curve and obtaining a laser closed stripe. Fig.15As shown by the intermittent small gaps in the left curve, in actual applications, it has been verified through experiments that due to the influence of design, process and on-site adjustment, it is difficult for the laser emitted by the laser to achieve complete closure. In this case, the degree of closure can meet the actual weld tracking requirements. Camera 2 is used to collect the laser closed stripe image formed by the laser stripe.

[0049] The type, quantity and installation position of the laser 1 and the camera 2 are not limited. The laser can also be a laser that emits straight lines or other laser stripes. As long as it can theoretically meet the requirement that some or all of the laser stripes emitted by the laser 1 can complete the closed encirclement of the periphery of the welding gun alone or in succession. Between the laser 1 and the camera 2, it is sufficient that the laser emitting laser and the camera 2 collecting images can detect the weld in the direction and position where the welding gun 8 may move.

[0050] The image processing module 7 is connected to the camera 2 via the cable 4, receives the image information collected by the camera 2, and completes the image processing to generate an image of a closed loop formed by a complete laser stripe, and further completes the closed contour recognition and stripe image comparison. In some embodiments, the image processing module 7 can also be connected to the camera 2 wirelessly, as long as the image information can be transmitted in a timely manner.

[0051] The filter 3 is installed at the front end of the camera 2 to block the smoke and spatter generated during welding and filter out arc interference. Preferably, three lasers 1 are provided, which are evenly distributed at 120° intervals around the welding gun mounting hole. Preferably, the laser 1 is a ring laser that emits a ring laser. The three cameras 2 are arranged in an interlaced manner with the lasers around the welding gun mounting hole.

[0052] Preferably, the camera 2 is a pinhole camera with a relatively small volume. This can further reduce the size of the sensor, making it easier to work with a soft welding gun into a narrow space. Other cameras and cameras that meet the volume and performance requirements of the application scenario can also be considered.

[0053] Preferably, the filter 3 is a bandpass filter, which allows the wavelength of light passing therethrough to be consistent with the wavelength of laser light emitted by the laser.

[0054] Furthermore, the filter is made of glass, and other materials that meet the requirements of filtering and adapting to the scene, such as resin materials, can also be used.

[0055] Furthermore, the cable 4 includes a signal line and a power line.

[0056] Furthermore, in some embodiments, the sensing device also includes a dimming film for reducing the input amount of light and filtering out strong arc light effects during the welding process.

[0057] like Figure 2-Figure 4As shown, a specific embodiment of the present invention is as follows: the sensor includes three ring lasers 1, three pinhole cameras 2, a bandpass filter 3, a cable 4, a sensor housing 5, a mounting bracket 6 and an image processing module 7.

[0058] The sensor is cylindrical in shape as a whole and is mounted on the outside of the welding gun 8 for use. The sensor housing 5 corresponds to the equipment housing.

[0059] like Figure 3 As shown, the laser 1 is disposed inside the sensor housing 5, and three lasers 1 are arranged circumferentially around the welding gun 8, with an interval of 120° between two adjacent lasers. Due to the shielding of the welding gun 8 nozzle, the ring laser emitted by the ring laser cannot be completely projected on the workpiece surface, so it is necessary to adjust the positions of the three lasers 1 so that they are coordinated and staggered to form a closed loop.

[0060] Three pinhole cameras 2 are arranged inside the sensor housing 5, and are arranged in a staggered ring with three lasers 1 at the front end of the sensor around the welding gun position, with an interval of 120° between two adjacent cameras 2. Also, due to the obstruction of the welding gun 8 nozzle, a single camera 2 cannot capture the complete annular laser stripe morphology, so the images captured by the three cameras 2 need to be stitched together into a complete image.

[0061] The bandpass filter 3 allows the wavelength of light to pass through to be consistent with the wavelength of the laser emitted by the laser. The bandpass filter 3 is fixed on the mounting bracket 6 at the front end of the sensor housing 5. It is used to block the smoke and spatter generated during the welding process and filter out the arc interference. A dimming film is also arranged before the glass filter to reduce the input amount of light.

[0062] Due to hardware limitations, this embodiment uses three single-point emitting lasers to emit annular laser stripes to form a closed loop, instead of a single ring laser. If the equipment process meets the requirements, a single ring laser is also applicable.

[0063] Example 2 The second aspect of the present invention provides an omnidirectional weld seam tracking welding device, including a welding gun 8, an information processing module and an instruction execution mechanism, and also includes an omnidirectional weld seam tracking laser vision sensor according to the first aspect of the present invention. The welding gun 8 is inserted into the welding mounting hole 51 and passes through the through hole 31 to form a socket installation with the weld seam tracking sensor.

[0064] The weld tracking sensor outputs a real-time laser closed stripe image around the welding gun working area to the information processing module; The information processing module obtains weld and weld bead information based on the real-time laser closed stripe image analysis, generates a welding gun operation control instruction, and then sends the welding gun operation control instruction to the instruction execution mechanism; the specific method is: compare the real-time laser closed stripe image with the reference laser closed stripe image, extract the stripe deviation characteristics, and calculate the weld and weld bead information, wherein the reference laser closed stripe image is a laser closed stripe image formed by the laser emitted by the laser on a flat plate without a weld.

[0065] The instruction execution mechanism receives the control instructions sent by the information processing module and controls the welding gun according to the instructions to complete the welding work.

[0066] Figure 5 A specific embodiment of a welding device using an omnidirectional weld tracking laser vision sensor is shown. The sensor sleeve of Example 1 is used outside the welding gun 8. The welding nozzle of the welding gun 8 will block the action area of ​​a single laser 1. The position of the laser 1 is adjusted so that the lasers emitted by the laser 1 can intersect and synthesize a closed loop on the horizontal substrate 16 in pairs, thereby obtaining laser closed stripes. At the same time, due to the presence of the welding gun 8, a single camera 2 cannot obtain a complete stripe image. The images detected by the three cameras 2 need to be synthesized in the image processing module 7 through the image processing algorithm 17, and feature information is extracted. During the welding process, the image processing algorithm 17 in the image processing module 7 processes the real-time image obtained by the camera 2, completes the stripe image splicing, further searches for the intersection of the annular stripes, obtains the closed outer contour, and then obtains the tracked weld information through stripe image comparison and outputs it to the welding gun control system for weld detection and welding gun control.

[0067] Example 3 like Figure 6 As shown, the present invention also provides an omnidirectional weld seam tracking laser visual sensing method, which is used in an omnidirectional weld seam tracking welding device described in the second aspect of the present invention, and the steps are as follows: Step S1: Adjust the position of laser 1 to form laser closed stripes: Use an omnidirectional weld seam tracking laser vision sensor with a welding gun mounted on it to scan the flat plate. The specific method is: place the flat plate directly below the omnidirectional weld seam tracking laser vision sensor with a welding gun mounted on it, and the laser emits laser light to the surface of the flat plate. Adjust the position of the laser to form laser stripes, so that one laser stripe or multiple laser stripes are connected to form laser closed stripes around the projection area of ​​the welding gun on the flat plate.

[0068] Optionally, the device is provided with a ring laser that can be installed around the welding gun mounting hole 51, and the emitted laser stripes can form a ring-shaped closed stripe around the working area of ​​the welding gun 8; or at least two lasers 1 are provided, and the lasers emitted by some or all of the lasers 1 intersect and merge to form a laser closed stripe around the welding gun; preferably, the laser is a ring laser. In some embodiments, there are three lasers 1.

[0069] Step S2: Acquire a reference laser closed stripe image: The image processing module outputs the flat laser closed stripe image to the information processing module as a reference laser closed stripe image.

[0070] Specifically, the camera 2 collects the original image of the laser closed stripes formed by the laser 1 on the flat plate in step S1, and the camera 2 sends the collected original image to the image processing module 7 through the signal line in the cable 4; the image processing module 7 receives the original image and completes the image processing to obtain a reference laser closed stripe image containing the complete laser stripes.

[0071] Optionally, three cameras 2 collect local images respectively, and some or all local images can be spliced ​​to generate an image including complete laser closed stripes. In some embodiments of the present invention, a filter is provided at the front end of the camera lens to filter out interfering light during operation. Preferably, a light reduction unit is also provided at the front end of the filter to reduce the input amount of light.

[0072] In some embodiments of the present invention, the image processing module obtains an image including complete laser closed stripes by splicing partial images. Further, during splicing, the image processing module searches for intersections of laser stripes and obtains the outer contour of the laser closed stripes to facilitate comparison in subsequent steps.

[0073] In this embodiment, there are two key components inside the sensor, one is the laser emitter, which emits laser stripes, and the focus is the focal plane of the laser; the other is a camera, which is used to capture the stripe image, and the focus is the focal length of the camera. Oblique projection will produce graphic distortion and sensor shooting distortion, but it can be corrected by precise image processing algorithms. Therefore, there is no limited range for the distance and angle between the plate and the workpiece and the sensor, but it is necessary to ensure that the oblique projection does not exceed the allowable range of the laser focal plane. For example: if the laser is projected obliquely onto the plate, if the circle close to the laser is on the focal plane, the far end of the circle will be far away from the focal plane, and the laser stripes will become wider.

[0074] Due to assembly errors, it is difficult to unify the distances between different laser emitters and the plate. The image actually obtained is not an ideal and regular closed loop. At this time, the process of obtaining the image of the reference laser closed stripes on the plate is necessary, if the position and size information of the laser closed stripes on the plate and the mapping relationship of the occlusion are known in advance by calculation. The S2 step can be omitted under ideal conditions, and subsequent comparisons can also be performed based on real-time images and calculated data. The algorithm that currently retains the S2 step is suitable for practical applications.

[0075] Step S3: Real-time acquisition of the laser closed stripe image of the weld of the workpiece to be welded: Use an omnidirectional weld tracking laser vision sensor connected to a welding gun to scan the workpiece to be welded in real time. The laser closed stripes intersect with the weld around the working area of ​​the welding gun. The image processing module outputs the real-time laser closed stripe image to the information processing module.

[0076] Specifically, the same method as step S2 is used to use an omnidirectional weld tracking laser vision sensor to scan the workpiece to be welded with a weld in real time to obtain a real-time laser closed stripe image. Preferably, the outer circle contour of the real-time laser closed stripe image can be obtained by the same method described in S2.

[0077] Step S4: The information processing module compares the reference laser closed fringe image with the real-time laser closed fringe image, extracts the characteristics of the fringe deviation, obtains the weld and weld bead information, and generates the welding gun operation control instruction. Specifically, the information processing module calculates the center position of the weld and / or the weld bead size according to the characteristics of the fringe deviation.

[0078] Repeat the above steps S3 and S4 until the welding is completed. In a specific example of the present invention, the image processing module stitches the local images of laser stripes formed by the three ring lasers 1 emitted by the three pinhole cameras 2 to form the complete morphology of the laser closed stripes.

[0079] Furthermore, the image processing module 7 uses the image stitching algorithm 17 to stitch the images captured by all cameras 2 by using the image processing algorithm 17 to remove noise and discrete spots, and extract the pixel positions of incomplete stripes in all images. The stripes are stitched according to the acquired stripe pixel positions. Preferably, the least square method is used to find the position with the smallest variance after stitching to complete the stitching.

[0080] Preferably, in order to facilitate comparison, the image processing module 7 applies the image processing algorithm 17 to find the intersection of the three annular laser stripes and obtain the closed contour of the stripes; Figure 7 By comparing the fringe image acquired in real time with the fringe image acquired on the flat plate, the center position of the weld can be identified in all directions.

[0081] The omnidirectional weld seam tracking welding device of Example 2 of the present invention uses the omnidirectional weld seam tracking laser visual sensing method to perform automatic welding, and the specific implementation is as follows: Figure 8 As shown, the following steps are included: The information processing module analyzes and obtains the weld trajectory information, generates an instruction for adjusting the working state of the welding gun 8 and sends it to the instruction execution mechanism, and the instruction execution mechanism controls the operation of the welding gun 8.

[0082] Figure 8 In the figure, a 90-degree folded-line weld 15 is arranged between workpiece 10 and workpiece 2 11, a welding gun 8 is inserted into a sensor housing 5, a throat hoop 9 is used to fix the welding gun 8, a bandpass glass filter 3 is arranged at a position where the sensor housing 5 abuts against the welding gun nozzle, and a cable 4 is used to transmit image information obtained by the sensing detection device to an image processing module 7. A weld tracking sensor and a sensing method using a ring laser are applied to the tracking of a 90-degree folded-line fillet weld, wherein the weld can be divided into three areas: a P1 area 12, a P2 area 13, and a P3 area 14.

[0083] P1 area 12 is a long straight weld, and the stripe morphology before welding is as follows Fig.10 As shown, the reference laser closed fringe image obtained by comparing the flat plate position Figure 7 ,exist Fig.10 At the upper and lower ends of the laser closed stripe image, corresponding to the front and rear directions of the welding gun, two gaps corresponding to the unwelded welds appear, indicating that there are unwelded welds in the front and rear directions of the welding gun. At this time, the welding gun 8 starts working. The morphology during the welding process is as follows Fig.11 As shown, at the upper end of the closed loop in the image, corresponding to the front of the welding gun 8, there is still a concave notch corresponding to the unwelded weld, and at the lower end of the closed loop in the image, corresponding to the rear of the welding gun, there is a bulge formed after welding, indicating that the weld at the rear of the welding gun has been welded. The sensor is mainly responsible for tracking the position of the unwelded front section, which can be achieved by calculating the image position, and the completed part can be used for welding quality evaluation.

[0084] The stripe morphology of P3 area 14 before welding is as follows Fig.12 As shown, by comparing the closed loop fringe image in the figure with the original image obtained from the plate position Figure 6 By comparison, it can be seen that the unwelded weld exists below and to the right of the welding gun, where the weld has a 90° angle. Fig.13 As shown in the figure, the lower end has been welded. After the welding of the lower end is completed, the welding gun should continue welding to the right. In fact, there is a certain distance between the two gaps detected in the image and the position of the welding wire in the middle of the welding gun. There is a blind area. Fig.13 I don't know the information about the bulge below. Fig. 9The information of whether and when the weld reaches the turning point is recorded by the sensor during detection in the P1 area 12. Specifically, when the sensor is working in the P1 area 12, when the upper edge of the stripe just touches the turning point, the upper edge of the stripe image changes, and the vertical weld is not yet completed. The actuator obtains the delay time by calculating the distance between the upper edge of the stripe and the center of the welding gun and dividing it by the welding speed. After waiting for the delay time, the welding gun 8 reaches the turning point, and at this time the gap in the P3 area 14 appears to the right of the center of the welding gun 8. When the actuator moves within the delay time, the gap on the right side corresponding to the stripe gradually transitions from the upper right to the right. In other words, after the center of the welding gun reaches the turning point, the direction to move should be determined according to the changes in the stripes during the delay time. In this case, it moves directly to the right.

[0085] At this time, since the tracking sensor realizes omnidirectional weld detection, when a 90° fold angle appears in the weld, there is no need to stop welding or rotate the welding gun 8, and continuous welding can be achieved.

[0086] After completing the welding of the P1 region 12 and the P3 region 14, the welding gun 8 can continue to weld to the P2 region 13 without interruption. The stripe morphology of the P2 region 13 before welding is as follows: Fig.14 As shown in FIG. 1 , the analysis of the P1 region 12 and the P3 region 14 shows that there are unwelded welds in the left and right directions of the welding gun 8. At this time, the welding gun 8 is moving in the left and right directions. Fig.15 As shown in the figure, welding has been completed on the left side, and the right side is the direction of the weld to be welded. At this time, we observed that in the detection image of the welded area, due to the obstruction of the welding gun, a small gap appeared in the image that should have been closed. Further explanation, in the actual installation, commissioning, and use of the equipment, due to the limitations of realistic conditions, it is not possible to consistently present a complete closed loop, as long as it does not affect the completion of the welding task. It can be seen that by comparing the above different fringe images with the original image obtained at the plate position Figure 7 By comparing the three-dimensional shape information of the current detection position, the control system can determine the three-dimensional shape information of the current detection position. During the detection process, the welding gun 8 and the sensor do not need to rotate, and the weld seam tracking in all directions can be realized. At the same time, when the welding gun 8 runs to the turning point of the P3 area 14, the welding direction can also be changed immediately, and there is no visual blind spot. Therefore, the sensor and detection method of the present invention can realize real-time detection of weld seams in all directions.

[0087] At the same time, the images before and after welding can be compared to obtain the net shape contour of the current weld bead. Through the solution described in the Chinese patent application with publication number CN116973540 A, the height and width information of the weld bead can be obtained.

[0088] The detection method of the omnidirectional weld tracking laser vision sensor using annular laser stripes can be applied to the omnidirectional tracking detection of spatial complex curved welds. During the automatic welding process, the robot arm does not need to adjust the direction of the welding gun. At the same time, the detection device disclosed in the present invention has a compact structure and is suitable for narrow spaces; therefore, the device is also suitable for arc additive forming size detection in narrow spaces. On the basis of the detection method of the present invention, it can be further expanded to obtain the actual deposited net forming contour by comparing the morphology before and after forming. The specific method has exceeded the scope of the present invention and will not be repeated here.

[0089] The invention and its implementation methods are described schematically above, and the description is non-restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention, and the actual structure is not limited thereto. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, and without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, they should all belong to the protection scope of the present application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The step name, as well as the words first, second, etc. are only used to indicate the name, and do not indicate any specific order.

Claims

1. An omnidirectional weld tracking laser vision sensor, comprising a sensor housing (5) and a mounting frame (6) disposed in the sensor housing (5), It is characterized in that The sensor housing (5) is a cylinder, one end of which is provided with a cylinder cover, on which a welding gun mounting hole (51) is provided and used for sleeve-connecting the welding gun; the other end of the cylinder is provided with a filter (3), on which a through hole (31) is provided; A laser (1) and at least three cameras (2) are provided on the mounting frame (6); the cameras (2) surround the welding gun mounting hole and are evenly distributed in the circumferential direction; the cameras (2) are used to capture a laser closed stripe image formed by the laser light emitted by the laser (1); The filter (3) is located at the front end of the camera (2) and allows the wavelength of light to pass through to be the same as the wavelength of the laser light emitted by the laser (1); An image processing module (7) is also provided in the cylinder, and the image processing module (7) is connected to the camera (2) and is used to receive image information captured by the camera (2), and to splice images captured by all cameras (2) to form an output containing a complete laser closed stripe image.

2. According to the omnidirectional weld tracking laser vision sensor of claim 1, the image processing module (7) stitches the images taken by all cameras (2) in the following manner: using an image processing algorithm (17) to remove noise and discrete spots, and extracting the pixel positions of incomplete stripes in all images; stitching the stripes according to the acquired stripe pixel positions, and using the least squares method to find the position with the smallest variance after stitching, and completing the stitching.

3. The omnidirectional weld tracking laser vision sensor according to claim 2, It is characterized in that The laser (1) is a ring laser, and the ring laser surrounds the welding gun mounting hole.

4. The omnidirectional weld tracking laser vision sensor according to claim 2, It is characterized in that The lasers (1) are three ring lasers, which are distributed alternately with the cameras (2) at a circumferential interval of 120° around the welding gun mounting hole (51).

5. An omnidirectional weld tracking laser vision sensor according to claim 3 or 4, It is characterized in that It also includes a cable (4), the cable (4) including a power line and a signal line, and the image processing module (7) is connected to the camera (2) via the signal line.

6. An omnidirectional weld tracking welding device, comprising a welding gun (8), an information processing module and an instruction execution mechanism, It is characterized in that It also includes an omnidirectional weld seam tracking laser vision sensor as claimed in any one of claims 1 to 5, wherein the welding gun (8) is inserted into the welding mounting hole (51) and passes through the through hole (31) to form a sleeve installation with the weld seam tracking sensor; The weld tracking sensor outputs a real-time laser closed stripe image around the welding gun working area to the information processing module; The information processing module obtains the weld and weld bead information based on the real-time laser closed fringe image analysis, generates the welding gun operation control instruction, and sends the welding gun operation control instruction to the instruction execution mechanism; The instruction execution mechanism receives the control instruction sent by the information processing module and controls the welding gun (8) according to the instruction to complete the welding work.

7. An omnidirectional seam tracking welding device according to claim 6, It is characterized in that The method for obtaining the weld seam and weld bead information by analyzing the information processing module is as follows: comparing the real-time laser closed fringe image with the reference laser closed fringe image, extracting the fringe deviation characteristics, and calculating the weld seam and weld bead information, wherein the reference laser closed fringe image is a laser closed fringe image formed on a flat plate without a weld seam by the laser emitted by the laser (1).

8. An omnidirectional seam tracking laser vision sensing method, used in an omnidirectional seam tracking welding device according to claim 6 or 7, comprising the following steps: Step S1: using an omnidirectional weld seam tracking laser vision sensor with a welding gun sleeved thereon to scan a flat plate, the specific method is: placing the flat plate directly below the omnidirectional weld seam tracking laser vision sensor with a welding gun sleeved thereon, emitting laser light from a laser to the surface of the flat plate, adjusting the position of the laser to form laser stripes, so that one or more laser stripes form closed stripes on the flat plate that continue around the welding gun; Step S2: the image processing module (7) outputs the flat laser closed stripe image to the information processing module as a reference laser closed stripe image; Step S3: using an omnidirectional weld seam tracking laser vision sensor sleeved with a welding gun to scan the workpiece to be welded in real time, the laser closed stripes surround the working area of ​​the welding gun and intersect the weld seam, and the image processing module (7) outputs the real-time laser closed stripe image to the information processing module; Step S4: the information processing module compares the reference laser closed fringe image with the real-time laser closed fringe image, extracts the characteristics of fringe deviation, obtains weld and weld bead information, and generates welding gun operation control instructions; Repeat steps S3 and S4 until welding is completed.

9. The omnidirectional weld tracking laser vision sensing method according to claim 8, It is characterized in that Before the image processing module (7) outputs the laser closed stripe image, it also includes the step of finding the intersection of the laser stripes to obtain the outer circle contour of the laser closed stripes, and the output laser closed stripe image only includes the outer circle contour.

10. The omnidirectional weld tracking laser vision sensing method according to claim 8, It is characterized in that The weld and weld bead information obtained by the information processing module includes the weld information that has been completed.

Citation Information

Patent Citations

  • Light source alternate strobe synchronous camera shooting method and vision detection system

    CN102780845A

  • Stable precision vision weld seam tracking method

    CN106513937A

  • Annular laser sensor used for welding seam tracking

    CN107900562A

  • Real-time welding seam tracker based on laser vision

    CN219632910U

  • Position detector

    JP1989282403A

Cited By

  • Laser sensor for mirror surface stainless steel welding seam tracking and welding seam tracking identification method

    CN120133706A

  • A laser sensor and a weld tracking identification method for mirror stainless steel weld tracking

    CN120133706B