Weld seam identification and tracking system and method based on 2D / 3D camera

Abstract

The present invention provides a weld seam recognition and weld seam tracking system and method based on 2D visual sensor and 3D camera, including industrial computer, welding robot, welding machine, 2D visual sensor, 3D camera, offline programming system, weld seam recognition and weld seam tracking software system, the industrial computer is used to install the weld seam recognition and weld seam tracking software system; the welding robot drives the welding gun to perform welding operation; the welding machine controls the welding robot to perform welding tasks; the offline programming system plans the weld seam and controls the movement of the welding robot; the weld seam recognition and weld seam tracking software system is installed on the industrial computer and is responsible for the process control of the whole system. By combining the 3D camera and the 2D visual sensor, the point cloud captured by the 3D camera can identify all weld seams in the point cloud and drive the robot to weld, which is much more efficient than weld tracking. When the 3D point cloud registration and point cloud matching fail, the weld seam can be located and tracked in real time to improve the welding accuracy.

Description

Background Art

[0002] In the process of intelligent robot welding, the key to achieving intelligence lies in the application of visual technology.

[0003] At present, the mainstream visual technologies are divided into 2D and 3D technologies. Among them, 2D visual sensors can realize real-time tracking of welds, and 3D technology uses 3D cameras to capture the three-dimensional point cloud of the welded workpiece, and uses point cloud registration and other processing technologies to identify all welds on the workpiece point cloud. However, 2D technology can only weld one weld at a time, which is relatively inefficient. The use of 3D technology requires a high degree of matching of the weld template and cannot achieve real-time tracking. Summary of the invention

[0004] In view of the defects in the prior art, the object of the present invention is to provide a weld identification and weld tracking system and method based on a 2D visual sensor and a 3D camera.

[0005] In the first aspect, the present application provides a weld identification and weld tracking system based on a 2D vision sensor and a 3D camera, including: an industrial computer, a welding robot, a welding machine, a 2D vision sensor, a 3D camera, and an offline programming system, wherein:

[0006] The industrial computer is loaded with a visual processing module, which is used to perform process control of weld seam identification and weld seam tracking according to the image and point cloud information of the welding workpiece collected by the 2D visual sensor and the 3D camera;

[0007] The welding robot is used to drive the welding gun to perform welding operations;

[0008] The welding machine is used to control the welding robot to perform welding tasks and provide power;

[0009] The 2D visual sensor and the 3D camera are used to collect images and point cloud information of the welding workpiece;

[0010] The offline programming system is used to plan the welding seam and control the movement of the welding robot.

[0011] Optionally, the 2D vision sensor and the 3D camera are respectively mounted on both sides of the welding gun through a clamp.

[0012] Optionally, the welding gun adopts an extended gooseneck welding gun, and the 2D visual sensor and the 3D camera are freely adjusted within the angle range of 0 to 60 degrees through a clamp.

[0013] Optionally, the visual processing module in the industrial computer includes: a 3D point cloud registration module, a 2D visual sensor module, and a 3D point cloud weld recognition module;

[0014] The 3D point cloud registration module is used to perform point cloud filtering, point cloud downsampling, and point cloud registration processing on the point cloud information of the welding workpiece collected by the 3D camera, and send the processing results to the offline programming system so that the offline programming system performs weld positioning and planning;

[0015] The 3D point cloud weld recognition module is used to match the workpiece 3D point cloud through the loaded weld type template to identify all welds of the workpiece when the 3D point cloud registration fails;

[0016] The 2D visual sensor module is used to locate the weld by loading the weld template and track the weld when the 3D point cloud weld recognition fails.

[0017] Optionally, the point cloud filtering algorithms adopted by the 3D point cloud registration module include: radius filtering algorithm and statistical filtering algorithm; the point cloud registration algorithms adopted by the 3D point cloud registration module include: ICP and NICP algorithms.

[0018] In a second aspect, an embodiment of the present application provides a weld seam identification and weld seam tracking method based on a 2D visual sensor and a 3D camera, which is applied to the weld seam identification and weld seam tracking system based on a 2D visual sensor and a 3D camera as described in any one of the first aspects, and the method includes:

[0019] Step 1: Plan the weld path of the workpiece according to the 3D point cloud registration results;

[0020] Step 2: If the 3D point cloud registration fails, the weld type template is called to match the 3D point cloud of the workpiece to identify all welds of the workpiece;

[0021] Step 3: If the weld type template fails to match the 3D point cloud of the workpiece, a 2D vision sensor is used to locate the weld, and after the location is completed, the welding robot is driven to track the weld.

[0022] Optionally, step 1 includes:

[0023] Step 1.1: Load the workpiece model point cloud;

[0024] Step 1.2: Use a 3D camera to photograph the workpiece and obtain a point cloud of the workpiece;

[0025] Step 1.3: Perform point cloud registration processing on the workpiece model point cloud and the workpiece point cloud to obtain the rotation and translation matrix between the point clouds;

[0026] Step 1.4: Process the rotation and translation matrix through the offline programming system to obtain the precise positioning result;

[0027] Step 1.5: The offline programming system plans a complete weld path according to the precise positioning result.

[0028] Optionally, the step 2 includes:

[0029] Step 2.1: If step 1.3 fails, retrieve the workpiece point cloud obtained in step 1.2;

[0030] Step 2.2: Select a weld type template that matches the workpiece, and identify all welds in the workpiece using a point cloud matching method based on the weld type template;

[0031] Step 2.3: Transmit the position information corresponding to all welds to the offline programming system via TCP communication for subsequent welding operations.

[0032] Optionally, the step 3 includes:

[0033] Step 3.1: Initial welding point location is performed through 3D camera point cloud processing;

[0034] Step 3.2: If 3D recognition fails, the initial solder joint location is performed using a 2D visual sensor;

[0035] Step 3.3: Starting welding from the initial welding point, the 2D vision sensor continuously captures weld feature points during the welding process;

[0036] Step 3.4: Transmit the weld feature points to an offline programming system, so that the offline programming system drives the welding robot to perform weld tracking.

[0037] In a third aspect, an embodiment of the present application provides a weld identification and weld tracking device based on a 2D visual sensor and a 3D camera, comprising: a processor and a memory, wherein executable program instructions are stored in the memory, and when the processor calls the program instructions in the memory, the processor is used to:

[0038] Execute the steps of the weld seam identification and weld seam tracking method based on a 2D visual sensor and a 3D camera as described in any one of the second aspects.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of the weld identification and weld tracking method based on a 2D visual sensor and a 3D camera as described in any one of the first aspects.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1) The weld seam identification and weld seam tracking system and method based on 2D visual sensors and 3D cameras provided in this application adopts a combination of 3D cameras and 2D visual sensors. A point cloud captured by a 3D camera can identify all weld seams in the point cloud, and can directly drive the robot to weld. Its efficiency is much higher than weld seam tracking. When 3D point cloud registration and point cloud matching fail, the weld seam can be tracked in real time to improve welding accuracy.

[0042] 2) The weld seam identification and weld seam tracking system and method based on 2D vision sensor and 3D camera provided in this application has easy-to-use visual software. Even personnel without professional training can learn how to use the software in a short time. It has strong scalability and is compatible with various brands of 3D cameras or 2D vision sensors. 3) The weld seam identification and weld seam tracking system and method based on 2D vision sensor and 3D camera provided in this application can realize communication with different 2D vision software / 3D vision software by separating the welding mode control module from the 2D / 3D vision module, achieving high cohesion and low coupling. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0044] Figure 1 A schematic diagram of a flow chart of a weld seam identification and weld seam tracking method based on a 2D visual sensor and a 3D camera provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a visual system composed of a 2D visual sensor and a 3D camera provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the interface of the visual processing software provided in an embodiment of the present application.

[0047] In the figure: 1-2D vision sensor, 2-3D camera, 3-gooseneck welding gun, 4-fixture, 5-3D point cloud registration module, 6-3D point cloud recognition module, 7-2D vision sensor module, 8-robot control module, 9-welding mode control module. DETAILED DESCRIPTION

[0048] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0049] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for fixing or for circuit connection.

[0050] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0051] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0052] The weld seam identification and tracking system based on 2D visual sensors and 3D cameras provided in the embodiment of the present application mainly includes: an industrial computer, a welding robot, a welding machine, a 2D visual sensor, a 3D camera, an offline programming system, etc. The industrial computer is used to install the visual software written by the present invention, which is responsible for the process control of the entire system. The welding robot is an actuator responsible for driving the welding gun to perform welding operations. The welding machine is responsible for the robot's motion execution and power supply. The 2D visual sensor and 3D camera are used to collect images and point cloud information of the welding workpiece. The offline programming system is used to plan the weld seam in the 3D module and control the motion of the welding robot. The 2D visual sensor and 3D camera obtain workpiece information and perform TCP communication with the visual software to transmit data.

[0053] Exemplarily, the 2D vision sensor and the 3D camera are installed on both sides of the welding gun through a special fixture. The welding gun is an extended gooseneck welding gun with sufficient space above. The cameras on both sides of the fixture have an adjustable angle of 60°, which can ensure that both cameras can shoot the workpiece in the best posture.

[0054] The 3D point cloud registration module in this embodiment is independently developed, and mainly includes: camera control module, precision positioning module, point cloud processing module, offline programming system communication module, etc. Among them, the point cloud registration module mainly includes: point cloud filtering, point cloud downsampling, point cloud registration, etc. The point cloud filtering algorithms used include radius filtering algorithm and statistical filtering algorithm; point cloud downsampling should reduce the point cloud density without losing point cloud features; the point cloud registration algorithms used include ICP and NICP algorithms.

[0055] The 3D point cloud recognition weld module in this embodiment mainly includes: a camera control module, a robot communication module, a weld template module, a point cloud recognition module, a function testing module, etc.

[0056] The visual sensor module in this embodiment mainly includes: a camera control module, a weld template module, an offline programming system communication module, a calibration module, a functional testing module, etc.

[0057] Figure 1 The present invention provides a flow chart of a weld seam identification and weld seam tracking method based on a 2D visual sensor and a 3D camera, which is applied to the above-mentioned system. Figure 1 As shown in the figure, first install the 2D vision sensor and 3D camera to the gooseneck of the welding gun through a special fixture, and adjust the installation angle so that both cameras have the best shooting angle; the vision software starts and completes the initialization of the 2D module, 3D module, and robot control module. Perform internal reference calibration and hand-eye calibration for the 2D vision sensor and 3D camera respectively. The vision software loads the model point cloud of the current welding workpiece; the vision software uses the robot control module to operate the robot to move near the welding workpiece to ensure that the main feature parts of the workpiece can be photographed at present; the 3D module shoots the workpiece point cloud; the model point cloud and the photographed workpiece point cloud are point cloud registered to obtain the rotation and translation matrix between the two point clouds.

[0058] Optionally, if the registration is successful, the obtained rotation and translation matrix, i.e., the precise positioning result, is transmitted to the offline programming system, i.e., the robot control module; the offline programming system performs weld planning on the model. The visual software uses the robot control module to operate the robot to move near the welding workpiece to ensure that all weld features of the workpiece can be captured, all welds should be in the middle of the captured workpiece point cloud, and the workpiece point cloud should occupy as large a field of view as possible of the 3D camera; based on the current workpiece type, a suitable weld matching template is selected for point cloud recognition.

[0059] Optionally, if the point cloud recognition is successful, a suitable point cloud matching template is selected according to the weld type of the actual welded workpiece. In this embodiment, a V-shaped template can be selected for point cloud matching; the weld feature point data obtained by recognition is stored in a corresponding data structure and transmitted to the offline programming system, i.e., the robot control module; the offline programming system controls the robot to perform welding according to the weld feature point data.

[0060] In this embodiment, the 3D module recognizes all welds at once, and the offline programming system can control the robot to directly complete the welding of all welds at once. On the premise that the welds of the welded workpiece are regular, the efficiency of intelligent welding can be greatly improved.

[0061] In this embodiment, the robot is operated by the visual software to move to the vicinity of the welding starting point; the 3D module captures the workpiece point cloud and processes the point cloud to obtain the initial weld point. If the 3D acquisition fails, the 2D module performs the initial weld point location search through line laser scanning to identify the starting weld point of the current weld; the robot returns to the initial weld point position to start welding, and the 2D module continuously collects a large amount of weld feature point data until the welding is completed; the collected data is transmitted to the offline programming system in real time through TCP communication, and the offline programming system controls the robot movement in real time to achieve weld tracking. The 2D module can achieve real-time weld tracking, but it sacrifices the efficiency of intelligent welding. It is generally used when the 3D camera fails to identify the weld or when there is a need for real-time tracking.

[0062] It should be noted that the three different weld identification or weld tracking methods are recommended to be performed in the order of 3D point cloud registration, 3D point cloud identification, and 2D weld tracking. In actual scenarios, operators can flexibly select according to the on-site conditions. This embodiment does not limit the execution order of the above methods.

[0063] Figure 2 A schematic diagram of the structure of a visual system composed of a 2D visual sensor and a 3D camera provided in an embodiment of the present application, such as Figure 2 As shown, it includes: a 2D vision sensor, a 3D camera, and an extended gooseneck welding gun. The two cameras are connected to the welding gun through a special clamp.

[0064] Figure 3 The interface diagram of the visual processing software provided in the embodiment of the present application is as follows: Figure 3 As shown in the figure, it includes: 3D point cloud registration module, 3D point cloud recognition module, 2D visual sensor module, robot control module, and welding mode control module. The selection of weld recognition and weld tracking mode is carried out in the weld mode control module. In the weld mode control module, 3D point cloud registration, 3D point cloud recognition, and 2D visual sensing are divided into three modes, and operators can flexibly choose according to scene requirements.

[0065] Exemplarily, an embodiment of the present application provides a method for using the above-mentioned weld seam identification and weld seam tracking system based on a 2D visual sensor and a 3D camera, comprising the following specific steps:

[0066] S1: The 3D camera captures the 3D point cloud of the welding workpiece, and the offline programming system loads the model point cloud of the workpiece, and attempts to use the point cloud registration method for precise positioning.

[0067] In this embodiment, step S1 includes:

[0068] Step S11: The 3D module loads the workpiece model point cloud;

[0069] Step S12: The 3D module photographs the workpiece point cloud;

[0070] Step S13: performing point cloud registration on the two point clouds to obtain a rotation and translation matrix between the two point clouds;

[0071] Step S14: Obtain the rotation and translation matrix in S13, i.e., the precise positioning result, by offline programming;

[0072] Step S15: Offline programming to plan the complete weld path.

[0073] In this embodiment, step S2 includes:

[0074] Step S21: using the workpiece point cloud obtained in step S12 in step S1;

[0075] Step S22: Select a weld type template that matches the workpiece, and use point cloud processing technology to identify all welds in the workpiece.

[0076] Step S23: Transmit the recognition result in S22 to the offline programming system via TCP communication for welding.

[0077] In this embodiment, step S3 includes:

[0078] Step S31: initial welding point location is performed through 3D camera point cloud processing;

[0079] Step S32: If 3D acquisition fails, the initial welding point location is searched by a 2D visual sensor;

[0080] Step S33: Welding starts from the initial welding point. During the welding process, the 2D vision sensor continuously captures the weld feature points and transmits them to the offline programming system to drive the robot movement to achieve weld tracking.

[0081] S2: Step S2 is performed only when the registration in step S1 fails, the 3D camera captures the 3D point cloud of the welded workpiece, and attempts to identify all welds in the 3D point cloud of the workpiece using a template matching method;

[0082] S3: Step S3 is performed only when step S2 recognition fails or the scene requires weld tracking. The 2D visual sensor is used to locate the weld. After the location is found, the robot is driven to track the weld.

[0083] In this embodiment, the 2D vision sensor and the 3D camera are installed on both sides of the welding gun through a special clamp. The welding gun is an extended gooseneck welding gun with sufficient space above. The cameras on both sides of the clamp have an adjustable angle of 60°, which can ensure that both cameras can shoot the workpiece in the best posture.

[0084] In this embodiment, the 3D camera used can identify all welds in the point cloud by taking a picture of the point cloud, and can directly drive the robot to perform welding, which is much more efficient than weld tracking.

[0085] In this embodiment, the visual sensor used is based on line laser. When higher welding accuracy is required or 3D camera recognition fails, the weld can be tracked in real time to improve welding accuracy.

[0086] The visual software written in this embodiment is easy to operate, and even personnel without professional training can learn how to use the software in a short time.

[0087] The visual software written in this embodiment can communicate with different 2D vision software / 3D vision software by separating the welding mode control module from the 2D / 3D vision module, thereby achieving high cohesion and low coupling.

[0088] The visual software written in this embodiment is highly extensible and compatible with various brands of 3D cameras or 2D visual sensors.

[0089] The embodiment of the present application also provides a 2D / 3D camera-based weld identification and weld tracking device which may include: a processor and a memory.

[0090] The memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above methods), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0091] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0092] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment.

[0093] For details, please refer to the relevant description in the previous method embodiment.

[0094] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0095] The weld identification and weld tracking device based on 2D visual sensor and 3D camera of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles refer to the relevant description in the above method, which will not be repeated here.

[0096] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "platform".

[0097] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When at least one processor of a user device executes the computer-executable instructions, the user device executes the above-mentioned various possible methods. Among them, the computer-readable medium includes a computer storage medium and a communication medium, wherein the communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also exist in a communication device as discrete components.

[0098] The above is the core idea of ​​the present invention. In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0099] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features disclosed herein. The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which does not affect the substantive content of the present invention.

Claims

1. A weld seam identification and tracking system based on 2D visual sensor and 3D camera, characterized in that: include: Industrial computer, welding robot, welding machine, 2D vision sensor, 3D camera, offline programming system, including: The industrial computer is loaded with a visual processing module, which is used to perform process control of weld identification and weld tracking according to the image and point cloud information of the welding workpiece collected by the 2D visual sensor and the 3D camera; the visual processing module in the industrial computer includes: a 3D point cloud registration module, a 2D visual sensor module, and a 3D point cloud weld recognition module; the 3D point cloud registration module is used to perform point cloud filtering, point cloud downsampling, and point cloud registration processing on the point cloud information of the welding workpiece collected by the 3D camera, and send the processing results to the offline programming system so that the offline programming system performs weld positioning and planning; the 3D point cloud weld recognition module is used to match the 3D point cloud of the workpiece through the loaded weld type template when the 3D point cloud registration fails, and identify all the welds of the workpiece; the 2D visual sensor module is used to perform weld location and weld tracking through the 2D visual sensor when the 3D point cloud weld recognition fails; The welding robot is used to drive the welding gun to perform welding operations; The welding machine is used to control the welding robot to perform welding tasks and provide power; The 2D visual sensor and the 3D camera are used to collect images and point cloud information of the welding workpiece; The offline programming system is used to plan the welding seam and control the movement of the welding robot.

2. The weld seam identification and weld seam tracking system based on 2D visual sensor and 3D camera according to claim 1, characterized in that: The 2D vision sensor and the 3D camera are respectively mounted on two sides of the welding gun through a clamp.

3. The weld seam identification and weld seam tracking system based on 2D visual sensor and 3D camera according to claim 2 is characterized in that: The welding gun adopts an extended gooseneck welding gun, and the 2D visual sensor and the 3D camera are freely adjusted within the angle range of 0 to 60 degrees through a clamp.

4. The weld seam identification and weld seam tracking system based on 2D visual sensor and 3D camera according to claim 1, characterized in that: The point cloud filtering algorithms used by the 3D point cloud registration module include: radius filtering algorithm and statistical filtering algorithm; the point cloud registration algorithms used by the 3D point cloud registration module include: ICP and NICP algorithms.

5. A weld seam identification and weld seam tracking method based on 2D visual sensor and 3D camera, characterized in that: In the weld seam identification and weld seam tracking system based on a 2D visual sensor and a 3D camera as described in any one of claims 1 to 4, the method comprises: Step 1: Plan the weld path of the workpiece according to the 3D point cloud registration results; Step 2: If the 3D point cloud registration fails, the weld type template is called to match the 3D point cloud of the workpiece to identify all welds of the workpiece; Step 3: If the weld type template fails to match the 3D point cloud of the workpiece, the weld is located and after the location is completed, the welding robot is driven to track the weld.

6. The weld seam identification and weld seam tracking method based on 2D visual sensor and 3D camera according to claim 5, characterized in that: The step 1 comprises: Step 1.1: Load the workpiece model point cloud; Step 1.2: Use a 3D camera to photograph the workpiece and obtain a point cloud of the workpiece; Step 1.3: Perform point cloud registration processing on the workpiece model point cloud and the workpiece point cloud to obtain the rotation and translation matrix between the point clouds; Step 1.4: Process the rotation and translation matrix through the offline programming system to obtain the precise positioning result; Step 1.5: The offline programming system plans a complete weld path according to the precise positioning result.

7. The weld seam identification and weld seam tracking method based on 2D visual sensor and 3D camera according to claim 6, characterized in that: The step 2 comprises: Step 2.1: If step 1.3 fails, retrieve the workpiece point cloud obtained in step 1.2; Step 2.2: Select a weld type template that matches the workpiece, and identify all welds in the workpiece using a point cloud matching method based on the weld type template; Step 2.3: Transmit the position information corresponding to all welds to the offline programming system via TCP communication for subsequent welding operations.

8. The weld seam identification and weld seam tracking method based on 2D visual sensor and 3D camera according to claim 6, characterized in that: The step 3 comprises: Step 3.1: Initial welding point location is performed through 3D camera point cloud processing; Step 3.2: If 3D recognition fails, the initial solder joint location is performed using a 2D visual sensor; Step 3.3: Starting welding from the initial welding point, the 2D vision sensor continuously captures weld feature points during the welding process; Step 3.4: Transmit the weld feature points to an offline programming system, so that the offline programming system drives the welding robot to perform weld tracking.

9. A weld seam identification and weld seam tracking device based on a 2D visual sensor and a 3D camera, characterized in that: include: A processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to: Execute the steps of the weld seam identification and weld seam tracking method based on a 2D visual sensor and a 3D camera as described in any one of claims 5-8.

Images