A method for realizing laser welding by a visual guidance-based mechanical arm

Abstract

The present application relates to nuclear power technology field, especially to a kind of method for realizing laser welding based on visual guidance mechanical arm.The method is: first image acquisition unit roughly identifies the three-dimensional space position of target object;With second image acquisition unit and target detection algorithm, the three-dimensional space position of target welding spot located on target object is accurately identified, mechanical arm carries laser generator and moves, so that laser emitter emission end accurately aims at target welding spot, whether the center of cross mark of coaxial camera coincides with target welding spot position is judged, if coincides, then the position alignment of mechanical arm end carried laser generator is completed once accurately;Mechanical arm end sends instruction signal to control laser generator, emits laser, and completes welding operation.The present application is suitable for the automatic repair of pressurized water reactor core thermocouple connector.

Description

Technical Field

[0001] This invention relates to the field of nuclear power technology, and in particular to a method for laser welding using a vision-guided robotic arm. Background Technology

[0002] Core thermocouples are used to measure the temperature of the coolant at the reactor fuel assembly outlet and in the head chamber of the pressure vessel. These thermocouples are used to calculate important parameters such as the reactor coolant's maximum temperature, average temperature, and minimum subcooling margin, playing a crucial role in reactor accident conditions.

[0003] With the long-term operation of the power plant, the failure rate of the reactor core thermocouple connectors is relatively high. This is mainly manifested in the degradation of the mechanical and electrical performance of the thermocouple connectors due to the disassembly and reassembly of the thermocouples during each refueling period, necessitating the replacement of the connectors. Because the diameter of the thermocouple wires inside the armored cables of the reactor core is only about 0.5mm, it is difficult for personnel to visually align and control the welding process between the thermocouple wires and the connectors. Coupled with the high radiation dose (20-50 mSv / h) and complex procedures at the work site, the on-site welding of thermocouples to connectors becomes a highly technically challenging task.

[0004] With the development of modern industry, robotic arms are increasingly widely used in manufacturing. Among these applications, laser welding technology has become a common processing method. However, traditional laser welding technology for robotic arms has some drawbacks, which to some extent limit its application scope and effectiveness.

[0005] From the perspective of welding results, laser welding technology has gone through the following two stages of development: Phase 1: Early Laser Welding Technology for Robotic Arms Early robotic arm laser welding technology primarily employed traditional control systems and sensor technologies, resulting in low robotic arm precision, inconsistent welding quality, and difficulty in achieving precise control. Furthermore, traditional robotic arm laser welding technology also suffered from low production efficiency and slow welding speed.

[0006] Phase Two: Introduction of Automated Control Systems To further improve the efficiency and precision of robotic arm laser welding technology, researchers have begun to introduce automated control systems. These systems can automate the welding process, thereby reducing the impact of human error on welding quality and improving production efficiency. However, these systems suffer from drawbacks such as high cost, large size, poor flexibility, and limited real-time performance.

[0007] In summary, current robotic arm laser welding technology has several shortcomings that limit its application scope and effectiveness in actual production, especially in industrial scenarios such as reactor core thermocouple repair where the precision and stability requirements for laser welding are extremely stringent. Therefore, new technological approaches are needed to overcome these deficiencies and promote the further development and application of robotic arm laser welding technology. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method for laser welding based on vision-guided robotic arms, which realizes the laser welding technology for thermocouple connectors in pressurized water reactor cores and is applicable to the repair work of thermocouple connectors in all pressurized water reactor cores.

[0009] This invention provides a method for laser welding using a vision-guided robotic arm, comprising the following steps: Step 1: Control the robotic arm to move to the designated initial position; Step 2: The first image acquisition unit deploys a lightweight target detection network to roughly identify the three-dimensional spatial position of the target object and transmits the position information to the robotic arm. The robotic arm receives the current target object position information fed back by the coaxial camera and the first image acquisition unit, and then moves to a position 10-30cm directly in front of the target object. Step 3: Use the second image acquisition unit to acquire target scene data containing thermocouples. Each target in the second image acquisition unit obtains a high-resolution two-dimensional image and the corresponding depth information. The real-time acquired two-dimensional images are input into the trained semantic segmentation network to perform semantic segmentation on the thermocouple core and sheath. In the segmentation results, it is assumed that the intersection area of ​​the even core pixel set and the sleeve pixel set is the seam that needs to be welded. Calculate the geometric center of the intersecting region to obtain the center point coordinates (u,v); Using the intrinsic parameter model of the second image acquisition unit itself, the center point coordinates (u,v) and the corresponding depth value Z are mapped to the three-dimensional spatial coordinates (X,Y,Z) of the target welding point, and the three-dimensional spatial coordinate information is sent to the robotic arm; Step 4: Determine whether the center of the crosshair in the field of view of the coaxial camera mounted on the laser generator coincides with the target welding point. If they coincide, a precise position alignment of the laser generator mounted on the robotic arm is completed, and then proceed to Step 5; if they do not coincide, proceed to Step 1. Step 5: The robotic arm sends a command signal to control the laser generator to emit a laser and complete the welding operation.

[0010] As a further technical solution, the first image acquisition unit adopts a compact close-range RGBD camera, fixed at the global viewing angle, so that the field of view completely covers the thermocouple, connector and surrounding area; the working distance is 0.05~1 m, the depth measurement error is <2 mm, and the weight is <100 g.

[0011] As a further technical solution, the second image acquisition unit adopts an industrial-grade near-field ultra-high precision 3D scanning camera, which is installed at the end of the robotic arm; its Z-axis single-point repeatability measurement accuracy is better than 1 μm, the working distance is 30~300 mm, and the weight is <2 kg.

[0012] As a further technical solution, in step 2, the RGB image acquired by the first image acquisition unit is input into the lightweight target detection network, and the detection result of each target is output: Di=(bi,ci,pi) in: bi=(xi,yi,wi,hi) is the bounding box, where (xi,yi) represents the pixel coordinates of the center point of the box in the image, and wi and hi represent the width and height of the box, respectively. ci is the category label; pi∈[0,1] represents the category confidence, indicating the probability that the target belongs to category ci.

[0013] As a further technical solution, in step 2, valid targets are filtered according to preset category priority and confidence threshold, and the center pixel coordinates (xi,yi) of their bounding boxes are extracted; using the aligned depth map, the depth value Z corresponding to the center pixel is read, and combined with the intrinsic parameter matrix K1 of the first image acquisition device, the three-dimensional coordinates of the coarse positioning point of the target in its own camera coordinate system {C1} are reconstructed. The coarse positioning coordinates are transformed into the robot arm base coordinate system via a chain transformation and transmitted to the robot arm controller in real time.

[0014] As a further technical solution, the category label includes a core, a sleeve, or a background.

[0015] As a further technical solution, the semantic segmentation network defines the weld seam and joint area as an independent semantic label during the training phase, which is listed alongside the category label; during semantic segmentation, the semantic segmentation network directly segments and outputs the target pixel region R representing the joint that needs to be welded.

[0016] As a further technical solution, in step 3, the arithmetic mean of all pixel coordinates of the weld area R is calculated to determine its geometric center point.

[0017] As a further technical solution, the semantic segmentation network is the UPerNet network.

[0018] Compared with the prior art, the method of laser welding based on vision-guided robotic arm of the present invention has the following beneficial effects.

[0019] 1) By combining a low-precision large-field-of-view image acquisition unit with a high-precision small-field-of-view image acquisition unit, it is possible to directly identify target objects from a distance. Then, when the robotic arm gets closer, the high-precision image acquisition unit is used to accurately segment the target object. This makes the whole solution achieve the effect of long recognition distance and accurate segmentation results, thus getting rid of the limitation that a single camera cannot complete long-distance high-precision recognition. 2) In the target recognition stage, the 2D image of the scene under test is processed to determine the position of the target object in the image coordinate system. The 2D image is acquired in real time by a monocular camera included in the 3D vision sensor, and the depth information output synchronously by the sensor is consistent in time and space. The 2D image and 3D point cloud are aligned in the image acquisition unit information to map and obtain the 3D information of the target object in the 2D image, and finally the 3D coordinate information of the target object in the world coordinate system is given. Therefore, the algorithm accuracy can be guaranteed while the detection speed is faster, and the number of model parameters is relatively small, making it suitable for deployment and application on lightweight devices.

[0020] 3) Under vision guidance, a high-precision robotic arm equipped with a laser generator is used for welding operations, which improves the flexibility and real-time performance of the technical solution. Attached Figure Description

[0021] Figure 1 A flowchart illustrating a vision-guided robotic arm method for laser welding. Figure 2 A schematic diagram of the welding apparatus; Figure 3 This is a top view of the welding apparatus. In the figure, 1-first image acquisition unit, 2-first fixed base, 3-second image acquisition unit, 4-second fixed base, 5-coaxial camera, 6-third fixed base, 7-locking module, 8-collimation module, 9-quick locking module, 10-coaxial nozzle, 11-protective lens module, 12-focusing module. Detailed Implementation

[0022] To further understand the present invention, embodiments of the present invention are described below in conjunction with examples. However, it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, and not for limiting the present invention.

[0023] An embodiment of the present invention discloses a method for laser welding using a vision-guided robotic arm, comprising the following steps: Step 1: Control the robotic arm to move to the designated initial position; Step 2: The first image acquisition unit deploys a lightweight target detection network to roughly identify the three-dimensional spatial position of the target object and transmits the position information to the robotic arm. The robotic arm receives the current target object position information fed back by the coaxial camera and the first image acquisition unit, and then moves to a position 10-30cm directly in front of the target object. The first image acquisition unit deploys a lightweight object detection network to perform real-time inference on the current frame; the lightweight object detection network has a YOLOX dimension. For the i-th detected target, the network outputs a quadruple Di: Di=(bi,ci,pi) in: bi=(xi,yi,wi,hi) is the bounding box, where (xi,yi) represents the pixel coordinates of the center point of the box in the image, and wi and hi represent the width and height of the box, respectively.

[0024] ci is the category label; pi∈[0,1] represents the category confidence, indicating the probability that the target belongs to category ci.

[0025] Only when pi is higher than the preset threshold T conf At that time, the corresponding test results are adopted and used for subsequent processing.

[0026] Valid targets are selected based on preset category priority and confidence threshold, and the center pixel coordinates (xi,yi) of their bounding boxes are extracted. Using the aligned depth map, the depth value Z corresponding to the center pixel is read, and combined with the intrinsic parameter matrix K1 of the first image acquisition device, the three-dimensional coordinates of the coarse positioning point of the target in its own camera coordinate system {C1} are reconstructed. The coarse positioning coordinates are transformed into the robot arm base coordinate system via a chain transformation and transmitted to the robot arm controller in real time.

[0027] This step enables focusing from the entire scene to a specific area.

[0028] The first image acquisition unit may be a compact near-field RGBD camera. This type of camera is characterized by integrating the optical system and depth calculation unit into a single module, and optimizing the depth perception mode within a near-field working range of 0.05m to 1m, so as to simultaneously output a color image with a resolution no lower than VGA and a pixel-aligned depth map within a limited space.

[0029] Based on the principle of depth measurement, the first image acquisition unit can be, but is not limited to, any of the following types of sensor modules: An active binocular stereo vision module includes two infrared image sensors and an infrared pattern projector, which calculates parallax by matching the left and right infrared images to obtain depth information. The indirect time-of-flight (iToF) module includes a modulated light source and a ToF image sensor, which calculates depth information by emitting modulated infrared light and measuring the phase difference of the reflected light; The structured light coding module includes an infrared projector and an infrared camera, which calculates depth information by projecting a known coded pattern and decoding the deformation.

[0030] All of the above-mentioned sensor modules must meet the following requirements: at the preset nearest working distance, the random depth error is less than 2mm, and the weight of the whole machine is less than 100g, so as to adapt to the compact space of the welding station and the initial positioning requirements of metal reflective targets.

[0031] In one specific embodiment, the first image acquisition unit is an Intel RealSense D405 camera, which has a depth error of less than 1mm in the range of 7cm to 50cm and weighs about 40g, which can meet the coarse positioning requirements of this application.

[0032] Those skilled in the art will understand that other commercial modules that meet the above performance boundaries, such as Orbbec Gemini 2 and Infineon IRS2975C ​​series, can also be used as alternatives.

[0033] Step 3: Use the second image acquisition unit to acquire target scene data containing thermocouples. Each target in the second image acquisition unit obtains a high-resolution two-dimensional image and the corresponding depth information. The real-time acquired two-dimensional images are input into the trained UPerNet network to perform semantic segmentation of the thermocouple core and sheath. In the segmentation results, it is assumed that the intersection area of ​​the even core pixel set and the sleeve pixel set is the seam that needs to be welded. Calculate the geometric center of the intersecting region to obtain the center point coordinates (u,v); Using the intrinsic parameter model of the second image acquisition unit itself, the center point coordinates (u,v) and the corresponding depth value Z are mapped to the three-dimensional spatial coordinates (X,Y,Z) of the target welding point, and the three-dimensional spatial coordinate information is sent to the robotic arm; The second image acquisition unit is configured to perform precise three-dimensional perception on the local area locked by the target search unit in order to obtain the high-precision three-dimensional coordinates of the surface feature points of the target object in a preset spatial coordinate system.

[0034] The second image acquisition unit has a Z-axis depth measurement accuracy better than 1 micrometer, weighs less than 2 kilograms, and has a working distance range covering 30 millimeters to 300 millimeters, making it suitable for installation in a compact nuclear environment shielded workstation and capable of distinguishing micrometer-level morphological undulations and edge features on the surface of the target object.

[0035] The second image acquisition unit can be an industrial-grade near-field ultra-high precision 3D scanning camera. The characteristic of this type of camera is that its depth perception principle adopts at least one of the following methods: coded structured light method, laser triangulation method, or laser line scanning method. It reconstructs the 3D point cloud of the target surface by projecting a preset coded optical pattern onto the target surface and analyzing its deformation, or by scanning a line laser and extracting the cross-sectional contour.

[0036] This type of camera must meet the following performance boundaries: At the rated working distance, the Z-axis single-point repeatability measurement accuracy is less than 1 micrometer; The spatial sampling interval of the depth point cloud generated by a single imaging or single scan is less than 20 micrometers; The module weighs less than 2 kg and has a gigabit Ethernet or USB 3.0 high-speed digital interface as its external physical interface.

[0037] In one specific embodiment, the second image acquisition unit uses the Shengxiang Technology M051040 structured light camera, which has a nominal Z-axis accuracy better than 0.1 micrometers and weighs approximately 0.52 kilograms. It can meet the requirements for precise 3D reconstruction of thermocouple solder joints and their surrounding micrometer-level features. Those skilled in the art will understand that other industrial sensors that meet the aforementioned accuracy and morphological boundaries, such as the LMI Gocator 3504 and Keyence VR-6200, can also be used as alternatives to the second image acquisition unit.

[0038] Step 4: Determine whether the center of the crosshair in the field of view of the coaxial camera mounted on the laser generator coincides with the target welding point. If they coincide, a precise position alignment of the laser generator mounted on the robotic arm is completed, and then proceed to Step 5; if they do not coincide, control the robotic arm to return to a fixed position, and then execute Step 1. Step 5: The robotic arm sends a command signal to control the laser generator to emit a laser and complete the welding operation; Step 6: The robotic arm returns to its initial position; An embodiment of the present invention discloses a welding device, comprising: a welding head and a robotic arm; the welding head is connected to a pull-rod type instrument case via an optical cable; a laser generator and related electrical components are integrated inside the pull-rod type instrument case; The welding head is as follows Figure 2 and Figure 3As shown, the first image acquisition unit 1 is mounted on the upper part of the housing via the first fixing base 2, and the second image acquisition unit 3 is mounted on the side of the housing via the second fixing base 4. The second image acquisition unit 3 is located on one side of the first image acquisition unit 1, and the two together constitute the side visual monitoring unit. The coaxial camera 5 is connected to the upper end of the third mounting base 6. The third mounting base 6 integrates a reflector, and the collimation module 8 and the locking module 7 are connected in sequence below it.

[0039] The coaxial camera 5 provides the possibility of real-time observation of the relative position between the laser focus and the weld seam on the workpiece, thereby assisting the operator in precisely adjusting the position of the focusing lens to ensure that the laser focus falls clearly on the workpiece surface and providing image data for welding quality traceability. The locking module 7 is used to connect the laser's QBH fiber optic connector, while the collimation module 8 is connected to the robot's manipulator on the side via the quick-locking module 9, realizing the mechanical fixation and rapid assembly / disassembly of the device.

[0040] The third fixed base 6 is connected in series with the focusing module 12, the protective lens module 11 and the coaxial air nozzle 10. The focusing module 12 has a built-in focusing lens to complete the laser focusing. The protective lens module 11 blocks smoke and splashes to protect the internal optical components. The coaxial air nozzle 10 delivers protective gas to the welding workpiece to form an air curtain for protection and cleaning. The connection part of the housing is located between the protective mirror module 11 and the coaxial air nozzle 10; Overall, the first image acquisition unit 1 and the second image acquisition unit 3 are located on the upper part and right side of the device, the coaxial camera 5 is arranged in the center, the laser light path is transmitted from bottom to top in the vertical direction, the mechanical installation and air path protection components are distributed on the periphery and end of the light path respectively, and each module is tightly connected by the shell, the fixed base and the threaded structure.

[0041] After multiple measurements and verifications, a high-precision camera positioned 100±5mm from the center of the thermocouple wire was used to acquire images. Applying relevant semantic segmentation algorithms, the camera could effectively segment the thermocouple wire at the millimeter level. Based on welding requirements, the 3D error of the thermocouple segmentation information in the camera met the accuracy requirements.

[0042] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0043] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for laser welding using a vision-guided robotic arm, characterized in that, Includes the following steps: Step 1: Control the robotic arm to move to the designated initial position; Step 2: The first image acquisition unit deploys a lightweight target detection network to roughly identify the three-dimensional spatial position of the target object and transmits the position information to the robotic arm. The robotic arm receives the current target object position information fed back by the coaxial camera and the first image acquisition unit, and then moves to a position 10-30cm directly in front of the target object. Step 3: Use the second image acquisition unit to acquire target scene data containing thermocouples. Each target in the second image acquisition unit obtains a high-resolution two-dimensional image and the corresponding depth information. The real-time acquired two-dimensional images are input into the trained semantic segmentation network to perform semantic segmentation on the thermocouple core and sheath. In the segmentation results, it is assumed that the intersection area of ​​the even core pixel set and the sleeve pixel set is the seam that needs to be welded. Calculate the geometric center of the intersecting region to obtain the center point coordinates (u,v); Using the intrinsic parameter model of the second image acquisition unit itself, the center point coordinates (u,v) and the corresponding depth value Z are mapped to the three-dimensional spatial coordinates (X,Y,Z) of the target welding point, and the three-dimensional spatial coordinate information is sent to the robotic arm; Step 4: Determine whether the center of the crosshair in the field of view of the coaxial camera mounted on the laser generator coincides with the target welding point. If they coincide, a precise position alignment of the laser generator mounted on the robotic arm is completed, and then proceed to Step 5; if they do not coincide, proceed to Step 1. Step 5: The robotic arm sends a command signal to control the laser generator to emit a laser and complete the welding operation.

2. The method for laser welding using a vision-guided robotic arm according to claim 1, characterized in that, The first image acquisition unit uses a compact close-range RGBD camera, fixed at the global viewpoint, so that the field of view completely covers the thermocouple, connector and surrounding area; the working distance is 0.05~1 m, the depth measurement error is <2 mm, and the weight is <100 g.

3. The method for laser welding using a vision-guided robotic arm according to claim 1, characterized in that, The second image acquisition unit uses an industrial-grade near-field ultra-high precision 3D scanning camera, which is installed at the end of the robotic arm; its Z-axis single-point repeatability measurement accuracy is better than 1 μm, the working distance is 30~300 mm, and the weight is <2 kg.

4. The method for laser welding using a vision-guided robotic arm according to claim 1, characterized in that, In step 2, the RGB image acquired by the first image acquisition unit is input into the lightweight target detection network, and the detection result of each target is output: Di=(bi,ci,pi) in: bi=(xi,yi,wi,hi) is the bounding box, where (xi,yi) represents the pixel coordinates of the center point of the box in the image, and wi and hi represent the width and height of the box, respectively. ci is the category label; pi∈[0,1] represents the category confidence, indicating the probability that the target belongs to category ci.

5. The method for laser welding using a vision-guided robotic arm according to claim 4, characterized in that, In step 2, valid targets are filtered according to preset category priority and confidence threshold, and the center pixel coordinates (xi,yi) of their bounding boxes are extracted; using the aligned depth map, the depth value Z corresponding to the center pixel is read, and combined with the intrinsic parameter matrix K1 of the first image acquisition device, the three-dimensional coordinates of the coarse positioning point of the target in its own camera coordinate system {C1} are reconstructed. The coarse positioning coordinates are transformed into the robot arm base coordinate system via a chain transformation and transmitted to the robot arm controller in real time.

6. The method for laser welding using a vision-guided robotic arm according to claim 1, characterized in that, The category labels include core, sleeve, or background.

7. The method for laser welding using a vision-guided robotic arm according to claim 6, characterized in that, During the training phase, the semantic segmentation network defines weld seams and joint areas as an independent semantic label, which is listed alongside the category label. During semantic segmentation, the semantic segmentation network directly segments and outputs the target pixel region R representing the joint that needs to be welded.

8. The method for laser welding using a vision-guided robotic arm according to claim 1, characterized in that, In step 3, the arithmetic mean of all pixel coordinates of the weld area R is calculated to determine its geometric center point.

9. The method for laser welding using a vision-guided robotic arm according to claim 1, characterized in that, The semantic segmentation network is the UPerNet network.

Images