Tube sheet workpiece welding method and system

Abstract

The application relates to a tube plate workpiece welding method and system, which comprises the following steps: acquiring global information collected for a tube plate workpiece; the tube plate workpiece comprises a plurality of tube holes; based on the global information, the position information of each tube hole is determined; for each tube hole, based on the position information of the tube hole, the information collection pose of the tube hole is determined; based on the tube hole information collected for the tube hole in the information collection pose, the expected welding track of the tube hole is fitted; and the welding robot is controlled to weld the tube plate workpiece according to the expected welding track of each tube hole. The above method can reduce the labor cost.

Description

Technical Field

[0001] This application relates to the field of welding processing technology, and in particular to a method, apparatus, system, computer equipment, computer-readable storage medium, and computer program product for welding tube sheet workpieces. Background Technology

[0002] Tube sheets are devices used to separate or connect different fluid flows. They are key components for ensuring fluid flow control and processing, and are widely used in equipment such as reactors, distillation columns, heat exchangers, boilers, and gas turbines. In practical applications, tube sheet components need to be welded to heat exchange tubes.

[0003] In traditional techniques, due to the large number and dense distribution of tube holes to be welded in tube sheet workpieces, welding is usually performed manually. With the development of technology, semi-automatic welding methods based on pneumatic expansion welding machines or teach-in welding robots have emerged. However, all of the above welding methods still require manual input, resulting in high labor costs. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, system, computer equipment, computer-readable storage medium, and computer program product for welding tube sheet workpieces that can reduce labor costs, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for welding tube sheet workpieces. The method includes:

[0006] Acquire global information collected from the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes.

[0007] Based on global information, the location information of each pipe hole is determined;

[0008] For each borehole, the information acquisition pose for that borehole is determined based on its location information.

[0009] Based on the pipe hole information acquired under the information acquisition pose, the desired welding trajectory of the pipe hole is obtained by fitting.

[0010] The welding robot is controlled to weld the tube sheet workpiece according to the desired welding trajectory of each tube hole.

[0011] Secondly, this application also provides a tube sheet workpiece welding apparatus. The apparatus includes:

[0012] The global information acquisition module is used to acquire global information collected from the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes.

[0013] The location information determination module is used to determine the location information of each borehole based on global information;

[0014] The information acquisition pose determination module is used to determine the information acquisition pose for each pipe hole based on the pipe hole's position information.

[0015] The trajectory fitting module is used to fit the desired welding trajectory of the pipe hole based on the pipe hole information acquired under the information acquisition pose.

[0016] The welding module is used to control the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each tube hole.

[0017] Thirdly, this application also provides a tube sheet workpiece welding system. The system includes a controller, a global acquisition device, a local acquisition device, and a welding robot connected to the controller;

[0018] The global acquisition device is used to acquire global information of the tube sheet workpiece to obtain global information of the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes;

[0019] The local acquisition device is used to collect information for each pipe hole separately, and obtain the pipe hole information for each pipe hole.

[0020] Welding robots are used for welding tube sheet workpieces;

[0021] The controller is used to implement the above-mentioned tube sheet workpiece welding method.

[0022] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the following method:

[0023] Acquire global information collected from the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes.

[0024] Based on global information, the location information of each pipe hole is determined;

[0025] For each borehole, the information acquisition pose for that borehole is determined based on its location information.

[0026] Based on the pipe hole information acquired under the information acquisition pose, the desired welding trajectory of the pipe hole is obtained by fitting.

[0027] The welding robot is controlled to weld the tube sheet workpiece according to the desired welding trajectory of each tube hole.

[0028] Fifthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the following method:

[0029] Acquire global information collected from the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes.

[0030] Based on global information, the location information of each pipe hole is determined;

[0031] For each borehole, the information acquisition pose for that borehole is determined based on its location information.

[0032] Based on the pipe hole information acquired under the information acquisition pose, the desired welding trajectory of the pipe hole is obtained by fitting.

[0033] The welding robot is controlled to weld the tube sheet workpiece according to the desired welding trajectory of each tube hole.

[0034] Sixthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements the steps of the following method:

[0035] Acquire global information collected from the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes.

[0036] Based on global information, the location information of each pipe hole is determined;

[0037] For each borehole, the information acquisition pose for that borehole is determined based on its location information.

[0038] Based on the pipe hole information acquired under the information acquisition pose, the desired welding trajectory of the pipe hole is obtained by fitting.

[0039] The welding robot is controlled to weld the tube sheet workpiece according to the desired welding trajectory of each tube hole.

[0040] The aforementioned tube sheet workpiece welding method, apparatus, system, computer equipment, computer-readable storage medium, and computer program product first determine the tube hole positions based on the global information of the tube sheet workpiece, achieving coarse positioning of each tube hole in the tube sheet workpiece. Then, based on the tube hole positions obtained from the global positioning, the information acquisition pose for each tube hole is determined. Next, based on the tube hole information acquired under the corresponding information acquisition pose, the desired welding trajectory of the tube hole is fitted, achieving precise positioning of each tube hole. Finally, according to the desired welding trajectory of each tube hole, the welding robot is controlled to weld the tube sheet workpiece. This is equivalent to determining the information acquisition pose based on the coarse positioning result, and then achieving precise positioning of the tube holes based on the tube hole information acquired under the information acquisition pose. It can complete the welding of the tube sheet workpiece without human intervention, which helps to reduce labor costs. Attached Figure Description

[0041] Figure 1 This is an application environment diagram of the tube sheet workpiece welding method in one embodiment;

[0042] Figure 2 This is a flowchart illustrating a tube sheet welding method in one embodiment;

[0043] Figure 3 This is a system schematic diagram of the tube sheet workpiece welding process in one embodiment;

[0044] Figure 4 This is a schematic diagram illustrating the effect of orifice macro-location recognition in one embodiment;

[0045] Figure 5 This is a schematic diagram illustrating the working pose calculation principle of a 3D camera in another embodiment;

[0046] Figure 6 This is a flowchart illustrating the tube sheet welding method in another embodiment;

[0047] Figure 7 This is a structural block diagram of a tube sheet workpiece welding system in one embodiment;

[0048] Figure 8 This is a structural block diagram of a tube sheet workpiece welding device in one embodiment;

[0049] Figure 9 This is an internal structural diagram of a computer device in one embodiment.

[0050] Explanation of reference numerals in the attached drawings: 1-Global acquisition device; 2-Robotic arm; 3-Local acquisition device; 4-Tube sheet workpiece; 41-Tube sheet end face; 411-Tube hole center; 412-Tube hole end face; 5-Tooling table; 6-Ground rail; 7-Welding machine; 8-C-shaped column; 9-Control cabinet; 10-Welding wire drum; 11-Gun cleaning station. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] In one embodiment, the tube sheet welding method provided in this application can be applied to, for example... Figure 1The application environment shown may include, for example, a terminal 101, a controller 102, and a welding robot 103. Data transmission can be achieved between the terminal 101 and the controller 102, and between the controller 102 and the welding robot 103, via wired or wireless connections. The terminal 101 may be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices may include smartwatches, smart bracelets, head-mounted devices, etc. In one specific embodiment, the terminal 101 may include one or more information acquisition devices for collecting information about the tube sheet workpiece.

[0053] The controller 102 can be a hardware module containing various processing chips and their peripheral circuits, possessing logic operation functions. This processing chip can be a microcontroller, a DSP (Digital Signal Processing) chip, or an FPGA (Field Programmable Gate Array) chip, etc. The welding robot 103 refers to an industrial robot containing a welding arm capable of welding.

[0054] Specifically, during the welding of the tube sheet workpiece, the controller 102 can acquire global information collected from the terminal 101 regarding the tube sheet workpiece. Based on this global information, it determines the position information of each hole on the tube sheet workpiece. Then, for each hole, the controller 102 determines the information acquisition pose based on the hole's position information, and fits the desired welding trajectory of the hole based on the hole information collected by the terminal 101 under this information acquisition pose. After obtaining the desired welding trajectory for each hole, the controller 102 controls the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each hole.

[0055] In one embodiment, such as Figure 2 As shown, a method for welding tube sheet workpieces is provided, which can be applied to... Figure 1 Taking controller 102 as an example, the following steps are included:

[0056] Step S202: Obtain global information collected from the tube sheet workpiece.

[0057] The tube sheet workpiece comprises multiple tube holes. In practical applications, a tube sheet workpiece is obtained by drilling holes slightly larger than the outer diameter of the heat exchange tubes into a circular steel plate. Inserting the heat exchange tubes through the tube holes in the tube sheet workpiece and welding them in place achieves the purpose of fixing the heat exchange tubes. Global information refers to information encompassing the entire tube sheet workpiece. In a specific application, global information includes the three-dimensional point cloud information and two-dimensional image information of the tube sheet end face. The tube sheet end face refers to the end face of the tube sheet workpiece that needs to be welded, such as... Figure 3 Tube sheet end face 41 of the tube sheet workpiece 4.

[0058] Specifically, a global acquisition device can be configured to collect global information about the tube sheet workpiece. The controller can then obtain the global information collected from the tube sheet workpiece from this device. This global acquisition device is a device with information acquisition capabilities, and may include at least one of a camera, radar, or laser. In practical applications, the installation position and orientation of the tube sheet workpiece and the global acquisition device can be adjusted so that the signal transmission direction of the global acquisition device is perpendicular to the tube sheet end face of the workpiece, thereby facilitating the acquisition of global information about the tube sheet workpiece.

[0059] In one specific embodiment, the tube sheet workpiece welding method further includes: acquiring the field of view of a global acquisition device; and adjusting the fixed position of the global acquisition device based on the position of the tube sheet workpiece within the field of view, so that the tube sheet workpiece is included within the field of view of the global acquisition device. In this embodiment, step S202 includes: acquiring global information acquired by the global acquisition device for the tube sheet workpiece.

[0060] In practical applications, the size and bore distribution of the tube sheet workpieces to be welded may vary. Therefore, the controller can adjust the fixed position of the global acquisition device before acquiring global information to ensure that the tube sheet workpiece is included within the field of view of the global acquisition device. Specifically, the controller can acquire the image captured by the global acquisition device at the current position, thereby determining the field of view at the current position. Then, based on the position of the tube sheet workpiece within this field of view, the fixed position of the global acquisition device is adjusted to ensure that the tube sheet workpiece is included within the field of view of the global acquisition device. Next, the controller acquires the global information acquired by the global acquisition device for the tube sheet workpiece.

[0061] In this embodiment, the position of the global information acquisition device is adjusted before acquiring global information so that the tube sheet workpiece is included in the field of view of the global acquisition device. Thus, only one information acquisition is needed to obtain the global information of the tube sheet workpiece, which can adapt to the welding requirements of different batches of tube sheet workpieces and realize flexible automatic welding.

[0062] Step S204: Based on global information, determine the location information of each borehole.

[0063] The positional information of the tube hole refers to information that characterizes its location. This positional information can be, for example, the geometric center or centroid of the tube hole. For instance, if the cross-sectional shape of the tube hole on the end face of the tube sheet is circular, the positional information of the tube hole could refer to the location of its center. Specifically, the controller can locate the tube holes in the tube sheet workpiece based on the global information of the tube sheet workpiece, determining the positional information of each tube hole.

[0064] Taking a scenario where global information includes both 3D point cloud information and 2D image information as an example, the controller can perform pipe hole identification on the 2D image information to determine the pipe hole region in the 2D image. Based on the mapping relationship between the 2D image information and the 3D point cloud information, it can then determine the corresponding 3D point cloud region in the 3D point cloud information. Next, it performs pipe hole identification on the 3D point cloud region to determine the pipe hole's location information. This location information can be understood as the 3D location information of the pipe hole, which can be represented by the 3D coordinates of the pipe hole's center. The specific method for pipe hole identification can be, for example, identification based on at least one of the pipe hole's shape or size. Specifically, an operator can send the pipe hole shape information or size information to the controller via a terminal, allowing the controller to perform pipe hole identification based on this shape or size information.

[0065] In a specific implementation, taking the circular shape of the orifice as an example, the controller can perform circle detection based on 2D image information to identify the orifice region with a circular structure in the 2D image, and further identify the 3D point cloud region that has a mapping relationship with the orifice region. Then, the controller performs 3D circle detection on the 3D point cloud region, fits the circular structure parameters, and thus obtains the position information of the orifice. First, determine the orifice region based on 2D image information, and then further detect the 3D position information, which can improve efficiency while ensuring the accuracy of the position information.

[0066] Step S206: For each pipe hole, determine the information acquisition pose for that pipe hole based on the pipe hole's position information.

[0067] Specifically, after determining the positional information of each hole in the tube sheet workpiece, hole information can be collected for each hole separately. The information collection pose refers to the position and orientation of the information collection device when collecting information from the hole. In this embodiment, the device for collecting information from the hole is called a local collection device. Correspondingly, the information collection pose for the hole refers to the information collection pose of the local collection device for the hole. As mentioned above, if the tube sheet workpiece includes multiple holes, a local collection device can be configured to collect hole information from multiple holes by changing the information collection pose. When there are many holes, multiple local collection devices can be configured to collect hole information to improve information collection efficiency. Furthermore, the local collection device and the global collection device can be integrated into the same terminal or set up separately.

[0068] In a specific embodiment, such as Figure 3 As shown, the global acquisition device 1 and the local acquisition device 3 can be separate devices located at different positions. The global acquisition device 1 is positioned above the C-shaped column 8 to collect global information about the tube sheet workpiece, while the local acquisition device 3 is located at the end of the welding robot arm 2 and can adjust its position to follow the arm 2. In this embodiment, the position of the local acquisition device 3 can be represented by the position information of the robot arm 2.

[0069] Step S208: Based on the pipe hole information acquired under the information acquisition pose, the desired welding trajectory of the pipe hole is fitted.

[0070] The desired welding trajectory of the pipe hole is the trajectory of its edge. Specifically, for each pipe hole, the controller can determine the information acquisition pose for that pipe hole, acquire the pipe hole information acquired under that pose, and then perform edge fitting based on this information to obtain the desired welding trajectory. The specific method for fitting the desired welding trajectory is not unique. For example, the controller can determine multiple position points on the edge of the pipe hole based on the pipe hole information, and then smoothly connect these points to obtain the desired welding trajectory. Alternatively, the controller can determine the desired welding trajectory based on the shape of the pipe hole and multiple position points on its edge, and perform edge fitting based on the shape of the pipe hole.

[0071] Step S210: Control the welding robot to weld the tube sheet workpiece according to the desired welding trajectory of each tube hole.

[0072] Welding robots, in this context, refer to industrial robots that include welding arms and are capable of performing welding. In a specific implementation, such as... Figure 3As shown, the welding robot may include components such as a robotic arm 2, a welding machine 7, a control cabinet 9, a wire hopper 10, and a torch cleaning station 11. These components can be fixed to a C-shaped column 8 to follow the C-shaped column 8 on a ground rail 6. The robotic arm 2 initiates the arc and feeds the wire to achieve welding of the tube sheet workpiece 4. The tube sheet workpiece 4 can be fixed on a tooling table 5. It should be noted that... Figure 3 The mechanical structure and assembly method are not unique and can be adjusted according to actual application requirements. For example, the welding robot can be set to a front-mounted or side-mounted type according to the size of the tube sheet workpiece 4; the C-shaped column can be replaced with a gantry type, the ground rail can be replaced with a top rail, and so on. Furthermore, multiple welding arms can be used to weld a single tube sheet workpiece; or, a single welding arm can be used to weld multiple tube sheet workpieces.

[0073] Specifically, the controller can control the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each tube hole. It should be noted that determining the desired welding trajectory and performing the welding action can be performed alternately; alternatively, the welding robot can be controlled to weld after the desired welding trajectory for each tube hole has been determined. For example, for each tube hole, the controller can first fit the desired welding trajectory for that tube hole, then control the welding robot to weld that tube hole, and then fit the desired welding trajectory for the next tube hole, and so on, until all tube holes in the tube sheet workpiece are welded. It can be understood that completing the welding of all tube holes in the tube sheet workpiece is equivalent to completing the welding of the entire tube sheet workpiece.

[0074] In one embodiment, step S210 includes: obtaining the welding sequence of each pipe hole; and controlling the welding robot to weld each pipe hole sequentially according to the welding sequence based on the expected welding trajectory of each pipe hole.

[0075] The welding sequence of the pipe holes can be from the outside in, clockwise, counterclockwise, radially, or circumferentially, etc. Specifically, the controller can acquire the welding sequence of each pipe hole, and then, based on the desired welding trajectory for each pipe hole, control the welding robot to weld each pipe hole sequentially according to the welding sequence. Welding according to the welding sequence allows the operator to rationally set the welding sequence based on the distribution of pipe holes in the tube sheet workpiece, thereby shortening the movement path of the welding robot and improving welding efficiency.

[0076] The aforementioned tube sheet welding method first determines the tube hole positions based on the global information of the tube sheet workpiece, achieving coarse positioning of each tube hole. Then, based on the tube hole positions obtained from the global positioning, the information acquisition pose for each tube hole is determined. Next, based on the tube hole information acquired under the corresponding information acquisition pose, the desired welding trajectory for the tube hole is fitted, achieving precise positioning of each tube hole. Finally, according to the desired welding trajectory for each tube hole, the welding robot is controlled to weld the tube sheet workpiece. This method is equivalent to determining the information acquisition pose based on the coarse positioning result, and then achieving precise positioning of the tube holes based on the tube hole information acquired under that information acquisition pose. It can complete the welding of the tube sheet workpiece without human intervention, which helps to reduce labor costs. Furthermore, determining the tube hole positions based on global information first, and then fitting the desired welding trajectory for each tube hole separately, is equivalent to performing global coarse positioning first and then precise positioning for local areas. This ensures the accuracy of positioning, thereby ensuring the accuracy of the fitted desired welding trajectory and improving the welding quality of the tube sheet workpiece.

[0077] In one embodiment, global information is acquired by a global acquisition device. In this embodiment, step S204 includes: acquiring a first extrinsic parameter of the global acquisition device; and determining the position information of each bore in the reference coordinate system based on the global information and the first extrinsic parameter.

[0078] The specific limitations regarding the global acquisition device are detailed above and will not be repeated here. The first extrinsic parameter of the global acquisition device characterizes the transformation relationship between the first coordinate system and the reference coordinate system corresponding to the global acquisition device. Taking the case where the global acquisition device is the first camera as an example, the first coordinate system can refer to the camera coordinate system of the first camera. The camera coordinate system is a three-dimensional Cartesian coordinate system established with the camera's focal center as the origin and the optical axis as the Z-axis. The reference coordinate system refers to the coordinate system used as a reference. This reference coordinate system can, for example, refer to the robot's base coordinate system or the world coordinate system.

[0079] Specifically, the controller can acquire the first extrinsic parameter of the global acquisition device, and then identify the pipe holes based on the global information to determine the position information of each pipe hole in the first coordinate system. Finally, the controller uses the first extrinsic parameter to transform the position information in the first coordinate system to obtain the position information of each pipe hole in the reference coordinate system.

[0080] In this embodiment, a reference coordinate system is set, which makes it easy to convert position information from multiple coordinate systems to the same coordinate system, thereby ensuring the accuracy of pipe hole positioning and improving the welding effect.

[0081] In one embodiment, the global information includes three-dimensional point cloud information and two-dimensional image information. In this embodiment, determining the position information of each hole in the reference coordinate system based on the global information and the first extrinsic parameter includes: determining the hole region in the tube sheet workpiece based on the two-dimensional image information; identifying the hole by performing hole identification on the three-dimensional point cloud information of the hole region to determine the first position information of each hole in the first coordinate system; and performing coordinate transformation on each first position information based on the first extrinsic parameter to obtain the position information of each hole in the reference coordinate system.

[0082] Specifically, the controller can perform pipe hole identification on the two-dimensional image information to determine the pipe hole regions in the two-dimensional image, and determine the three-dimensional point cloud information of the pipe hole regions based on the mapping relationship between the two-dimensional image information and the three-dimensional point cloud information. Then, it performs pipe hole identification on the three-dimensional point cloud information of the pipe hole regions to determine the first position information of each pipe hole in a first coordinate system. Finally, the controller performs coordinate transformation on each first position information based on a first extrinsic parameter to obtain the position information of each pipe hole in a reference coordinate system.

[0083] In the above embodiments, the orifice region is first determined based on two-dimensional image information, and then the three-dimensional position information of the orifice region is further detected, which can improve efficiency while ensuring the accuracy of position information.

[0084] In one embodiment, determining the information acquisition pose for the pipe hole based on the pipe hole location information includes: determining the coordinate system information corresponding to the local acquisition device in the reference coordinate system based on the pipe hole location information; and determining the working pose that matches the coordinate system information as the information acquisition pose of the local acquisition device for the pipe hole.

[0085] The coordinate system information can be used to characterize the coordinate system, including the position information of the origin and the direction information of the coordinate axes. The coordinate system information corresponding to the local acquisition device in the reference coordinate system includes the position of the origin and the direction of the coordinate axes of the local acquisition device's coordinate system in the reference coordinate system. Taking the local acquisition device as a second camera as an example, the coordinate system corresponding to the local acquisition device can refer to the camera coordinate system of the second camera, denoted as the second coordinate system; the coordinate system information corresponding to the local acquisition device can include the position of the origin and the direction of the coordinate axes of the second coordinate system. Specifically, for each pipe hole, the controller can determine the coordinate system information corresponding to the local acquisition device in the reference coordinate system based on the position information of the pipe hole, and determine the working pose matching this coordinate system information as the information acquisition pose of the local acquisition device for the pipe hole.

[0086] In this embodiment, the information acquisition pose in the reference coordinate system is determined based on the position information of the pipe hole in the reference coordinate system. This is equivalent to using an absolute positioning method to determine the information acquisition pose for the pipe hole, which can improve the accuracy of the information acquisition pose, thereby ensuring the smooth progress of pipe hole information acquisition and improving the accuracy of the expected welding trajectory obtained based on pipe hole information fitting.

[0087] In one embodiment, determining the coordinate system information corresponding to the local acquisition device in the reference coordinate system based on the position information of the pipe hole includes: determining the center of the pipe hole represented by the position information of the pipe hole, and the end face of the pipe hole to be welded; determining the feature points of the pipe hole from the center of the pipe hole and the end face of the pipe hole based on the device type of the local acquisition device, and determining the feature direction pointing to the feature points; determining the recommended viewing distance of the local acquisition device as the distance between the origin of the coordinate system corresponding to the local acquisition device in the reference coordinate system and the feature points; and determining the feature direction as the direction of the first coordinate axis corresponding to the local acquisition device in the reference coordinate system.

[0088] The direction of the first coordinate axis coincides with the direction of the detection signal emitted by the local acquisition device. The direction of the detection signal emitted by the local acquisition device can refer to the optical axis of a 3D camera or a line structured light acquisition device; it can also refer to the electromagnetic wave transmission direction of radar.

[0089] Specifically, by obtaining the position information of each bore in the reference coordinate system, it is equivalent to completing the macro-positioning and recognition of the bores in the tube sheet workpiece, which can determine the approximate position and shape of the bores in the tube sheet workpiece. Based on this, the controller can determine the bore center represented by the bore position information, as well as the end face of the bore to be welded. Figure 4 As shown, taking the case of a circular tube hole as an example, the controller can determine the tube hole center 411 and the tube hole end face 412 to be welded for each tube hole based on the position information of each tube hole.

[0090] Furthermore, the device type can be used to characterize the principle by which the local acquisition device acquires image information. This device type could be, for example, a 3D camera, a line structured light acquisition device, or a point structured light acquisition device, etc. It is understandable that different information acquisition principles will result in different information acquisition poses for the same pipe hole. For example, a 3D camera can acquire pipe hole information by taking a top-down view; a line structured light acquisition device, however, requires information acquisition from multiple positions in the lateral direction to fit the desired welding trajectory. Based on this, the controller can determine the feature points of the pipe hole from the center and end face of the pipe hole, and determine the feature direction pointing to the feature points, based on the device type of the local acquisition device. Then, the controller determines the recommended viewing distance of the local acquisition device as the distance between the origin of the local acquisition device and the feature points in the reference coordinate system. This recommended viewing distance can be found in the recommended operating parameters of the local acquisition device; for example, the recommended viewing distance could be the optimal viewing distance value for a 3D camera. Finally, the controller determines the feature direction as the direction of the first coordinate axis corresponding to the local acquisition device in the reference coordinate system. Based on this, the directions of the second and third coordinate axes can be further determined in a coordinate plane perpendicular to the direction of the first coordinate axis. For ease of understanding, this application refers to the direction of the first coordinate axis as the Z direction, and the directions of the second and third coordinate axes as the X direction and Y direction, respectively.

[0091] In the above embodiments, by combining the position information of the bore in the reference coordinate system and the device type of the local acquisition device, the coordinate system information corresponding to the local acquisition device in the reference coordinate system can be determined, which can improve the accuracy of the coordinate system information and thus improve the accuracy of the information acquisition pose represented by the coordinate system information.

[0092] In one specific embodiment, based on the device type of the local acquisition device, the feature points of the pipe hole are determined from the center of the pipe hole and the end face of the pipe hole, and the feature direction pointing to the feature points is determined, including: when the local acquisition device is a three-dimensional camera, the center of the pipe hole is determined as the feature point of the pipe hole; and the direction that is perpendicular to the end face of the pipe hole and points to the feature point is determined as the feature direction.

[0093] Specifically, such as Figure 5 As shown, when the local acquisition device is a 3D camera, the center O of the pipe hole is... i The feature point of the pipe hole is determined by pointing perpendicular to the end face of the pipe hole and pointing towards feature point O. i The direction of this direction is determined as the characteristic direction. That is, the origin O of the second coordinate system... C2 Center O of the tube hole i The line connecting the two holes is perpendicular to the end face of hole i, and the origin O is perpendicular to the line connecting the two holes. C2 Center O of the tube hole i The distance is h, where h is the recommended viewing distance for the 3D camera.

[0094] In another specific embodiment, based on the device type of the local acquisition device, the feature points of the tube hole are determined from the center of the tube hole and the end face of the tube hole, and the feature direction pointing to the feature points is determined, including: when the local acquisition device is a line structured light acquisition device, multiple edge points on the end face of the tube hole are determined as multiple feature points; and the feature direction corresponding to each feature point is determined respectively.

[0095] In this configuration, the feature line corresponding to the feature direction of any feature point is perpendicular to the tangent line of the tube edge passing through that feature point, and the angle between the line connecting the feature point and the center of the tube and the feature line is an acute angle. Specifically, when the local acquisition device is a line structured light acquisition device, multiple shooting positions need to be set for each tube. Based on this, the controller can determine multiple edge points on the end face of the tube as multiple feature points, and determine the feature direction corresponding to each feature point, thereby obtaining multiple information acquisition poses.

[0096] For example, such as Figure 5 As shown, when the local acquisition device is a line structured light acquisition device, any edge point P on the end face of the tube hole i can be determined as a feature point, and point P and the center O of the tube hole can be connected. i The line connecting the points is denoted as l1. A tangent l2 is drawn from point P to the edge C of the pipe. Based on l1 and l2, a feature line l3 is determined, and then the feature direction z corresponding to the feature point P is determined on this feature line l3. The origin O is the coordinate system. C2 Center O of the tube hole i The distance is h, which is the recommended line-of-sight distance for the structured light acquisition device; l3 passes through point P and is perpendicular to l2, and the angle between l3 and l1 is α, where α < 90°. For example, α can be 45° or 50°, etc., which can be determined according to the actual imaging effect of the structured light acquisition device to ensure that the laser stripes on the weld can be fully exposed in the field of view and are not blocked by adjacent pipe holes.

[0097] In the above embodiments, different methods for determining feature points and feature directions are provided for two different types of local acquisition devices, which can match the characteristics of the local acquisition devices and help improve the flexibility of the tube sheet workpiece welding method.

[0098] In a specific application, the tube sheet welding method further includes: determining the end face shape of each tube hole based on global information. In this embodiment, multiple edge points on the end face of the tube hole are determined as multiple feature points, including: determining the number of feature points matching the end face shape; and determining multiple feature points evenly distributed on the end face of the tube hole based on the number of feature points.

[0099] The end face shape can be circular, elliptical, or square, etc. The number of feature points matching the end face shape refers to the number of feature points required to fit that end face shape. For example, a circle requires at least three feature points, and an ellipse requires at least four. Specifically, the controller can determine the end face shape of each orifice based on global information. Then, it determines the number of feature points matching the end face shape and, based on this number, determines multiple evenly distributed feature points on the orifice end face. For example, when the orifice end face shape is circular, four evenly distributed feature points can be determined on the orifice end face, thereby determining four evenly distributed information acquisition positions.

[0100] In this embodiment, based on the number of feature points matching the end face shape, multiple feature points are uniformly distributed on the end face of the pipe hole. This ensures that the desired welding trajectory is fitted while improving the efficiency of pipe hole information acquisition and trajectory fitting.

[0101] In one embodiment, the tube sheet workpiece welding method further includes: determining the end face shape of each tube hole based on global information. In this embodiment, step S208 includes: acquiring tube hole point cloud information acquired by the local acquisition device in the information acquisition pose; and performing position fitting on the tube hole point cloud information based on a fitting algorithm that matches the end face shape of the tube hole to obtain the desired welding trajectory of the tube hole.

[0102] Specifically, the controller can determine the end face shape of each tube hole based on the global information of the tube sheet workpiece. Then, for each tube hole, the controller can acquire the tube hole point cloud information collected by the local acquisition device under the information acquisition pose, and perform position fitting on the tube hole point cloud information based on a fitting algorithm that matches the end face shape of the tube hole to obtain the desired welding trajectory of the tube hole. Taking the case where the end face shape of the tube hole is circular as an example, the controller can calculate the desired welding trajectory of the tube hole based on a spatial circle fitting algorithm.

[0103] In this embodiment, a fitting algorithm based on matching the end face shape of the tube hole is used to fit the position of the tube hole point cloud information to obtain the desired welding trajectory of the tube hole. This algorithm can match the trajectory fitting requirements of tube holes of different shapes and can complete the automatic welding of tube sheet workpieces when the tube hole workpiece includes tube holes of various shapes, which is beneficial to improving the flexibility of the tube sheet workpiece welding method.

[0104] In one embodiment, the reference coordinate system is the robot base coordinate system of the welding robot, and the local acquisition device is fixed to the tool flange at the end of the welding robot. In this embodiment, the working pose matching the coordinate system information is determined as the information acquisition pose of the local acquisition device for the pipe hole, including: acquiring a second extrinsic parameter of the local acquisition device; determining a third extrinsic parameter of the local acquisition device based on the coordinate system information; combining the second and third extrinsic parameters to determine the robot coordinate transformation relationship between the robot tool coordinate system and the robot base coordinate system; and determining the working pose represented by the robot coordinate transformation relationship as the information acquisition pose of the local acquisition device for the pipe hole.

[0105] The robot base coordinate system is a Cartesian coordinate system based on the robot mounting base, used to describe the robot's body motion. The robot tool coordinate system is a coordinate system fixed on the flange, remaining constant relative to the tool flange center of the robotic arm. The origin (TCP) of the tool coordinate system, also known as the robot arm's motion center point, is the working point of the tool mounted on the robot. The second extrinsic parameter characterizes the transformation relationship between the second coordinate system corresponding to the local acquisition device and the robot tool coordinate system of the welding robot; the third extrinsic parameter characterizes the transformation relationship between the second coordinate system and the robot base coordinate system; the robot coordinate transformation relationship characterizes the transformation relationship between the robot tool coordinate system and the robot base coordinate system. For the same welding system, the second extrinsic parameter, the third extrinsic parameter, and the robot coordinate transformation relationship satisfy the following condition: second extrinsic parameter * robot coordinate transformation relationship = third extrinsic parameter. Therefore, any two of the above three transformation relationships can be used to determine the third transformation relationship.

[0106] Specifically, the second extrinsic parameter is determined by the mounting position of the local acquisition device on the tool flange at the end of the robotic arm. That is, after the local acquisition device is installed on the tool flange, as long as the relative position between the local acquisition device and the tool flange remains unchanged, the second extrinsic parameter of the local acquisition device will not change. Based on this, the controller can obtain the second extrinsic parameter of the local acquisition device based on the relative position between the local acquisition device and the tool flange. On the other hand, the controller can determine the third extrinsic parameter of the local acquisition device based on the coordinate system information of the second coordinate system under the reference coordinate system. Then, the controller combines the second and third extrinsic parameters to determine the robot coordinate transformation relationship between the robot tool coordinate system and the robot base coordinate system. It can be understood that this robot coordinate transformation relationship can characterize the pose of the robotic arm end effector relative to the robot mounting base, i.e., the pose of the robotic arm end effector. Since the local acquisition device remains in a constant relative position with the end flange of the robotic arm after installation and completes information acquisition during the movement of the robotic arm, the working pose characterized by the robot coordinate transformation relationship can also characterize the information acquisition pose of the local acquisition device. Based on this, the controller can determine the working pose represented by the robot's coordinate transformation relationship as the information acquisition pose of the local acquisition device for the pipe hole.

[0107] In the above embodiments, the local acquisition device is fixed to the tool flange at the end of the welding robot. The information acquisition posture of the local acquisition device is characterized by the robot end pose. The information acquisition posture can be controlled by controlling the welding robot, thereby realizing weld seam tracking during the welding process, which is beneficial to further improve the welding effect.

[0108] In one embodiment, such as Figure 6 As shown, a method for welding tube sheet workpieces is provided, the method comprising the following steps:

[0109] Step S601: Obtain the field of view of the global acquisition device;

[0110] Step S602: Based on the position of the tube sheet workpiece in the field of view, adjust the fixed position of the global acquisition device so that the tube sheet workpiece is included in the field of view of the global acquisition device.

[0111] Step S603: Obtain the global information collected by the global acquisition device for the tube sheet workpiece, and the first external parameter of the global acquisition device;

[0112] The global information includes three-dimensional point cloud information and two-dimensional image information; the first external parameter is used to characterize the transformation relationship between the first coordinate system corresponding to the global acquisition device and the reference coordinate system; the reference coordinate system is the robot base coordinate system of the welding robot;

[0113] Step S604: Based on the two-dimensional image information, determine the tube hole area in the tube sheet workpiece;

[0114] Step S605: Identify the pipe holes in the three-dimensional point cloud information of the pipe hole area, and determine the first position information of each pipe hole in the first coordinate system and the end face shape of each pipe hole.

[0115] Step S606: Based on the first external parameter, perform coordinate transformation on each first position information to obtain the position information of each pipe hole in the reference coordinate system;

[0116] Step S607: For each pipe hole, determine the center of the pipe hole represented by the position information of the pipe hole, and the end face of the pipe hole to be welded.

[0117] Step S608: When the local acquisition device is a 3D camera, the center of the pipe hole is determined as the feature point of the pipe hole; the direction perpendicular to the end face of the pipe hole and pointing to the feature point is determined as the feature direction.

[0118] Step S609: When the local acquisition device is a line structured light acquisition device, determine the number of feature points that match the end face shape of the tube hole.

[0119] Step S610: Based on the number of feature points, determine multiple feature points that are evenly distributed on the end face of the pipe hole, and determine the feature direction corresponding to each feature point respectively.

[0120] Among them, the feature line corresponding to the feature direction of any feature point is perpendicular to the tangent of the pipe edge passing through the feature point; the angle between the line connecting the feature point and the center of the pipe and the feature line is an acute angle.

[0121] Step S611: The recommended line-of-sight distance of the local acquisition device is determined as the distance between the origin of the local acquisition device and the feature point in the reference coordinate system.

[0122] Step S612: Determine the feature direction as the direction of the first coordinate axis corresponding to the local acquisition device in the reference coordinate system;

[0123] Among them, the coordinate line containing the direction of the first coordinate axis coincides with the optical axis of the local acquisition device;

[0124] Step S613: Obtain the second extrinsic parameter of the local acquisition device, and determine the third extrinsic parameter of the local acquisition device based on the coordinate system information;

[0125] The local acquisition device is fixed to the tool flange at the end of the welding robot; the second external parameter is used to characterize the transformation relationship between the second coordinate system corresponding to the local acquisition device and the robot tool coordinate system of the welding robot; the third external parameter is used to characterize the transformation relationship between the second coordinate system and the robot base coordinate system.

[0126] Step S614: Combine the second and third external parameters to determine the robot coordinate transformation relationship between the robot tool coordinate system and the robot base coordinate system;

[0127] Step S615: Determine the working pose represented by the robot coordinate transformation relationship as the information acquisition pose of the local acquisition device for the pipe hole.

[0128] Step S616: Obtain the pipe hole information collected by the local acquisition device in the information acquisition pose.

[0129] Step S617: Based on the fitting algorithm that matches the end face shape of the pipe hole, position fitting is performed on the pipe hole information to obtain the expected welding trajectory of the pipe hole.

[0130] Step S618: Based on the desired welding trajectory of each pipe hole, control the welding robot to weld each pipe hole sequentially according to the set welding sequence.

[0131] In the above embodiments, the positions of the tube holes are first determined based on the global information of the tube sheet workpiece, achieving coarse positioning of each tube hole in the tube sheet workpiece. Then, based on the tube hole positions obtained from the global positioning, the information acquisition pose for each tube hole is determined. Next, based on the tube hole information acquired under the corresponding information acquisition pose, the desired welding trajectory of the tube hole is fitted, achieving precise positioning of each tube hole. Finally, according to the desired welding trajectory of each tube hole, the welding robot is controlled to weld the tube sheet workpiece. This is equivalent to determining the information acquisition pose based on the coarse positioning result, and then achieving precise positioning of the tube holes based on the tube hole information acquired under that information acquisition pose. This allows welding of the tube sheet workpiece to be completed without human intervention, which helps reduce labor costs. Furthermore, determining the tube hole positions based on global information first, and then fitting the desired welding trajectory for each tube hole separately, is equivalent to performing global coarse positioning followed by local precise positioning. This ensures the accuracy of the positioning, thereby ensuring the accuracy of the fitted desired welding trajectory and improving the welding quality of the tube sheet workpiece.

[0132] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0133] Based on the same inventive concept, this application also provides a tube sheet workpiece welding system for implementing the tube sheet workpiece welding method described above. The solution provided by this system is similar to the solution described in the above method; therefore, the specific limitations in one or more tube sheet workpiece welding system embodiments provided below can be found in the limitations of the tube sheet workpiece welding method described above, and will not be repeated here.

[0134] In one embodiment, such as Figure 7 As shown, a tube sheet workpiece welding system is provided, including a controller 701, a global acquisition device 702, a local acquisition device 703, and a welding robot 704 connected to the controller 701. The global acquisition device 702 is used to acquire global information of the tube sheet workpiece to obtain global information of the tube sheet workpiece; the local acquisition device 703 is used to acquire information for each tube hole separately to obtain the tube hole information of each tube hole; the welding robot 704 is used to weld the tube sheet workpiece; and the controller 701 is used to implement the above-described tube sheet workpiece welding method.

[0135] The tube sheet workpiece includes multiple tube holes. A global acquisition device is a device with information acquisition capabilities, specifically including at least one of a camera, radar, or laser. A local acquisition device is a device with information acquisition capabilities, including a camera, a line structured light acquisition device, a point structured light acquisition device, etc. The recommended line-of-sight distance for the global acquisition device is greater than that for the local acquisition device, and the positioning accuracy of the local acquisition device is greater than that of the global acquisition device.

[0136] Specifically, such as Figure 3 As shown, the global acquisition device 1 can be fixed directly above the tube sheet workpiece 4 so that global information acquisition of the entire tube sheet end face can be completed in one acquisition. The local acquisition device 3 can be fixed to the end flange of the robotic arm 2 of the welding robot and can follow the movement of the robotic arm 2 to acquire tube hole information for each tube hole in different poses.

[0137] During the welding process of the tube sheet workpiece, the controller 701 acquires global information collected by the global acquisition device 702. Based on this global information, it determines the position information of each hole on the tube sheet workpiece. Then, for each hole, the controller 701 determines the information acquisition pose based on the hole's position information, and fits the desired welding trajectory of the hole based on the hole information collected by the local acquisition device 703 under this information acquisition pose. After obtaining the desired welding trajectory for each hole, the controller 701 controls the welding robot 704 to weld the tube sheet workpiece according to the desired welding trajectory for each hole.

[0138] The aforementioned tube sheet welding system first determines the tube hole positions based on the global information of the tube sheet workpiece, achieving coarse positioning of each tube hole. Then, based on the tube hole positions obtained from the global positioning, it determines the information acquisition pose for each tube hole. Next, based on the tube hole information acquired under the corresponding information acquisition pose, it fits the desired welding trajectory for the tube hole, achieving precise positioning of each tube hole. Finally, it controls the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each tube hole. Essentially, it determines the information acquisition pose based on the coarse positioning result, and then achieves precise positioning of the tube holes based on the tube hole information acquired under that pose. This allows for welding of the tube sheet workpiece without human intervention, reducing labor costs. Furthermore, determining the tube hole positions based on global information first, and then fitting the desired welding trajectory for each tube hole separately, is equivalent to performing global coarse positioning followed by local precise positioning. This ensures the accuracy of the positioning, thereby ensuring the accuracy of the fitted desired welding trajectory and improving the welding quality of the tube sheet workpiece.

[0139] In a specific embodiment, taking the case where the global acquisition device is the first camera, the local acquisition device is the second camera, and the end face of the tube hole is circular, the tube sheet workpiece welding method and system are described in detail.

[0140] Specifically, a macro-positioning vision system can be used to identify the number of pipe holes and measure their outer diameter, thus coarsely positioning the center of each pipe hole. Then, based on the coordinates of the pipe hole centers obtained from the coarse positioning by the macro-positioning vision system, the pose of the fine positioning vision system is calculated. The fine positioning vision system is then used to precisely measure the characteristic positions of the pipe holes, obtaining pipe hole information. Finally, based on this information, a precise trajectory for the pipe holes is fitted. This precise trajectory is the desired welding trajectory for the pipe holes.

[0141] The macro-positioning vision system may include a first camera and a controller. For example... Figure 3 As shown, the global acquisition device 1 (i.e., the first camera) can be installed directly above the tube sheet workpiece 4, with its relative position to the robot's base coordinate system fixed. The camera's field of view must completely cover all the holes to be welded on the tube sheet workpiece 4, allowing for the acquisition of 3D point cloud data and 2D image data of the tube sheet end face in a single photograph. The controller can be deployed in the control cabinet 9 to acquire the 3D point cloud data and 2D image data, and to combine the 3D point cloud data and 2D image data to identify and fit the circular trajectory of the outer edge of the tube hole on the tube sheet. For example... Figure 5 As shown, based on the fitted circular trajectory of the tube hole, the controller can calculate the position of the circle's center in the camera coordinate system (i.e., the first coordinate system) of the first camera. Cam O i =(x c_i , y c_i, z c_i ) and the outer diameter D of the pipe hole i , where i = 1, 2, 3…n, and n is the total number of tube holes on the tube sheet.

[0142] In a specific application, the process of identifying pipe holes includes the following steps:

[0143] 1. Circle detection in 2D images:

[0144] A continuous set of edge segments is generated using a parameter-free edge detection algorithm. Considering the influence of various noises during welding, non-edge pixels can be eliminated through non-maximum suppression, hysteresis thresholding, and erosion operations. This effectively avoids the impact of welding noise such as arc light and smoke on image quality. Then, the least-squares fitted linear equations of these edge segment sets are calculated, along with the fitting error. If the fitting error is less than a given threshold, the edge segment is considered a straight line segment; otherwise, it is divided into two sub-edge segments. Each sub-edge segment is recursively fitted and divided until all edge segments are fitted as straight lines or reach a pre-set minimum length. Based on the angular change rate of adjacent straight lines, straight lines with an angular change rate less than a threshold are converted into arc segments. An adaptive polygon approximation algorithm is then used to detect candidate circles and near-circular ellipses. Based on the Helmholtz principle, a reverse verification step is used to verify the candidate points, eliminating detected erroneous results.

[0145] After performing circle detection on the image, circles within a certain radius are filtered out and those within a certain range are merged. Once the 2D detected circles are obtained, they are converted into 2D candidate regions for detection in the point cloud based on their center coordinates and radius. Optionally, image object detection methods or image segmentation methods can also be used to perform circle detection on the 2D image.

[0146] 2. Convert the 2D region into a 3D point cloud region:

[0147] Based on the mapping relationship between a 2D image and a 3D point cloud, the point cloud information corresponding to the orifice region in the 2D image can be determined. Then, by identifying the position of the point cloud information, the 3D coordinates of the corresponding point in the camera coordinate system can be obtained. For example, the first position can be identified as the upper left corner of the corresponding 2D region, and the second position as the lower left corner. Converting the coordinates of the upper left and lower left corners into point clouds yields the coordinates of the corresponding point in the camera coordinate system, thus obtaining the 3D point cloud region corresponding to the orifice region. Assuming the coordinates of a pixel in the depth map are (u, v) and the corresponding depth value is d, then the coordinates of this pixel in 3D space are (x, y, z). Assuming the camera's intrinsic parameter matrix is ​​K, it is represented as:

[0148]

[0149] Among them, f x and f y It is the focal length along the x and y axes, s is the skew coefficient, and c is the focal length along the x and y axes. x and c y These are the principal point coordinates. The formula for transforming an image point (u,v) to a point cloud (x,y,z) is expressed as:

[0150]

[0151] To prevent abnormal areas from appearing in individual pixels during the conversion process due to sparks or arcs, median filtering can be applied to the depth map, using the median depth information of each pixel as the depth information to be converted. For a two-dimensional region, based on the upper left corner coordinates (u... l ,v l ) and the coordinates of the lower right corner (u r ,v r ) to obtain the corresponding three-dimensional coordinates (x l ,y l ,z l ) and (x r ,y r ,z r Thus, the three-dimensional region satisfies the following relationship:

[0152]

[0153] Here, μ is used to expand the conversion region from the redundant region, μ is the set high-probability region expansion coefficient, and x min and x max The x-axis coordinates and y-axis coordinates represent the minimum and maximum values ​​of the transformed 3D region. min and y max The z-coordinates represent the minimum and maximum y-coordinates of the transformed 3D region. min and z max This represents the minimum and maximum z-axis coordinates of the transformed 3D region.

[0154] 3. Preprocess the acquired 3D point cloud data:

[0155] After passing through filtering the point cloud, to perform point cloud circle detection, considering the potential for outliers and missing values ​​due to arc light, metal spatter, etc., in welding scenarios, the maximum connected region of the point cloud was first calculated to remove outliers. Then, moving least squares was used to complete the point cloud. The processed point cloud eliminated the influence of outliers such as arc light and filled in small-scale missing values ​​caused by spatter.

[0156] 4. Perform point cloud circle detection on the preprocessed point cloud data:

[0157] A random consistency sampling algorithm can be used for point cloud circle detection. After obtaining the curve equation of the fitted circle, all points within the equation range are classified as points on the circle, and the remaining points are used as input points for the next stage of point cloud detection. Specifically, after obtaining the results of 2D circle detection, high-probability regions marked as point cloud circle detection are prioritized, and 3D circle detection algorithms are applied to the point clouds within these regions. Then, the same 3D circle detection algorithm is applied to the remaining point cloud regions to improve the efficiency and accuracy of circle detection. In this way, the information from 2D circle detection can be fully utilized to pre-determine regions with a high probability of circular structures, narrowing the detection range and saving computational resources. It also avoids the shortcomings of 2D circle detection; for cases where 2D circle detection cannot identify or misses detection, the 3D circle detection algorithm can still discover circular structures from the remaining point cloud. Optionally, point cloud segmentation methods can also be used for 3D point cloud circle detection.

[0158] 5. Candidate circle filtering:

[0159] The results obtained from the point cloud circle detection algorithm can be filtered to include circles within a specified radius range, followed by a merging operation on circles whose centers fall within that range. The merged circle result is the final circle result, as shown below. Figure 4 As shown, the overall image is the input point cloud, and the bolded circular curve is the final detected circular result.

[0160] After obtaining the positions of the tube holes in the first coordinate system, the controller can combine the camera hand-eye matrix (i.e., the first extrinsic parameter of the first camera) to calculate the position information of the centers of all tube holes in the robot base coordinate system. The camera hand-eye calibration process is completed in advance before the measurement begins, and the pose matrix H of the camera coordinate system and the robot base coordinate system is calculated by the calibration software. hand-eye Then the position of the center of the tube hole in the robot's base coordinate system can be obtained as follows:

[0161]

[0162] The above completes the coarse positioning of the tube sheet workpiece. Next, we will perform precise positioning for each tube hole.

[0163] The precision positioning vision system may include a second camera and a controller. The second camera is mounted on the tool flange at the robot's end effector. The second camera can be a weld seam tracking sensor (e.g., a line structured light acquisition device) or a small-field-of-view 3D camera. First, the second camera needs to be calibrated offline to determine the position transformation matrix (i.e., the second extrinsic parameter of the second camera) between the camera coordinate system (i.e., the second coordinate system) and the robot tool coordinate system. Then, the controller calculates the coordinate system information of the second camera based on the center position and the outer diameter of the pipe hole identified by the macro-positioning vision system, thereby determining the image-taking pose of the second camera.

[0164] The calculation method for the coordinate system information of the second camera is as follows: Figure 5 As shown. The center of the pipe hole is O. i Let C be the trajectory of the pipe's edge. Let l1 be an auxiliary line passing through the center of the circle and intersecting C at point P. Let l2 be a tangent line on C passing through point P. Let l3 pass through point P and be perpendicular to l2. The angle between l3 and l1 is α. c2 This is the coordinate system for the second camera.

[0165] When the second camera is a small-field-of-view 3D camera, make the second phase coordinate system O c2 The z-axis is perpendicular to the plane containing C and maintains a certain height h, where h is the optimal viewing distance of the camera. These two constraints control the robot's pose, and images of the circular trajectory C are taken from positions that meet these conditions. Only one image is needed to obtain a high-precision point cloud of the pipe hole. Then, a spatial circle fitting algorithm is used to calculate the precise trajectory of the pipe hole in the robot's coordinate system, such as the least squares method, RANSAC method, conjugate gradient method, and Hough transform method.

[0166] When the second camera is a weld seam tracking sensor, at least four imaging positions need to be set for each pipe hole. First, set the first imaging position: for example... Figure 3 As shown, make the second camera coordinate system O c2 The z-axis points in the l3 direction, and the angle α between the z-axis and l1 is typically 45°. This angle can be adjusted according to the actual camera imaging effect to ensure that the laser stripes on the weld are fully exposed in the camera's field of view and are not obstructed by adjacent pipe holes. c2 The distance between the origin and point P is h, and the optimal viewing distance of the camera is taken. After the first shooting position is determined, the remaining shooting positions are uniformly and symmetrically distributed around the trajectory C in 360°. The axis of symmetry is parallel to the normal vector of the plane containing C and passes through O. i The rotation radius of the distributed area is the same as l1. Taking four imaging positions as an example, the remaining three imaging positions are obtained by rotating 90° along the axis of symmetry starting from the first imaging position. After calculating the camera pose at each imaging position, the robot can be automatically moved to the corresponding position by program control and the 2D camera of the weld seam tracking sensor can be activated to take pictures. Through image recognition algorithm, the position of weld seam feature points can be identified from the laser stripes captured by the 2D camera. Finally, the four position points on trajectory C are calculated by hand-eye conversion matrix. The precise center, radius, and spatial expression of trajectory C are obtained by fitting the coordinates of the four position points. It should be noted that although the macro-positioning vision system can identify the position of the hole center in the robot's base coordinate system and use it as the basis for setting the imaging position of the fine positioning camera, the center coordinates obtained by macro-positioning are relatively large because the camera's line of sight is much larger than that of the fine positioning camera. Therefore, it is expected that the welding trajectory is obtained by fitting the measurement results of the fine positioning vision system.

[0167] Finally, based on the desired welding trajectory for each pipe hole, the welding robot is controlled to perform arc welding on each pipe hole sequentially according to the welding sequence. This welding sequence can be determined as needed, for example, welding sequentially from the outer ring to the inner ring or from the inner ring to the outer ring, clockwise or counterclockwise, radially or circumferentially, etc.

[0168] The aforementioned tube sheet welding method and system employs a macro- and fine-vision positioning strategy. The macro-positioning vision system automatically identifies the number of tube holes and the position of their centers, and autonomously calculates the photographic pose of the fine-positioning camera for each tube hole. Under the corresponding photographic pose, it performs fine-positioning of each tube hole, fits the desired welding trajectory, and controls the robot to automatically weld according to the desired trajectory. This method automatically identifies the weld position in the tube holes and performs welding, with a fully automated process requiring no teaching or manual operation. Each workstation can save at least one welding technician or robot operator, thus reducing labor costs. Furthermore, errors in incoming materials, assembly, and positioning are eliminated through visual positioning by using absolute positioning to calculate the weld's position in the robot's base coordinate system, ensuring welding quality. Simultaneously, for different batches of tube sheets, as long as the macro-positioning camera's field of view covers the tube sheet end face, it can fully adapt to changes in the number and size of tube holes without additional settings or teaching operations, exhibiting high flexibility and facilitating easy and convenient changeovers.

[0169] Based on the same inventive concept, this application also provides a tube sheet workpiece welding apparatus for implementing the tube sheet workpiece welding method described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more tube sheet workpiece welding apparatus embodiments provided below can be found in the limitations of the tube sheet workpiece welding method described above, and will not be repeated here.

[0170] In one embodiment, such as Figure 8 As shown, a tube sheet workpiece welding device is provided, including: a global information acquisition module 801, a position information determination module 802, an information acquisition pose determination module 803, a trajectory fitting module 904, and a welding module 805, wherein:

[0171] The global information acquisition module 801 is used to acquire global information collected from the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes.

[0172] The location information determination module 802 is used to determine the location information of each pipe hole based on global information;

[0173] The information acquisition pose determination module 803 is used to determine the information acquisition pose for each pipe hole based on the position information of the pipe hole.

[0174] The trajectory fitting module 804 is used to fit the desired welding trajectory of the pipe hole based on the pipe hole information acquired under the information acquisition pose.

[0175] The welding module 805 is used to control the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each tube hole.

[0176] In one embodiment, the tube sheet workpiece welding apparatus further includes: a position adjustment module, used to acquire the field of view of the global acquisition device; and to adjust the fixed position of the global acquisition device based on the position of the tube sheet workpiece within the field of view, so that the tube sheet workpiece is included within the field of view of the global acquisition device. In this embodiment, the global information acquisition module 801 is specifically used to: acquire the global information acquired by the global acquisition device for the tube sheet workpiece.

[0177] In one embodiment, global information is acquired by a global acquisition device. In this embodiment, the position information determination module 802 includes: a first extrinsic parameter acquisition submodule, used to acquire a first extrinsic parameter of the global acquisition device; and a position information determination submodule, used to determine the position information of each orifice in a reference coordinate system based on the global information and the first extrinsic parameter. The first extrinsic parameter characterizes the transformation relationship between the first coordinate system corresponding to the global acquisition device and the reference coordinate system.

[0178] In one embodiment, the global information includes three-dimensional point cloud information and two-dimensional image information. In this embodiment, the position information determination submodule is specifically used to: determine the tube hole region in the tube sheet workpiece based on the two-dimensional image information; perform tube hole identification on the three-dimensional point cloud information of the tube hole region to determine the first position information of each tube hole in a first coordinate system; and perform coordinate transformation on each first position information based on a first extrinsic parameter to obtain the position information of each tube hole in a reference coordinate system.

[0179] In one embodiment, the information acquisition pose determination module 803 includes: a coordinate system information determination submodule, used to determine the coordinate system information corresponding to the local acquisition device in the reference coordinate system based on the position information of the pipe hole; and an information acquisition pose determination submodule, used to determine the working pose that matches the coordinate system information as the information acquisition pose of the local acquisition device for the pipe hole.

[0180] In one embodiment, the coordinate system information determination submodule includes: a pipe hole determination unit, used to determine the pipe hole center represented by the pipe hole position information and the pipe hole end face to be welded; a feature analysis unit, used to determine the feature points of the pipe hole from the pipe hole center and the pipe hole end face based on the device type of the local acquisition device, and determine the feature direction pointing to the feature points; a coordinate origin determination unit, used to determine the recommended viewing distance of the local acquisition device as the distance between the coordinate origin corresponding to the local acquisition device in the reference coordinate system and the feature points; and a coordinate axis direction determination unit, used to determine the feature direction as the first coordinate axis direction corresponding to the local acquisition device in the reference coordinate system; the first coordinate axis direction coincides with the detection signal transmission direction of the local acquisition device.

[0181] In one embodiment, the feature analysis unit is specifically used to: determine the center of the pipe hole as the feature point of the pipe hole when the local acquisition device is a three-dimensional camera; and determine the direction that is perpendicular to the end face of the pipe hole and points to the feature point as the feature direction.

[0182] In one embodiment, the tube sheet workpiece welding apparatus further includes: an end face shape determination unit, used to determine the end face shape of each tube hole based on global information. In this embodiment, the trajectory fitting module 804 is specifically used to: acquire the tube hole point cloud information acquired by the local acquisition device for the tube hole under the information acquisition pose; and perform position fitting on the tube hole point cloud information based on a fitting algorithm that matches the end face shape of the tube hole to obtain the desired welding trajectory of the tube hole.

[0183] In one embodiment, the feature analysis unit is specifically used to: determine multiple edge points on the end face of the tube hole as multiple feature points when the local acquisition device is a line structured light acquisition device; determine the feature direction corresponding to each feature point; the feature line corresponding to the feature direction of any feature point is perpendicular to the tangent of the edge of the tube hole passing through the feature point, and the angle between the line connecting the feature point and the center of the tube hole and the feature line is an acute angle.

[0184] In one embodiment, the tube sheet workpiece welding apparatus further includes an end face shape determination unit, used to: determine the end face shape of each tube hole based on global information. In this embodiment, the feature analysis unit is specifically used to: determine the number of feature points matching the end face shape; and, based on the number of feature points, determine a plurality of uniformly distributed feature points on the end face of the tube hole.

[0185] In one embodiment, the reference coordinate system is the robot base coordinate system of the welding robot, and the local acquisition device is fixed to the tool flange at the end of the welding robot. In this embodiment, the information acquisition pose determination submodule is specifically used to: obtain the second extrinsic parameter of the local acquisition device; the second extrinsic parameter is used to characterize the transformation relationship between the second coordinate system corresponding to the local acquisition device and the robot tool coordinate system of the welding robot; determine the third extrinsic parameter of the local acquisition device based on the coordinate system information; the third extrinsic parameter is used to characterize the transformation relationship between the second coordinate system and the robot base coordinate system; combine the second extrinsic parameter and the third extrinsic parameter to determine the robot coordinate transformation relationship between the robot tool coordinate system and the robot base coordinate system; and determine the working pose characterized by the robot coordinate transformation relationship as the information acquisition pose of the local acquisition device for the pipe hole.

[0186] In one embodiment, the welding module 805 is specifically used to: obtain the welding sequence of each pipe hole; and control the welding robot to weld each pipe hole in sequence according to the expected welding trajectory of each pipe hole.

[0187] Each module in the aforementioned tube sheet welding device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0188] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a tube sheet workpiece welding method or an object classification method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0189] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0190] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0191] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0192] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0193] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0194] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for welding tube sheet workpieces, characterized in that, The method includes: The global information acquired by the global acquisition device for the tube sheet workpiece is obtained; the tube sheet workpiece includes multiple tube holes. Based on the global information, the position information and end face shape of each of the pipe holes are determined; For each of the aforementioned holes, determine the hole center represented by the hole position information, and the hole end face to be welded; Based on the device type of the local acquisition device, feature points of the tube are determined from the center of the tube and the end face of the tube, and feature directions pointing to the feature points are determined. The number of feature points is determined according to the shape of the end face of the tube, and when there are multiple feature points, each feature point is evenly distributed on the end face of the tube. When the local acquisition device is a line structured light acquisition device, the multiple feature points are evenly and symmetrically distributed 360° along the circumference of the end face of the tube, and the length of the line connecting each feature point to the center of the tube is equal. The feature line containing the feature direction is perpendicular to the tangent line of the tube edge passing through the feature point, and the angle between the line connecting the feature point and the center of the tube and the feature line is an acute angle. Based on the feature points and the feature directions, determine the coordinate system information corresponding to the local acquisition device in the reference coordinate system; The working pose that matches the coordinate system information is determined as the information acquisition pose of the local acquisition device for the pipe hole; when the local acquisition device is a line structured light acquisition device, the information acquisition pose includes multiple poses that correspond one-to-one with the multiple feature points. Based on the pipe hole information acquired by the local acquisition device under the information acquisition pose, the desired welding trajectory of the pipe hole is fitted; the information acquisition pose refers to the position and posture of the local acquisition device when acquiring information about the pipe hole. The welding robot is controlled to weld the tube sheet workpiece according to the desired welding trajectory of each of the tube holes.

2. The method according to claim 1, characterized in that, The step of determining the position information of each of the pipe holes based on the global information includes: Obtain the first extrinsic parameter of the global acquisition device; the first extrinsic parameter is used to characterize the transformation relationship between the first coordinate system and the reference coordinate system corresponding to the global acquisition device; Based on the global information and the first extrinsic parameter, the position information of each of the pipe holes in the reference coordinate system is determined.

3. The method according to claim 2, characterized in that, The global information includes three-dimensional point cloud information and two-dimensional image information; determining the position information of each of the pipe holes in the reference coordinate system based on the global information and the first extrinsic parameter includes: Based on the two-dimensional image information, the tube hole region in the tube sheet workpiece is determined; The three-dimensional point cloud information of the pipe hole region is used to identify the pipe hole, and the first position information of each pipe hole in the first coordinate system is determined. Based on the first extrinsic parameter, coordinate transformation is performed on each of the first position information to obtain the position information of each of the pipe holes in the reference coordinate system.

4. The method according to claim 1, characterized in that, The step of determining the coordinate system information of the local acquisition device in the reference coordinate system based on the feature points and the feature directions includes: The recommended viewing distance of the local acquisition device is determined as the distance between the origin of the local acquisition device in the reference coordinate system and the feature point. The characteristic direction is determined as the first coordinate axis direction corresponding to the local acquisition device in the reference coordinate system; the first coordinate axis direction coincides with the detection signal transmission direction of the local acquisition device.

5. The method according to claim 1, characterized in that, The device type based on the local acquisition device determines the feature points of the pipe hole from the center of the pipe hole and the end face of the pipe hole, and determines the feature direction pointing to the feature points, including: When the local acquisition device is a 3D camera, the center of the pipe hole is determined as the feature point of the pipe hole; The direction perpendicular to the end face of the tube hole and pointing towards the feature point is defined as the feature direction.

6. The method according to claim 5, characterized in that, The step of fitting the desired welding trajectory of the pipe hole based on the pipe hole information acquired by the local acquisition device under the information acquisition pose includes: Acquire the point cloud information of the pipe hole obtained by the local acquisition device under the information acquisition pose; Based on a fitting algorithm that matches the end face shape of the pipe hole, the point cloud information of the pipe hole is fitted with position to obtain the desired welding trajectory of the pipe hole.

7. The method according to claim 1, characterized in that, The reference coordinate system is the robot base coordinate system of the welding robot, and the local acquisition device is fixed to the tool flange at the end of the welding robot; The step of determining the working pose that matches the coordinate system information as the information acquisition pose of the local acquisition device for the pipe hole includes: Obtain the second extrinsic parameter of the local acquisition device; the second extrinsic parameter is used to characterize the transformation relationship between the second coordinate system corresponding to the local acquisition device and the robot tool coordinate system of the welding robot; The third extrinsic parameter of the local acquisition device is determined based on the coordinate system information; the third extrinsic parameter is used to characterize the transformation relationship between the second coordinate system and the robot base coordinate system; By combining the second and third extrinsic parameters, the robot coordinate transformation relationship between the robot tool coordinate system and the robot base coordinate system is determined; The working pose represented by the robot coordinate transformation relationship is determined as the information acquisition pose of the local acquisition device for the pipe hole.

8. The method according to claim 1, characterized in that, The step of controlling the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each of the tube holes includes: Obtain the welding sequence of each of the aforementioned pipe holes; Based on the desired welding trajectory for each of the aforementioned pipe holes, the welding robot is controlled to weld each of the aforementioned pipe holes sequentially according to the welding sequence.

9. A tube sheet workpiece welding system, characterized in that, The system includes a controller, and a global acquisition device, a local acquisition device, and a welding robot connected to the controller; The global acquisition device is used to acquire global information of the tube sheet workpiece to obtain global information of the tube sheet workpiece; the tube sheet workpiece includes multiple tube holes; The local acquisition device is used to acquire information for each of the pipe holes to obtain the pipe hole information for each pipe hole. The welding robot is used to weld the tube sheet workpiece; The controller is used to implement the method as described in any one of claims 1 to 8.

10. A tube sheet workpiece welding device, characterized in that, The device includes: A global information acquisition module is used to acquire global information collected by a global acquisition device for a tube sheet workpiece; the tube sheet workpiece includes multiple tube holes. The position information determination module is used to determine the position information and end face shape of each of the pipe holes based on the global information; The pipe hole determination unit is used to determine, for each of the pipe holes, the center of the pipe hole represented by the position information of the pipe hole, and the end face of the pipe hole to be welded. The feature analysis unit is used to determine feature points of the orifice from the center and end face of the orifice based on the device type of the local acquisition device, and to determine the feature direction pointing to the feature points; wherein, the number of feature points is determined according to the shape of the end face of the orifice, and when there are multiple feature points, each feature point is evenly distributed on the end face of the orifice; when the local acquisition device is a line structured light acquisition device, the multiple feature points are evenly and symmetrically distributed 360° along the circumferential direction of the end face of the orifice, and the length of the line connecting each feature point to the center of the orifice is equal; the feature line containing the feature direction is perpendicular to the tangent of the edge of the orifice passing through the feature point, and the angle between the line connecting the feature point to the center of the orifice and the feature line is an acute angle; The coordinate system information determination submodule is used to determine the coordinate system information of the local acquisition device in the reference coordinate system based on the feature points and the feature directions. The information acquisition pose determination submodule is used to determine the working pose that matches the coordinate system information as the information acquisition pose of the local acquisition device for the pipe hole; when the local acquisition device is a line structured light acquisition device, the information acquisition pose includes multiple poses that correspond one-to-one with the multiple feature points. The trajectory fitting module is used to fit the desired welding trajectory of the pipe hole based on the pipe hole information collected by the local acquisition device under the information acquisition pose. The information acquisition pose refers to the position and orientation of the local acquisition device when it collects information from the pipe hole. The welding module is used to control the welding robot to weld the tube sheet workpiece according to the desired welding trajectory for each of the tube holes.

Images