An underwater robot welding repair device and method based on image recognition

Abstract

This invention provides an underwater robot welding repair device and method based on image recognition. The device includes an underwater robot, an external covering plate device, a dry drainage device, and an internal welding repair device. The external covering plate device includes an external image acquisition module for collecting underwater scene image data, an external robotic arm one for covering the underwater damaged area with repair material, and an external robotic arm two for spot welding the repair material to initially fix it. The internal welding repair device includes a built-in image acquisition module for collecting internal image data of the dry drainage device, a built-in robotic arm for welding fillet welds on the sides of the repair material, and a detection image acquisition module. This device can autonomously detect underwater damage locations of marine equipment and perform welding repairs. It is applicable to both large and small damages and can simultaneously repair multiple similar cracks, significantly improving automation and repair efficiency.

Description

Technical Field

[0001] This invention relates to the field of underwater welding technology, and more specifically, to an underwater robot welding and repair device and method based on image recognition. Background Technology

[0002] Marine equipment is not only crucial for supporting the marine economy, exploring the marine environment, and exploiting marine resources, but also an important engine for promoting overall scientific and technological progress and innovative engineering applications. Due to its heavy workload, harsh operating environment, and susceptibility to unpredictable loads such as wind, waves, and earthquakes, marine equipment is highly susceptible to damage and destruction.

[0003] The repair of marine structures primarily employs underwater welding techniques, requiring experienced underwater welders. Manual underwater welding is inherently dangerous, labor-intensive, demands highly skilled welders, and is limited by water depth. While some existing underwater welding robots are equipped with underwater cameras, they are mainly used for transmitting underwater images to facilitate operator observation of underwater damage. They lack the capability to utilize image recognition methods for high-precision underwater damage detection, thus hindering intelligent planning of welding repair trajectories. This severe lack of automation results in welding quality and efficiency that fail to meet the requirements of deep-sea welding. Existing underwater welding robots employ localized dry welding repair methods, but these can only repair a single crack at a time with a short repair distance, leading to unsatisfactory repair efficiency and difficulty in repairing larger holes. Summary of the Invention

[0004] To overcome the shortcomings and deficiencies of the existing technology, the present invention aims to provide an underwater robot welding repair device and method based on image recognition. This device can autonomously detect the underwater damage location of marine equipment and perform welding repair. It is applicable to both large and small damages and can repair multiple similar cracks simultaneously, greatly improving the degree of automation and repair efficiency, and providing a more automated, intelligent and digital method for welding repair in complex underwater environments.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution: an underwater robot welding repair device based on image recognition, comprising an underwater robot, an external covering plate device, a dry drainage device, and an internal welding repair device; the external covering plate device, the dry drainage device, and the internal welding repair device are all mounted on the underwater robot;

[0006] The external covering plate device includes an external image acquisition module for acquiring underwater scene image data, an external robotic arm one for covering the underwater damaged area with repair plate, and an external robotic arm two for spot welding the repair plate locally to initially fix the repair plate; the external image acquisition module, external robotic arm one and external robotic arm two are respectively set on the outside of the dry drainage device, and the end of the external robotic arm two is provided with an external welding gun for spot welding;

[0007] The internal welding repair device includes one or more built-in image acquisition modules for collecting internal image data of the dry drainage device, a built-in robotic arm for welding fillet welds on the side of the repair plate, and a detection image acquisition module for collecting image data of the fillet welds formed by welding. The built-in image acquisition modules and the built-in robotic arm are respectively installed inside the dry drainage device; the detection image acquisition module is installed at the end of the built-in robotic arm.

[0008] Preferably, the built-in image acquisition module includes four internal cameras, which are respectively fixed at the four corners inside the dry drainage device; the external image acquisition module is a binocular camera.

[0009] The welding repair method of the underwater robot welding repair device based on image recognition mentioned above includes the following steps:

[0010] S1. Collect underwater scene image data based on the external image acquisition module, identify the underwater damaged area and the surface to be repaired where the underwater damaged area is located, use external robotic arm one to cover the underwater damaged area with a repair plate of the same material as the surface to be repaired, and use external robotic arm two to complete local spot welding fixation.

[0011] S2. Using the coordinate system of the external image acquisition module and the external parameters of the built-in robotic arm base coordinate system, the underwater robot is controlled to move, so that the repair board is within the working space of the dry drainage device and the built-in robotic arm coordinate system.

[0012] S3. Control the dry drainage device to sink and adhere to the surface to be repaired, and use high-pressure gas to remove the moisture inside the dry drainage device, creating a dry environment inside the dry drainage device.

[0013] S4. Collect internal image data of the dry drainage device based on the built-in image acquisition module, locate the position of the repair board, and convert the position of the repair board to the base coordinate system of the built-in robotic arm;

[0014] S5. Based on the location of the repair plate, plan the motion trajectory of the built-in robotic arm to complete the welding repair work and form a fillet weld on the side of the repair plate; use the detection image acquisition module to collect image data of the fillet weld formed by welding and check the appearance of the fillet weld.

[0015] Preferably, step S1 includes the following steps:

[0016] S11. Underwater scene image data are acquired using an external image acquisition module, which is a stereo camera. Underwater damage detection is performed on two frames of images from the stereo camera at the same time using a YoloV7 network and a pre-trained target detection model to obtain the underwater damage area and the surface to be repaired where the underwater damage area is located. The position of the underwater damage area relative to the stereo camera coordinate system is determined using the pre-calibrated stereo camera parameters.

[0017] S12. Using the extrinsic parameters of the binocular camera coordinate system and the underwater robot coordinate system, the position of the underwater damage area is transformed to the underwater robot coordinate system, and the movement of the underwater robot is controlled so that the underwater damage area is within the travel range of external robotic arm one and external robotic arm two.

[0018] S13. Using the external parameters of the binocular camera coordinate system and the coordinate systems of external robotic arm 1 and external robotic arm 2, the position of the underwater damaged area is transformed to the coordinate systems of external robotic arm 1 and external robotic arm 2 respectively. The external robotic arm 1 is controlled to cover the underwater damaged area with a repair plate of the same material as the surface to be repaired. The external robotic arm 2 is used to drive the external welding gun to complete the local spot welding to fix the repair plate.

[0019] Preferably, in step S11, the binocular camera parameters refer to the intrinsic parameters of each camera in the binocular camera system and the extrinsic parameters between the binocular cameras; the intrinsic parameters include the intrinsic parameter matrix K and the distortion matrix D, and the extrinsic parameters between the binocular cameras include the rotation matrix R and the translation matrix T, respectively, in the following forms:

[0020]

[0021] D = [k1 k2 k3 p1 p2],

[0022]

[0023] Among them, f x f y represents the focal length of the pixel along the X and Y axes, respectively; (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system; k1, k2, and k3 represent radial distortion, respectively; p1 and p2 represent tangential distortion, respectively; r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 These represent the rotation amounts in the rotation matrix R; t x t y t zThese represent the translation amounts in the X, Y, and Z directions of the translation matrix T, respectively.

[0024] Preferably, in step S12, the extrinsic parameters of the binocular camera coordinate system and the underwater robot coordinate system refer to the rotation matrix and translation matrix between the binocular camera coordinate system and the underwater robot coordinate system; the underwater damage area within the travel range of external robotic arm one and external robotic arm two means that external robotic arm one and external robotic arm two reach the underwater damage area by adjusting the joint positions.

[0025] In step S13, the extrinsic parameters of the binocular camera coordinate system and the coordinate systems of external robotic arm 1 and external robotic arm 2 refer to the rotation and translation matrices between the three coordinate systems: the binocular camera coordinate system, the coordinate system of external robotic arm 1, and the coordinate system of external robotic arm 2. The repair plate covering the surface to be repaired is made by selecting a repair plate of the same material as the surface to be repaired, fixing the repair plate to the end of external robotic arm 1, and controlling the movement of external robotic arm 1 to ensure the repair plate completely covers the underwater damaged area. The partial spot welding fixing of the repair plate refers to controlling the movement of external robotic arm 2 and using an external welding gun fixed to the end of external robotic arm 2 to perform multi-point welding, initially fixing the repair plate to the surface to be repaired.

[0026] Preferably, in step S2, the external parameters of the coordinate system of the external image acquisition module and the base coordinate system of the built-in robotic arm refer to the rotation matrix and translation matrix between the coordinate system of the external image acquisition module and the base coordinate system of the built-in robotic arm; the repair plate being in the working space of the dry drainage device and the coordinate system of the built-in robotic arm means that the repair plate is completely covered inside the dry drainage device, and the built-in robotic arm reaches the underwater damage area by adjusting the joint position.

[0027] Preferably, in step S3, the sinking of the dry drainage device refers to controlling the dry drainage device to move downward as a whole through the drive motor of the dry drainage device, so that the bottom rubber material is completely in contact with the surface to be repaired; creating a dry environment refers to controlling the land gas cylinder to supply high-pressure gas to the inside of the dry drainage device through the gas pipe, so that the moisture inside the dry drainage device is discharged through multiple pre-set one-way valves, thereby creating a dry environment inside the dry drainage device.

[0028] Preferably, step S4, locating the repair board position, means that the built-in image acquisition module consists of four internal cameras, which are fixed at the four corners inside the dry drainage device; the four internal cameras are used to acquire image data inside the dry drainage device to perform target detection on the repair board, thereby locating the repair board position.

[0029] Transforming to the built-in robotic arm base coordinate system means using the pre-calibrated intrinsic parameters of the internal cameras and the extrinsic parameters of the four internal cameras and the built-in robotic arm to transform the position of the repair plate detected by the internal camera of each dry drainage device to the built-in robotic arm base coordinate system; the extrinsic parameters of the four internal cameras and the built-in robotic arm refer to the rotation matrix and translation matrix between the coordinate system of each internal camera and the base coordinate system of the built-in robotic arm.

[0030] Preferably, step S5 includes the following steps:

[0031] S51. Based on the position of the repair plate in the coordinate system of the built-in robotic arm and the target detection results of the repair plate by the built-in image acquisition module, locate all sides of the repair plate as the area to be welded; sort the three-dimensional point set of the repair plate side to form a sequential point set with a start point and an end point.

[0032] S52. Based on the length and shape of the sequence point set on the side of the repair plate, plan the welding motion trajectory of the built-in robotic arm, and move continuously along the sequence point set at the optimal welding speed and angle.

[0033] S53. Use the land-based welding power source to provide appropriate welding voltage, current and duty cycle process parameters, use the land-based wire feeding device to provide wire and wire feeding speed for welding repair operations, control the movement of the built-in robotic arm, and complete the welding repair work.

[0034] S54. After completing the welding repair work, a fillet weld is formed on the side of the repair plate. The built-in robotic arm is controlled to move along the welding trajectory again. The detection image acquisition module fixed on the built-in robotic arm is used to inspect the fillet weld and determine whether the appearance of the weld meets the repair requirements.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] 1. This invention proposes an integrated underwater structure damage repair solution for marine equipment using underwater cladding plate spot welding and dry welding. The advantage of this solution lies in its use of a multi-robotic arm collaborative welding method. External robotic arms one and two spot weld the repair plate to the surface to be repaired, while the internal robotic arm only needs to perform welding repair work on the repair plate and the connection points with the surface to be repaired. Compared to manual wet welding repair methods, the internal environment of the dry drainage device in this invention is no different from that of land-based welding, increasing repair efficiency by more than 85% and significantly improving safety. Compared to existing robotic local dry welding repair methods, which can only repair a single crack at a time with a short repair stroke, this invention can simultaneously repair multiple similar cracks and also repair larger holes, offering comprehensive functionality and increasing repair efficiency by more than 30%.

[0037] 2. This invention utilizes underwater image target detection technology to provide precise locations of underwater damaged areas for cladding spot welding, realizing autonomous underwater damage detection and collaborative planning and control of multiple robotic arms, which can significantly improve the automation level of underwater welding repair.

[0038] 3. This invention proposes a method for detecting and locating repair plates using a combination of multiple internal cameras inside a dry drainage device. This method solves the problem that traditional single cameras have a narrow field of view and cannot completely detect repair plates. By using multi-camera joint calibration and fusion detection, the target positioning error is reduced to less than 5mm and all sides of the repair plate can be completely detected. It can also autonomously plan the welding and repair trajectory of the robotic arm, promoting the development of underwater welding and repair of marine equipment towards automation, intelligence, and digitalization. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the external cover plate device of the underwater robot welding and repair device based on image recognition of the present invention;

[0040] Figure 2 This is a schematic diagram of the internal welding and repair device of the underwater robot welding and repair device based on image recognition of the present invention;

[0041] Figure 3 This is a flowchart of the welding repair method of the underwater robot welding repair device based on image recognition according to the present invention;

[0042] Figure 4 This is a schematic diagram of coordinate system transformation for the welding repair method of the underwater robot welding repair device based on image recognition according to the present invention;

[0043] Wherein, 1 represents the surface to be repaired; 2 represents external robotic arm one; 3 represents the repair plate; 4 represents the underwater damaged area; 5 represents the external image acquisition module; 6 represents the external welding gun; 7 represents external robotic arm two; 8 represents the dry drainage device; 9 represents the built-in image acquisition module; 10 represents the built-in robotic arm; 11 represents the detection image acquisition module; and 12 represents the fillet weld. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0045] Example

[0046] This embodiment describes an underwater robot welding and repair device based on image recognition, such as... Figure 1 and Figure 2 As shown in the figure, this embodiment presents an underwater robot welding and repair device based on image recognition, such as... Figure 1 and Figure 2As shown, it includes an underwater robot, an external cover plate device, a dry drainage device 8, and an internal welding repair device; the external cover plate device, the dry drainage device 8, and the internal welding repair device are all mounted on the underwater robot.

[0047] The external covering device includes an external image acquisition module 5 for acquiring underwater scene image data, an external robotic arm 2 for covering the underwater damaged area 4 with the repair plate 3, and an external robotic arm 7 for spot welding the repair plate 3 locally to initially fix the repair plate 3; the external image acquisition module 5, the external robotic arm 2, and the external robotic arm 7 are respectively set on the outside of the dry drainage device 8; the external image acquisition module 5 is preferably a binocular camera; the external robotic arm 7 has an external welding gun 6 at its end for spot welding;

[0048] The internal welding repair device includes one or more built-in image acquisition modules 9 for acquiring internal image data of the dry drainage device 8, a built-in robotic arm 10 for welding fillet welds 12 on the side of the repair plate 3, and a detection image acquisition module 11 for acquiring image data of the welded fillet welds 12. The built-in image acquisition modules 9 and the built-in robotic arm 10 are respectively installed inside the dry drainage device 8; in this embodiment, the built-in image acquisition module 9 includes four internal cameras, which are respectively fixed at the four corners inside the dry drainage device 8; the detection image acquisition module 11 is located at the end of the built-in robotic arm 10.

[0049] The welding repair method of the underwater robot welding repair device based on image recognition mentioned above, such as Figure 3 As shown, it includes the following steps:

[0050] S1. Based on the external image acquisition module 5, underwater scene image data is acquired, underwater damaged area 4 and the surface 1 to be repaired where underwater damaged area 4 is located are identified, and the repair plate 3 of the same material as the surface 1 to be repaired is covered on the underwater damaged area 4 using the external robotic arm 1 2, and the local spot welding is completed using the external robotic arm 2 7.

[0051] Specifically, step S1 includes the following steps:

[0052] S11. Use the external image acquisition module 5 to acquire underwater scene image data. The external image acquisition module 5 is a stereo camera. Use the YOLOV7 network and the pre-trained target detection model to perform underwater damage detection on two frames of images from the stereo camera at the same time to obtain the underwater damage area 4 and the surface 1 to be repaired where the underwater damage area 4 is located. Use the pre-calibrated stereo camera parameters to determine the position of the underwater damage area 4 relative to the stereo camera coordinate system.

[0053] Binocular camera parameters refer to the intrinsic parameters of each camera in the binocular camera system and the extrinsic parameters between the cameras. The intrinsic parameters include the intrinsic parameter matrix K and the distortion matrix D, and the extrinsic parameters between the cameras include the rotation matrix R and the translation matrix T, respectively, in the following forms:

[0054]

[0055] D = [k1 k2 k3 p1 p2],

[0056]

[0057] Among them, f x f y represents the focal length of the pixel along the X and Y axes, respectively; (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system; k1, k2, and k3 represent radial distortion, respectively; p1 and p2 represent tangential distortion, respectively; r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 These represent the rotation amounts in the rotation matrix R; t x t y t z These represent the translation amounts in the X, Y, and Z directions of the translation matrix T, respectively.

[0058] S12. Using the extrinsic parameters of the binocular camera coordinate system and the underwater robot coordinate system, the position of the underwater damage area 4 is transformed to the underwater robot coordinate system, and the underwater robot is controlled to move so that the underwater damage area 4 is within the travel range of the external robotic arm 1 2 and the external robotic arm 2 7.

[0059] The extrinsic parameters of the binocular camera coordinate system and the underwater robot coordinate system refer to the rotation matrix and translation matrix between the binocular camera coordinate system and the underwater robot coordinate system; the underwater damage area 4 within the travel range of external robotic arm 2 and external robotic arm 7 means that external robotic arm 2 and external robotic arm 7 reach the underwater damage area 4 by adjusting the joint positions.

[0060] S13. Using the external parameters of the binocular camera coordinate system and the coordinate systems of external robotic arm 1-2 and external robotic arm 2-7, the position of the underwater damaged area 4 is transformed to the coordinate systems of external robotic arm 1-2 and external robotic arm 2-7 respectively. The external robotic arm 1-2 is controlled to cover the underwater damaged area 4 with a repair plate 3 of the same material as the surface 1 to be repaired. The external robotic arm 2-7 is used to drive the external welding gun 6 to complete the local spot welding to fix the repair plate 3.

[0061] The extrinsic parameters of the binocular camera coordinate system, the external robotic arm 1-2 coordinate system, and the external robotic arm 2-7 coordinate system refer to the rotation and translation matrices among these three coordinate systems. The repair plate 3, which covers the same material as the surface to be repaired 1, means selecting a repair plate 3 of the same material as the surface to be repaired 1, fixing the repair plate 3 to the end of the external robotic arm 1-2, and controlling the movement of the external robotic arm 1-2 to ensure that the repair plate 3 completely covers the underwater damaged area 4. The local spot welding fixing of the repair plate 3 means controlling the movement of the external robotic arm 2-7 and using the external welding gun 6 fixed to the end of the external robotic arm 2-7 to perform multi-point welding to initially fix the repair plate 3 to the surface to be repaired 1.

[0062] S2. Using the coordinate system of the external image acquisition module 5 and the external parameters of the base coordinate system of the built-in robotic arm 10, the underwater robot is controlled to move, so that the repair plate 3 is within the working space of the dry drainage device 8 and the coordinate system of the built-in robotic arm 10.

[0063] Specifically, the external parameters of the coordinate system of the external image acquisition module 5 and the base coordinate system of the built-in robotic arm 10 refer to the rotation matrix and translation matrix between the coordinate system of the external image acquisition module 5 and the base coordinate system of the built-in robotic arm 10; the repair plate 3 is in the working space of the coordinate system of the dry drainage device 8 and the built-in robotic arm 10, meaning that the repair plate 3 is completely covered inside the dry drainage device 8, and the built-in robotic arm 10 reaches the underwater damage area 4 by adjusting the joint position.

[0064] S3. Control the dry drainage device 8 to sink and adhere to the surface 1 to be repaired, and use high-pressure gas to discharge the moisture inside the dry drainage device 8, creating a dry environment inside the dry drainage device 8.

[0065] Specifically, the sinking of the dry drainage device 8 refers to the control of the dry drainage device 8 drive motor to move the entire dry drainage device 8 downward, so that the bottom rubber material is completely in contact with the surface 1 to be repaired; creating a dry environment refers to the control of the land gas cylinder to supply high-pressure gas to the inside of the dry drainage device 8 through the gas pipe, so that the moisture inside the dry drainage device 8 is discharged through multiple pre-set one-way valves, thereby creating a dry environment inside the dry drainage device 8.

[0066] S4. Based on the built-in image acquisition module 9, the internal image data of the dry drainage device 8 is acquired, the position of the repair board 3 is located, and the position of the repair board 3 is converted to the coordinate system of the built-in robotic arm 10.

[0067] Specifically, locating the position of the repair board 3 means that the built-in image acquisition module 9 consists of four internal cameras, which are fixed at the four corners inside the dry drainage device 8. The four internal cameras are used to acquire image data inside the dry drainage device 8 to perform target detection on the repair board 3, thereby locating the position of the repair board 3.

[0068] Transformation to the base coordinate system of the built-in robotic arm 10 means using the pre-calibrated intrinsic parameters of the internal cameras and the extrinsic parameters of the four internal cameras and the built-in robotic arm 10 to transform the position of the repair plate 3 detected by the internal camera of each dry drainage device 8 to the base coordinate system of the built-in robotic arm 10; the extrinsic parameters of the four internal cameras and the built-in robotic arm 10 refer to the rotation matrix and translation matrix between the coordinate system of each internal camera and the base coordinate system of the built-in robotic arm 10.

[0069] Because a single camera has a narrow field of view, it is difficult to achieve omnidirectional detection of the contact surface between the repair material and the surface to be repaired, thus preventing the use of a built-in robotic arm for automated welding repair. The objective of S4 is to use four internal cameras fixed above the dry drainage device to detect the contact surface between the repair material and the surface to be repaired, and to convert the detection results to the base coordinate system of the built-in robotic arm. The detection results from the four internal cameras are then stitched together in a unified coordinate system. This stitching involves correcting distortion in the images and detection results from each internal camera, sampling three-dimensional points from each internal camera's detection results, merging the sets of three-dimensional points from the four internal cameras, determining the distance between each three-dimensional point and its nearest three-dimensional point, and if the distance exceeds a pre-set threshold, re-detection and sampling are performed, ultimately forming a set of three-dimensional points for all sides of the repair material.

[0070] S5. Based on the position of the repair plate 3, plan the motion trajectory of the built-in robotic arm 10 to complete the welding repair work and form a fillet weld 12 on the side of the repair plate 3; use the detection image acquisition module 11 to collect image data of the fillet weld 12 formed by welding and check the appearance of the fillet weld 12.

[0071] Specifically, step S5 includes the following steps:

[0072] S51. Based on the position of the repair plate 3 in the coordinate system of the built-in robotic arm 10, and the target detection results of the built-in image acquisition module 9 on the repair plate 3, locate all sides of the repair plate 3 as the area to be welded; sort the three-dimensional point set of the side of the repair plate 3 to form a sequential point set with a start point and an end point.

[0073] S52. Based on the length and shape of the sequence point set on the side of the repair plate 3, plan the welding motion trajectory of the built-in robotic arm 10, and move continuously along the sequence point set at the optimal welding speed and angle.

[0074] S53. Use the land-based welding power source to provide appropriate welding voltage, current and duty cycle process parameters, use the land-based wire feeding device to provide wire and wire feeding speed for welding repair work, control the movement of the built-in robotic arm 10, and complete the welding repair work.

[0075] S54. After completing the welding repair work, a fillet weld 12 is formed on the side of the repair plate 3. The built-in robotic arm 10 is controlled to move along the welding trajectory again. The detection image acquisition module 11 fixed on the built-in robotic arm 10 is used to inspect the fillet weld 12 and determine whether the appearance of the weld meets the repair requirements.

[0076] like Figure 4 As shown, the coordinate system is defined as follows: binocular camera coordinate system, underwater robot coordinate system, external robotic arm coordinate system 1, external robotic arm coordinate system 2, internal robotic arm coordinate system, internal camera coordinate system, and detection image acquisition module coordinate system.

[0077] In step S1, the underwater damage location is located in the binocular camera coordinate system and then transformed to the underwater robot coordinate system, the external robotic arm coordinate system 1, and the external robotic arm coordinate system 2, respectively, and a repair plate is applied to the underwater damage area.

[0078] In step S2, the position of the repair plate is converted to the base coordinate system of the built-in robotic arm to ensure that the repair plate is within the repair range of the built-in robotic arm.

[0079] In step S4, the precise position of the repair plate is obtained using the detection results of the repair plate in the internal camera coordinate system and then converted to the base coordinate system of the built-in robotic arm.

[0080] In step S5, welding repair is completed in the coordinate system of the built-in robotic arm, and the appearance and morphology of the obtained fillet weld are inspected in the coordinate system of the detection image acquisition module.

[0081] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A welding repair method for an underwater robot welding repair device based on image recognition, characterized in that: The image recognition-based underwater robot welding repair device includes: an underwater robot, an external covering plate device, a dry drainage device, and an internal welding repair device; the external covering plate device, the dry drainage device, and the internal welding repair device are all mounted on the underwater robot; The external covering plate device includes an external image acquisition module for acquiring underwater scene image data, an external robotic arm one for covering the underwater damaged area with repair plate, and an external robotic arm two for spot welding the repair plate locally to initially fix the repair plate; the external image acquisition module, external robotic arm one and external robotic arm two are respectively set on the outside of the dry drainage device, and the end of the external robotic arm two is provided with an external welding gun for spot welding; The internal welding repair device includes one or more built-in image acquisition modules for acquiring internal image data of the dry drainage device, a built-in robotic arm for welding fillet welds on the side of the repair plate, and a detection image acquisition module for acquiring image data of the welded fillet welds; the built-in image acquisition modules and the built-in robotic arm are respectively installed inside the dry drainage device; the detection image acquisition module is installed at the end of the built-in robotic arm; The welding repair method includes the following steps: S1. Collect underwater scene image data based on the external image acquisition module, identify the underwater damaged area and the surface to be repaired where the underwater damaged area is located, use external robotic arm one to cover the underwater damaged area with a repair plate of the same material as the surface to be repaired, and use external robotic arm two to complete local spot welding fixation. S2. Using the coordinate system of the external image acquisition module and the external parameters of the built-in robotic arm base coordinate system, the underwater robot is controlled to move, so that the repair board is within the working space of the dry drainage device and the built-in robotic arm coordinate system. S3. Control the dry drainage device to sink and adhere to the surface to be repaired, and use high-pressure gas to remove the moisture inside the dry drainage device, creating a dry environment inside the dry drainage device. S4. Collect internal image data of the dry drainage device based on the built-in image acquisition module, locate the position of the repair board, and convert the position of the repair board to the base coordinate system of the built-in robotic arm; S5. Based on the location of the repair plate, plan the motion trajectory of the built-in robotic arm to complete the welding repair work and form a fillet weld on the side of the repair plate; use the detection image acquisition module to collect image data of the fillet weld formed by welding and check the appearance of the fillet weld. In step S2, the external parameters of the coordinate system of the external image acquisition module and the base coordinate system of the built-in robotic arm refer to the rotation matrix and translation matrix between the coordinate system of the external image acquisition module and the base coordinate system of the built-in robotic arm; the repair plate is in the working space of the dry drainage device and the coordinate system of the built-in robotic arm, meaning that the repair plate is completely covered inside the dry drainage device, and the built-in robotic arm reaches the underwater damage area by adjusting the joint position.

2. The welding repair method according to claim 1, characterized in that: The built-in image acquisition module includes four internal cameras, which are fixed at the four corners inside the dry drainage device; the external image acquisition module is a binocular camera.

3. The welding repair method according to claim 1, characterized in that: Step S1 includes the following steps: S11. Underwater scene image data are acquired using an external image acquisition module, which is a stereo camera. Underwater damage detection is performed on two frames of images from the stereo camera at the same time using a YoloV7 network and a pre-trained target detection model to obtain the underwater damage area and the surface to be repaired where the underwater damage area is located. The position of the underwater damage area relative to the stereo camera coordinate system is determined using the pre-calibrated stereo camera parameters. S12. Using the extrinsic parameters of the binocular camera coordinate system and the underwater robot coordinate system, the position of the underwater damage area is transformed to the underwater robot coordinate system, and the movement of the underwater robot is controlled so that the underwater damage area is within the travel range of external robotic arm one and external robotic arm two. S13. Using the external parameters of the binocular camera coordinate system and the coordinate systems of external robotic arm 1 and external robotic arm 2, the position of the underwater damaged area is transformed to the coordinate systems of external robotic arm 1 and external robotic arm 2 respectively. The external robotic arm 1 is controlled to cover the underwater damaged area with a repair plate of the same material as the surface to be repaired. The external robotic arm 2 is used to drive the external welding gun to complete the local spot welding to fix the repair plate.

4. The welding repair method according to claim 3, characterized in that: In step S11, the binocular camera parameters refer to the intrinsic parameters of each camera in the binocular camera system and the extrinsic parameters between the binocular cameras. The intrinsic parameters include the intrinsic parameter matrix K and the distortion matrix D, and the extrinsic parameters between the binocular cameras include the rotation matrix R and the translation matrix T, respectively, in the following forms: ， D=[k1 k2 k3 p1 p2], ； Among them, f x f y represents the focal length of the pixel along the X and Y axes, respectively; (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system; k1, k2, and k3 represent radial distortion, respectively; p1 and p2 represent tangential distortion, respectively; r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 These represent the rotation amounts in the rotation matrix R; t x t y t z These represent the translation amounts in the X, Y, and Z directions of the translation matrix T, respectively.

5. The welding repair method according to claim 3, characterized in that: In step S12, the extrinsic parameters of the binocular camera coordinate system and the underwater robot coordinate system refer to the rotation matrix and translation matrix between the binocular camera coordinate system and the underwater robot coordinate system; the underwater damage area within the travel range of external robotic arm one and external robotic arm two means that external robotic arm one and external robotic arm two reach the underwater damage area by adjusting the joint positions. In step S13, the extrinsic parameters of the binocular camera coordinate system and the coordinate systems of external robotic arm 1 and external robotic arm 2 refer to the rotation and translation matrices between the three coordinate systems: the binocular camera coordinate system, the coordinate system of external robotic arm 1, and the coordinate system of external robotic arm 2. The repair plate covering the surface to be repaired is made by selecting a repair plate of the same material as the surface to be repaired, fixing the repair plate to the end of external robotic arm 1, and controlling the movement of external robotic arm 1 to ensure the repair plate completely covers the underwater damaged area. The partial spot welding fixing of the repair plate refers to controlling the movement of external robotic arm 2 and using an external welding gun fixed to the end of external robotic arm 2 to perform multi-point welding, initially fixing the repair plate to the surface to be repaired.

6. The welding repair method according to claim 1, characterized in that: Step S3, sinking of the dry drainage device, refers to controlling the dry drainage device to move downwards as a whole through the drive motor, so that the bottom rubber material is completely in contact with the surface to be repaired; creating a dry environment refers to controlling the land gas cylinder to supply high-pressure gas to the inside of the dry drainage device through the gas pipe, so that the moisture inside the dry drainage device is discharged through multiple pre-set one-way valves, thereby creating a dry environment inside the dry drainage device.

7. The welding repair method according to claim 1, characterized in that: Step S4, locating the repair board position, means that the built-in image acquisition module consists of four internal cameras, which are fixed at the four corners inside the dry drainage device. The four internal cameras are used to collect image data inside the dry drainage device to perform target detection on the repair board, thereby locating the repair board position. Transforming to the built-in robotic arm base coordinate system means using the pre-calibrated intrinsic parameters of the internal cameras and the extrinsic parameters of the four internal cameras and the built-in robotic arm to transform the position of the repair plate detected by the internal camera of each dry drainage device to the built-in robotic arm base coordinate system; the extrinsic parameters of the four internal cameras and the built-in robotic arm refer to the rotation matrix and translation matrix between the coordinate system of each internal camera and the base coordinate system of the built-in robotic arm.

8. The welding repair method according to claim 1, characterized in that: Step S5 includes the following steps: S51. Based on the position of the repair plate in the coordinate system of the built-in robotic arm and the target detection results of the repair plate by the built-in image acquisition module, locate all sides of the repair plate as the areas to be welded. The three-dimensional point set on the side of the repair board is sorted to form a sequential point set with a start point and an end point; S52. Based on the length and shape of the sequence point set on the side of the repair plate, plan the welding motion trajectory of the built-in robotic arm, and move continuously along the sequence point set at the optimal welding speed and angle. S53. Use the land-based welding power source to provide appropriate welding voltage, current and duty cycle process parameters, use the land-based wire feeding device to provide wire and wire feeding speed for welding repair operations, control the movement of the built-in robotic arm, and complete the welding repair work. S54. After completing the welding repair work, a fillet weld is formed on the side of the repair plate. The built-in robotic arm is controlled to move along the welding trajectory again. The detection image acquisition module fixed on the built-in robotic arm is used to inspect the fillet weld and determine whether the appearance of the weld meets the repair requirements.

Images